Commit Graph

360 Commits

Author SHA1 Message Date
Mateusz Mandera 4917e01ffb push_notifications: Migrate to FCM HTTP v1 API.
The legacy API we use via python-gcm is deprecated and about to be
disabled.

Fixes #29768.
2024-06-17 18:26:26 -07:00
Mateusz Mandera 297141f8c4 push_notifs: Rename parse_gcm_options to parse_fcm_options. 2024-06-17 18:22:59 -07:00
Mateusz Mandera 497be370f5 push_notifs: Rename GCM->FCM in a bunch of comments.
This is called FCM now, even though we're using the legacy API that used
to be called GCM.
2024-06-17 18:22:59 -07:00
Mateusz Mandera 9c10b0d700 push_notifs: Rename has_gcm_credentials to has_fcm_credentials. 2024-06-17 18:22:59 -07:00
Mateusz Mandera 00b8cce50e push_notifs: Rename PushDeviceToken.GCM to FCM. 2024-06-17 18:22:59 -07:00
Mateusz Mandera 4591202032 push_notifs: Remove "canonical" handling in FCM API.
We believe this to already be obsolete and dead code and is about to be
removed with the migration to the FCM HTTP v1 API, where the concept
doesn't exist anymore.
2024-06-17 18:22:59 -07:00
Prakhar Pratyush 55a8bdfbd9 push_notifications: Use is_same_server_message_link lib function.
This commit replaces the local 'is_same_server_message_link'
function used in 'get_mobile_push_content' with the
'is_same_server_message_link' lib function.

The lib function is the same logically but uses urllib instead
of regex for parsing and is backed by tests, hence more robust.
2024-06-17 10:42:32 -07:00
Prakhar Pratyush de0c22592f push_notifications: Compress blockquotes when replying to a message.
To make better use of the limited characters in mobile push
notifications for messages quoting another message, we compress
the blockquotes and "user said" paragraphs to make space for the
actual message.

Fixes #28951.
2024-06-14 17:26:22 -07:00
Alex Vandiver 46a57e37aa push_notifications: Support test notifications from old servers.
4430ab9cbe changed this, assuming that all servers would send
`realm_url` -- however, only servers running that commit do.  Update
to accept either `realm_url` or `realm_uri` payload properties.
2024-06-04 23:07:39 -07:00
Vector73 4430ab9cbe zerver: Replace `realm_uri` with `realm_url` in backend files.
Co-authored-by: Junyao Chen <junyao.chen@socitydao.org>
2024-06-03 10:07:10 -07:00
Vector73 ac4dde24ae realm: Add an alias `realm_url` and deprecate `realm_uri` in the API.
The naming `uri` is deprecated while `url` should be used in order to
satisfy URL standards. For this reason, four endpoints are affected:

* The response content of three endpoints `/server_settings`,
`/register` and `/realm` that contain a field `realm_uri` is
changed to `realm_url`.

* In one of the common fields for all mobile push notifications payloads,
`realm_url` field is now added as an alias to `realm_uri`.

For backwards compatibility, we keep the field `realm_uri` and add
an alias `realm_url`.

Co-authored-by: Junyao Chen <junyao.chen@socitydao.org>
2024-05-08 17:39:15 -07:00
Vector73 8ab526a25a models: Replace realm.uri with realm.url.
In #23380, we are changing all occurrences of uri with url in order to
follow the latest URL standard. Previous PRs #25038 and #25045 has
replaced the occurences of uri that has no direct relation with realm.

This commit changes just the model property, which has no API
compatibility concerns.
2024-05-08 11:12:43 -07:00
Prakhar Pratyush c727b36e9c soft_reactivate: Soft reactivate if group mention has < 12 members.
Earlier, we didn't soft-reactivate users for group mentions
at all because it wasn't easy to calculate group size.

Now, we will soft reactivate if the user group mentions has
less than 12 members.

We don't reactivate all users because a user group can have a
very large size, which can lead to large backlogs in the
deferred-work queue.

Fixes part of #27586.
2024-04-24 17:40:37 -07:00
Prakhar Pratyush 43136abb70 push_notification: Use 'get_mentioned_user_group' function.
This commit updates the code in 'handle_push_notifications'
to use a common lib function 'get_mentioned_user_group'.

The code logically does the same thing in both the places,
hence the change.
2024-04-24 17:40:37 -07:00
Mateusz Mandera 962ab13203 push_notifs: Make push_notifications_enabled more resistant to flapping.
Fixes #28403

Uses redis to remember the last time push notifications were experienced
working. This needs to work across processes, so can't be done just in
memory.
As this is transient data that's fairly harmless to lose and thus
doesn't require the persistence benefits of the database, and we're
keeping a single "row", so don't need an entire new db table, we settle
on using redis instead of postgres. This is also consistent with how we
store other kinds of such transient data.
2024-04-19 10:25:34 -07:00
Alex Vandiver 5654d051f7 worker: Split into separate files.
This makes each worker faster to start up.
2024-04-16 23:00:02 -07:00
Alex Vandiver 436dab0e01 messages: Remove use of @overload in access_message.
f92d43c690 added uses of `@overload` to probide multiple type
signatures for `access_message`, based on the `get_user_message`
parameter.  Unfortunately, mypy does not check the function body
against overload signatures, so it allows type errors to go
undetected.

Replace the overloads with two functions, for one of which also
returns the usermessage.  The third form, of only returning if the
usermessage exists, is not in a high-enough performance endpoint that
a third form is worth maintaining; it uses the usermessage form.
2024-04-12 11:41:27 -07:00
Alex Vandiver 936c2b54cb push_notifications: Use ignore_conflicts, over catching IntegrityError.
The IntegrityError shows up in the database logs, which looks
unnecessarily concerning.  Use `ON CONFLICT IGNORE` to mark this as
expected, especially since the return value is never used.
2024-04-12 11:36:40 -07:00
Anders Kaseorg a82a3eb4d7 ruff: Fix UP033 Use `@functools.cache`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-01 18:32:52 -07:00
Aditya Bajaj a20b454ebb
notifications: Shorten string for redacted content.
Replace the long string for organisations that have notification
body/content disabled (settings.PUSH_NOTIFICATION_REDACT_CONTENT
set to true) with "New message".

This allows more of the limited space on the mobile device screen to
be used for additional messages rather than this verbose content.

Fixes #29152
2024-03-22 09:37:51 -07:00
Alex Vandiver fd5a091b30 messages: Only check the UserMessage row if necessary.
For the common case of not needing to reference the UserMessage row
later, and for being a stream without private history, the UserMessage
row is irrelevant.  Convert `has_user_message` to a thunk, and defer
loading it unless necessary.
2024-03-22 09:30:17 -07:00
Alex Vandiver f92d43c690 messages: Use overloads to only return a user_message if needed. 2024-03-22 09:30:17 -07:00
John Lu a5cf0ec526
refactor: Replace HUDDLE with DIRECT_MESSAGE_GROUP.
Replaced HUDDLE attribute with DIRECT_MESSAGE_GROUP using VS Code search,
part of a general renaming of the object class.

Fixes part of #28640.

Co-authored-by: JohnLu2004 <JohnLu10212004@gmail.com>
2024-03-21 16:39:33 -07:00
Anders Kaseorg 570f3dd447 python: Reformat with Ruff formatter.
https://docs.astral.sh/ruff/formatter/

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-02-29 17:07:16 -08:00
Mateusz Mandera 5672595c2a push_notifs: Gracefully handle exception when server cant push.
The problem was that earlier this was just an uncaught JsonableError,
leading to a full traceback getting spammed to the admins.
The prior commit introduced a clear .code for this error on the bouncer
side, meaning the self-hosted server can now detect that and handle it
nicely, by just logging.error about it and also take the opportunity to
adjust the realm.push_notifications_... flags.
2024-02-07 10:36:33 -08:00
Anders Kaseorg 93198a19ed requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-29 10:41:54 -08:00
Mateusz Mandera 3ec3ac63f2 zilencer: Have server send realm_uuid to remaining bouncer endpoints.
Requests to these endpoint are about a specified user, and therefore
also have a notion of the RemoteRealm for these requests. Until now
these endpoints weren't getting the realm_uuid value, because it wasn't
used - but now it is needed for updating .last_request_datetime on the
RemoteRealm.
2024-01-05 13:09:09 -08:00
Anders Kaseorg c9c819e1d7 models: Extract zerver.models.scheduled_jobs.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Anders Kaseorg 3c11fd9466 models: Move some functions to zerver.lib.display_recipient.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Anders Kaseorg cd96193768 models: Extract zerver.models.realms.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Anders Kaseorg 45bb8d2580 models: Extract zerver.models.users.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Tim Abbott 63e5712837 push_notifications: Improve logging when not contacting bouncer. 2023-12-15 08:15:36 -08:00
Mateusz Mandera c1988a14a7 zilencer: Return can_push info at the push/notify endpoint.
This provides the remote server this information to refresh it on its
Realm attributes whenever it sends a push notification.

Fixes #27483.
2023-12-12 08:06:12 -08:00
Mateusz Mandera 3bcfb9c005 push_notifs: Order device args to send_notifications_to_bouncer by id.
This ensures determinism in these tests doing mock_send.assert_called
with - avoids producing test flakes due to a different order of
retrieval of these objects from the database.
2023-12-11 18:36:23 -08:00
Mateusz Mandera 2916a601b3 push_notifs: Don't send request to bouncer if no devices found. 2023-12-11 18:36:23 -08:00
Mateusz Mandera b82ea179ac zilencer: Have push/notify endpoint return registrations to delete.
- The server sends the list of registrations it believes to have with
  the bouncer.
- The bouncer includes in the response the registrations that it doesn't
  actually have and therefore the server should delete.
2023-12-11 18:36:23 -08:00
Prakhar Pratyush c1daabd3c0 remote_server: Rename to 'send_server_data_to_push_bouncer'.
This commit renames 'send_analytics_to_push_bouncer'
to 'send_server_data_to_push_bouncer'.
2023-12-11 14:07:39 -08:00
Prakhar Pratyush d763fae9d0 remote_server: Eliminate separate realms-only code path.
Given that most of the use cases for realms-only code path would
really like to upload audit logs too, and the others would likely
produce a better user experience if they upoaded audit logs, we
should just have a single main code path here i.e.
'send_analytics_to_push_bouncer'.

We still only upload usage statistics according to documented
option, and only from the analytics cron job.

The error handling takes place in 'send_analytics_to_push_bouncer'
itself.
2023-12-11 14:07:39 -08:00
Aman Agrawal c5bfdaaeeb push_notifications: Return after receiving an error. 2023-12-08 12:25:46 -08:00
Lauryn Menard a018f2611b push-notifications: Update payload for realm name and event string.
Adds `user.realm.string_id` as the realm name to the base payload
for notifications. Uses this realm name in the body of the alert
in the `apns_data`.

Changes the event string from "test-by-device-token" to "test".

Fixes #28075.
2023-12-07 14:14:37 -08:00
Tim Abbott 19ac558d5f push_notifications: Fix handling of 500s from bouncer.
The comments explain the context, but we shouldn't mark our access to
push notifications as disabled incorrectly here.
2023-12-07 13:45:57 -08:00
Sahil Batra 6c3fbcc383 push_notifications: Do not include details of inaccessible users.
This commit adds code to include original name, email and avatar
for inaccessible users which can happen when a user sends message
to an unsubscribed stream.
2023-12-06 00:09:53 -08:00
Tim Abbott f6c7eaf1e5 models: Add push_notifications_enabled & corresponding end_timestamp.
Add two fields to Realm model:
*push_notifications_enabled
*push_notifications_enabled_end_timestamp

Co-authored-by: Prakhar Pratyush <prakhar@zulip.com>
2023-12-04 07:08:39 -08:00
Mateusz Mandera 7f33d6f0ea zilencer: Tie RemotePushDeviceToken to RemoteRealm at registration.
This consists of the following pieces:
1. Makes servers using the bouncer send realm_uuid in requests for token
   registration. (Sidenote: realm_uuid is already sent in the "send
   notification" codepath as of
   48db4bf854)
2. This allows the bouncer to tie RemotePushDeviceToken to the
   RemoteRealm with matching realm_uuid at registration time.
3. Introduce handling of some potential weird edge cases around the
   realm_uuid and RemoteRealm objects in get_remote_realm_helper.
2023-12-03 09:51:45 -08:00
Tim Abbott 4ef6b7cc44 notifications: Rename apns_enabled and gcm_enabled.
The new names are a lot clearer when thinking about self-hosted
systems that might indirectly use these via a bouncer.
2023-11-29 15:06:46 -08:00
Tim Abbott b71c5746ab notifications: Rename push_notifications_enabled for clarity.
This doesn't actually check if push notifications are working, just
whether there is configuration for them.
2023-11-29 15:06:46 -08:00
Prakhar Pratyush 6f3b25d749 push_notification: Fix bad rendering of math formulas.
Earlier, for the push notifications having latex math
like "$$1 \oplus 0 = 1$$, the notification had the math
included multiple times.

This commit fixes the incorrect behavior by replacing
the KaTeX with the raw LaTeX source.

Fixes part of #25289.
2023-11-26 23:30:24 -08:00
Greg Price 528a76a419 push_notifs: Set APNs topic individually on each notification.
This makes it possible to send notifications to more than one app ID
from the same server: for example, the main Zulip mobile app and the
new Flutter-based app, which has a separate app ID for use through its
beta period so that it can be installed alongside the existing app.
2023-11-16 12:03:30 -08:00
Greg Price ff32d51700 push_notifs: Support APNs token auth, as well as cert auth.
This will make it possible to send notifications to multiple
distinct app IDs over the same connection.
2023-11-16 12:03:30 -08:00
Mateusz Mandera 48db4bf854 counts: Add new mobile_pushes RemoteRealmCount stats.
This requires a bit of complexity to avoid a name collision in
COUNT_STATS with the RemoteInstallationCount stats with the same name.
2023-11-10 16:09:11 -08:00