Commit Graph

7934 Commits

Author SHA1 Message Date
Mateusz Mandera 4917e01ffb push_notifications: Migrate to FCM HTTP v1 API.
The legacy API we use via python-gcm is deprecated and about to be
disabled.

Fixes #29768.
2024-06-17 18:26:26 -07:00
Mateusz Mandera 496aa2a0f1 test_push_notifs: Remove gcm_client mock in test_connection_error.
This wasn't doing anything. The test simulates failure to connect to the
bouncer, so the codepaths inside the bouncer don't even get triggered.
2024-06-17 18:23:00 -07:00
Mateusz Mandera 2628af9ef4 test_push_notifs: Rename GCM->FCM in some var and func names. 2024-06-17 18:22:59 -07:00
Mateusz Mandera 297141f8c4 push_notifs: Rename parse_gcm_options to parse_fcm_options. 2024-06-17 18:22:59 -07:00
Mateusz Mandera 00b8cce50e push_notifs: Rename PushDeviceToken.GCM to FCM. 2024-06-17 18:22:59 -07:00
Mateusz Mandera 4591202032 push_notifs: Remove "canonical" handling in FCM API.
We believe this to already be obsolete and dead code and is about to be
removed with the migration to the FCM HTTP v1 API, where the concept
doesn't exist anymore.
2024-06-17 18:22:59 -07:00
nimish 81ea09be19 search: Add is:followed filter.
Create the is:followed search operator.
Fetch all messages that are from followed topics
using exists.
Update API documentation and changelog.

Co-authored-by: Kenneth Rodrigues <kenneth.nrk123@gmail.com>

Fixes #27309.
2024-06-17 18:22:50 -07:00
Prakhar Pratyush 484befe9ce url_decoding: Add 'is_same_server_message_link' function.
This prep commit adds a lib function 'is_same_server_message_link'.

This will be currently used while compressing quote and reply
in push notifications and later can be used at other places.
2024-06-17 10:42:32 -07:00
Prakhar Pratyush 5fd676c4e0 test_fixtures: Add 'message_link_test_cases' test fixture.
This is a prep commit to extract out the test cases
for 'test_is_same_server_message_link' in hash_util.test.js
as a test fixture.

This will help in reusing the same test cases in the node and
backend tests.

Also, it will help in making sure that the logic for
'is_same_server_message_link' is in sync, in both the
web client and server code.
2024-06-17 10:42:32 -07:00
Mateusz Mandera 7ff1227984 presence: Improve INSERT conflict handling in do_update_user_presence.
This was discussed in the review of #29999:
https://github.com/zulip/zulip/pull/29999#discussion_r1620818568

The previous way of handling wasn't entirely correct, as unnecessary
events were omitted, with a bad guarantee of even being in the correct
order.

This is an improvement as now the function detects that it ended up
doing nothing and can skip sending an event.

The race condition is hard to make up in an automated test, but we can
hackily simulate it by injecting a side_effect which will create a
conflicting UserPresence row when the function requests a cursor. Aside
of that, the actual race was simulated in manual testing to verify the
expected behavior.
2024-06-16 11:57:47 -07:00
Sahil Batra ff8f797c5c test_event_system: Fix test_queries test.
Before this commit, 3 DB queries were actually needed for "realm"
event type but the test mentioned 1 because, the related stream
setting fields were already fetched when testing the case for all
event types above it.

So, when testing the query count for only "realm" event type the
cached realm object was used and no extra queries were made for
the stream settings.

This commit updates the test to refetch the realm before testing
each event type separately, so that we test the actual query count.
2024-06-16 11:55:28 -07:00
Sahil Batra e23af20079 realm: Prefetch group settings only for "/register" response.
This commit updates code to prefetch realm group settings like
"can_create_public_channel_group" only when computing settings
for "/register" response by refetching the realm object with
select_related instead of fetching those settings in UserProfile
query.

This change is done because we do not need to prefetch these
settings for every UserProfile object and for most of the cases
where these settings are actually accessed, we can afford extra
query like when checking permission to create streams. But we
cannot afford one query extra for each setting when computing
these settings for "/register" response, so we re-fetch the
realm object with select_related leading to only one extra
query.

The query count changes in tests are -
- Query count increases by 1 when calling fetch_initial_state_data
for computing can_create_public_streams because Realm object from
UserProfile does not have prefetched setting fields.

- Query count increases by one in test_subs where streams are
created which is as expected due to the setting not being prefetched.

- Query count increases by 2 in tests in test_home.py where one
query is to refetch the realm object and one for computing
can_create_public_streams as mentioned above.
2024-06-16 11:55:28 -07:00
Sahil Batra dea62a3fd9 events: Pass realm to fetch_initial_state_data always.
This commit makes passing realm mandatory to fetch_initial_state_data.

This is a prep commit for refetching the realm object with
select_related for group setting fields so that extra queries
can be avoided when computing "/registe" response.
2024-06-16 11:55:28 -07:00
Prakhar Pratyush de0c22592f push_notifications: Compress blockquotes when replying to a message.
To make better use of the limited characters in mobile push
notifications for messages quoting another message, we compress
the blockquotes and "user said" paragraphs to make space for the
actual message.

Fixes #28951.
2024-06-14 17:26:22 -07:00
Kenneth Rodrigues 07d0854ed3 storage: Migrate to typed_endpoint. 2024-06-14 11:24:36 -07:00
Kenneth Rodrigues 6364010df5 typing: Migrate to typed_endpoint. 2024-06-14 10:58:35 -07:00
Gaurav Pandey 1c5007461a topic: Add resolve topic undo grace period.
Currently we send a notification to the topic if it has been resolved
or unresolved even if there is an immediate event of resolving and
then unresolving or vice-versa. This adds a setting of
RESOLVE_TOPIC_UNDO_GRACE_PERIOD_SECONDS under which if a topic has
been unresolved after being resolved immediately and the last message
was the notification of resolving, then delete the last message and
don't send a new notification and vice-versa.

We use the new message.type field to precisely identify relevant
messages.

Fixes #19181.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2024-06-10 13:34:23 -07:00
Sahil Batra 222995b1be settings: Remove create_public_stream_policy.
This commit removes create_public_stream_policy setting
since public channel creation permissions are now handled
by group-based setting.

We still pass "realm_create_public_stream_policy" in
"/register" response though for older clients with its
value being set depending on the value of group based
setting. If we cannot set its value to an appropriate
enum corresponding to the group setting, then we set
it to "Members only" considering that server will not
allow the users without permissions to create public
channels but the client can make sure that UI is
available to the users who have permission.
2024-06-10 12:24:45 -07:00
Sahil Batra 9365604c03 streams: Check permission to create streams based on group setting. 2024-06-10 12:24:45 -07:00
Sahil Batra 7655c69d89 realm: Set default for can_create_public_channel_group based on org type. 2024-06-10 12:24:45 -07:00
Sahil Batra 3bda17949e realm: Add new group setting for who can create public streams. 2024-06-10 12:24:44 -07:00
Vector73 faa06497ed api_docs: Document "/invites/{invite_id}/resend" endpoint. 2024-06-10 11:48:39 -07:00
Vector73 d60150d39d api_docs: Document "/users/me/presence" endpoint.
Co-authored-by: Tim Abbott <tabbott@zulip.com>
Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2024-06-07 14:28:54 -07:00
Mateusz Mandera f0ec4f5732 presence: Return last_update_id=-1 on zephyr mirror realm.
This was a bug, where in the realm.presence_disabled (synonymous to
being a zephyr mirror realm) case we would return None. We have decided
on the convention of using only integers here, and -1 representing lack
of data.
2024-06-07 13:57:40 -07:00
Mateusz Mandera 512f4d1476 presence: Backend implementation of the last_update_id API.
This builds on top of 016880f54d which
maintains correct .last_update_id for UserPresence objects; now we add
the related API changes to utilize it.
2024-06-06 17:25:54 -07:00
Kenneth Rodrigues 428ca713d0 users: Migrate to typed_endpoint. 2024-06-06 16:50:56 -07:00
Alex Vandiver 04450389e1 push_notifications: Provide a stable ordering.
Otherwise, this can cause test failures based on internals of the database.
2024-06-05 11:48:27 -07:00
Prakhar Pratyush 22f3aebb33 compose: Show a one-time banner for jump to sent message conversation.
We immediately navigate the user to the conversation they just
sent a message to if they are not already in the appropriate
conversation view.

This commit adds a first-time banner to explain the same.

Fixes #29575.
2024-06-05 09:31:17 -07:00
Sahil Batra c8b8b836fd test_events: Update code to test for only allowed system groups.
This commit updates do_set_realm_permission_group_setting_test
to test for only allowed system groups.
2024-06-04 09:55:27 -07:00
Alex Vandiver 7792b12f05 test_push_notifications: Fix a typo. 2024-06-03 12:35:35 -07:00
Alex Vandiver 6c17cca208 zilencer: Drop unwanted data that old servers might still send. 2024-06-03 12:35:35 -07:00
Alex Vandiver 09e9c75ec6 analytics: Remove `active_users` and `active_users_log` metrics.
Both of these are inaccurate, not currently used anywhere, and have
been superseded by the `active_users_audit` metric.
2024-06-03 12:35:35 -07:00
Sahil Batra d3ea6520dc user_groups: Add server level setting disallow anonymous groups for settings.
This commit adds a server level setting which controls whether the setting
can be set to anonymous user groups. We only allow it in the tests for
now because the UI can only handle named user groups.
2024-06-03 09:45:26 -07:00
Sahil Batra 6711f0af09 message_delete: Fix event sent for updating first_message_id.
This commit fixes the event sent for updating first_message_id
when a message is deleted, to include the name field as it is
required for all "stream/update" events.

This commit also adds a test in test_events for the case when
first_message_id of a stream is updated on deleting a message.
2024-06-03 08:52:11 -07:00
Mateusz Mandera b1d50b511c presence: Handle PresenceSequence in the export/import system. 2024-06-02 22:08:28 -07:00
Aman Agrawal bb6e6ecaa5 hello: Redesign landing page.
Co-authored-by: Vlad Korobov <terpimost@gmail.com>
Co-authored-by: Alya Abbott <alya@zulip.com>
2024-06-02 21:45:37 -07:00
Mateusz Mandera 355f05ffbc import_realm: Import message.edit_history correctly.
Fixes #26369.

There are two important fixes to make to the dicts in edit_history:
1. Update the user_id so that it points to the imported sender.
2. Apply fix_message_rendered_content to the prev_rendered_content data
to fix up mentions and other such syntax.
2024-06-02 21:10:50 -07:00
roanster007 384a43c79c narrow: Fix server error of operand of "id" operator at large values.
Previously, when the operand of id operator was more than
2147483647, it was raising server error. This is because the
maximum permissible PostgreSQL integers value is 2147483647.

This is fixed by raising a BadNarrowOperatorError in case the
id operand is larger than 2147483647.
2024-06-01 22:38:18 -07:00
Anders Kaseorg e53e3af0f6 codespell: Fix spelling mistakes caught by codespell.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-05-31 14:32:33 -07:00
Prakhar Pratyush f9fca5b469
remove_single_newlines: Fix regex to parse list syntax correctly.
Earlier, the 'remove_single_newlines' function wasn't working
correctly when '\n' was followed by asterix, hyphen, or number.
Specifically, they were not added as a syntax for ordered or
unordered list in markdown.

For example, see the workaround fix in 330439a83b.

This commit updates the function to replace '\n' with ' ', when
"\n" is not preceded by "\n" and not followed by:
* Another newline (\n)
* A hyphen followed by a space
* An asterisk followed by a space
* A number followed by a dot and a space

We won't have to do fixes like 330439a83b in the future.
2024-05-30 11:32:52 -07:00
Prakhar Pratyush 4deecfa58d zulip_updates: Fix assertion error while sending update messages.
Earlier, for new realm with zulip_update_announcements_stream
set to None, an assertion error was raised in
'is_group_direct_message_sent_to_admins_within_days' because
no 'None' to new level change took place for such realm.

A new realm is on the latest level and a default stream is set.

This commit updates the logic to simply skip sending
update messages as the stream was manually set to None
in such cases.
2024-05-30 10:52:32 -07:00
Sahil Batra aa84080ad6 tests: Add a helper function to create anonymous groups.
This commit adds a new helper function to create or update
a UserGroup object for a setting. We could have used existing
update_or_create_user_group_for_setting but that also validates
user IDs and subgroup IDs which we can skip in tests.
2024-05-28 07:24:07 -07:00
Sahil Batra 81765a1e83 user_groups: Fix audit log data when changing group settings.
This commit fixes the code store correct old value in audit
log data when changing can_mention_group setting from a
anonymous group to another anonymous group. The bug was
because the old value was being computed after updating
the UserGroup object with new members and subgroups and
is fixed by computing the old value for all the cases
and passing it to do_change_user_group_permission_setting.
2024-05-28 07:24:07 -07:00
Vector73 93262e03ad emails: Replace occurrences of uri with url in email templates.
This commit replaces occurrences of realm_uri with realm_url in email templates
and other related backend files.

Co-authored-by: Junyao Chen <junyao.chen@socitydao.org>
2024-05-24 11:18:35 -07:00
Alex Vandiver 5c2fd1de5a docs: Update Django links to our current version. 2024-05-24 10:18:37 -07:00
Prakhar Pratyush 0ce79c8c50 message_edit: Update do_update_message codepath to send event on commit.
Earlier, we were using 'send_event' in 'do_update_message'
which can lead to a situation where we enqueue events but
the transaction fails at a later stage.

Events should not be sent until we know we're not rolling back.
2024-05-24 10:17:35 -07:00
Alex Vandiver e5a0b3b3c5 zilencer: Disambiguate no MX records from the domain not existing.
This switches to dnspython, since it offers the higher-level
`resolve_name` method to look up both A and AAAA records, and set
timeouts.
2024-05-24 09:34:31 -07:00
Alex Vandiver 9d66a8eb73 zilencer: Give a better error message for foo@example.com emails. 2024-05-24 09:34:31 -07:00
Vector73 62dfd93a83 api: Add "users/<int:user_id>/status" endpoint.
The documentation Creates a shared UserStatus schema that's used for
the return value of this new endpoint and for the existing user_status
objects returned by the register queue endpoint.

Co-authored-by: Suyash Vardhan Mathur <suyash.mathur@research.iiit.ac.in>

Fixes #19079.
2024-05-22 18:07:22 -07:00
Vector73 d6672c57ff user_status: Rename `get_user_status_dict` function.
Renames `get_user_status_dict` function to `get_all_users_status_dict`
to distinguish it from new `get_user_status` function.
2024-05-22 18:07:22 -07:00