Commit Graph

7974 Commits

Author SHA1 Message Date
Anders Kaseorg f4e7a11c35 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-11-15 15:27:54 -08:00
Anders Kaseorg a688e753de test_helpers: Fix logging in cursor_executemany mock.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-11-15 15:27:54 -08:00
Anders Kaseorg 2c9c105bbb db: Check connection type in wrapper_execute.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-11-15 15:27:54 -08:00
Alex Vandiver 091e2f177b linkifiers: Match JS implementation to server implementation.
Since the server-side implementation no longer uses look-ahead
or (more importantly) look-behind, it is possible to exactly implement
in Javascript.  This removes a common class which would prevent local
echo.

This requires reworking the topic linking algorithm, to march the
server's as well.  The tests and behaviour are adjusted in so doing --
previously, the JS implementation would have linked `#foo` with a
`foo` regex on the linkifier, but the server implementation would not
have.
2023-11-14 20:43:39 -08:00
Alex Vandiver 70b20e9d2b markdown: Use \p{White_Space} equivalent for linkifier boundaries.
We do not use \p{White_Space} itself because re2 does not support it.
2023-11-14 20:43:39 -08:00
Alex Vandiver 549dd8a4c4 markdown: Expand prepare_linkifier_pattern to make it more legible. 2023-11-14 20:43:39 -08:00
Alex Vandiver 1f3a53db2d cache: Drop the huddle cache.
This cache was only used in one place, which is infrequently
called (only when sending messages, or searching explicitly for a list
of users) and the overhead of maintaining the cache is not worth
trying to avoid the well-indexed lookup of the huddle.
2023-11-13 10:01:29 -08:00
Sahil Batra 8f79cec51a users: Pass bogus data for inaccessible users.
We now pass bogus data for inaccessible users when sending
the users data in "realm_users" field of "register" response
or when using endpoints like "GET /users" to get data of
all the users in realm.

We would add a client capability field in future commits
such that new clients would receive data only for accessible
users and they can form the bogus data by themselves.
2023-11-13 08:04:45 -08:00
Sahil Batra c82bb3ec76 models: Add can_access_all_users_group setting.
This commit adds new setting for controlling who can access
all users in the realm which would have "Everyone" and
"Members only" option.

Fixes part of #10970.
2023-11-13 08:04:45 -08:00
Sahil Batra f20d427a4d users: Remove unnecessary comment.
This comment can be removed as the code was updated in 2ab71fb67d.
2023-11-13 08:04:45 -08:00
Mateusz Mandera 48db4bf854 counts: Add new mobile_pushes RemoteRealmCount stats.
This requires a bit of complexity to avoid a name collision in
COUNT_STATS with the RemoteInstallationCount stats with the same name.
2023-11-10 16:09:11 -08:00
Mateusz Mandera 8a6d5b4997 counts: Add new Add new mobile_pushes_sent::day LoggingCountStat.
This is a CountStat for tracking how many mobile notifications the
server requested.
1. On a self-hosted server, that means requesting from the push bouncer.
2. On a server that's its own push bouncer, that's just the number
   directly sent.

This number has room for inaccuracy due to incrementing by the number of
user devices on a self-hosted server, as it doesn't account for errors
that may occur in the GCM/APNs low-level sending codepaths on the bouncer.

Also tests that a server that's its own push bouncer correctly
increments its mobile_pushes_sent::day CountStat, by basing it on the
values returned from the send_apple/android_push_notification functions
which tell us the actual number of successfully sent notifications.

Since the return values of send_..._push_notification are now
used in those codepaths, we need to tweak our mocks in some unrelated
tests to set up some return value to avoid errors.
2023-11-10 16:09:11 -08:00
Prakhar Pratyush c597de6a1d topic_mentions: Rename wildcard_mentioned to stream_wildcard_mentioned.
Rename the existing 'wildcard_mentioned' flag to
'stream_wildcard_mentioned'.

The 'wildcard_mentioned' flag is deprecated and exists for
backwards compatibility.

We have two separate flags for stream and topic wildcard mentions,
i.e., 'stream_wildcard_mentioned' and 'topic_wildcard_mentioned',
respectively.

* stream wildcard mentions: `@all`, `@everyone`, and `@stream`
* topic wildcard mentions: `@topic`

The `wildcard_mentioned` flag is included in the events and
API response if either `stream_wildcard_mentioned` or
`topic_wildcard_mentioned` is set.
2023-11-10 11:06:26 -08:00
Mateusz Mandera 76e0511481 zilencer: Add new model RemoteRealm and send the data to the bouncer.
Add the new model for recording basic information about Realms on remote
server, to go with the other analytics data. Also adds necessary changes
to the bouncer endpoint and the send_analytics_to_push_bouncer()
function to submit such Realm information.
2023-11-08 15:54:22 -08:00
David Rosa 0c4ef58acd help: Document new help menu.
Updates help center docs to reflect the new dedicated help menu.

- Adds `keyboard_tip` to document the `?` keyboard shortcut.

Fixes #27384.
2023-11-08 14:17:13 -08:00
Alex Vandiver acb4e9e783 cache_helpers: Show item and DB query counts. 2023-11-08 11:00:00 -08:00
roanster007 dc492867af user_mention: Fix mentions of deactivated users.
Previously, when a deactivated user was mentioned, he wasn't
rendered as a Pill. This is because the dataset for validating mentions
only included active users, which is fixed by removing that filter.

To allow only silent mentions of them, an extra is_active property
added to FullNameInfo class, which is populated from the query,
which tells if user is deactivated. This is used to convert any
mentions of them to silent mentions in the backend markdown.

Fixes #26857
2023-11-08 09:48:31 -08:00
Sahil Batra 617d2d509c users: Update format_user_row to return a TypedDict.
This commit updates format_user_row to return a TypedDict.

This commit is a prep commit for feature of restricting user
access such that code can be easy to read and understand when
we add that feature.
2023-11-08 09:22:26 -08:00
Sahil Batra 2ab71fb67d users: Update user_profile_to_user_row to return a TypedDict.
This commit updates user_profile_to_user_row to return a TypedDict
and also updates the return type of get_realm_user_dicts to be a
TypedDict.

This commit is a prep commit for feature of restricting user
access such that code can be easy to read and understand when
we add that feature.
2023-11-08 09:22:26 -08:00
Sahil Batra deb8431a47 users: Rename get_raw_user_data to get_users_for_api.
This is a prep commit for adding feature of restricting
user access to guests such that we can keep the code
easy to read and understand when that feature is added.
2023-11-08 09:22:26 -08:00
Greg Price 7a0ebc0d18 push_notifs: Log an error if ios_app_id missing on any APNs token. 2023-11-07 16:19:42 -08:00
Prakhar Pratyush 94679d590f email_notification: Include prior message context only when mentioned.
Earlier, email message notifications included prior messages sent
to the same topic for context. This is more confusing than helpful
for messages that the user is likely to have received notifications
for all the prior messages in the conversation already (or read them
in the Zulip UI).

Now, we include prior context only when the user is mentioned via
personal, group, stream or topic wildcard mention.

Fixes #27479.
2023-11-07 11:44:38 -08:00
Prakhar Pratyush e6e156709a typing_notifications: Don't notify long_term_idle subscribers.
The event for stream typing notifications is no longer sent
to the long_term_idle subscribers of the stream.

This helps to reduce the tornado's work of parsing super-long
JSON-encoded lists of user IDs in large streams. Now the lists
are shorter.
2023-11-07 09:30:27 -08:00
Aman Agrawal be6f467f42 home: Add `sponsorship_pending` page_param.
This will be used in gear menu to inform admin of their
sponsorship application status.

This includes some additional tweaks for when to show
billing and plans to users.
2023-11-06 14:55:14 -08:00
David Rosa 56f725e39b help: Document ability to drag-and-drop anywhere to upload a file.
- Replaces the "Via Markdown" tab with "Via drag-and-drop", and
modifies the instructions to explain that you can drag and drop
anywhere in the app, whether or not the compose box is open.
- Adds "Via paste" tab for the copy-pasting instructions.

Fixes #26894.
2023-11-04 12:49:54 -07:00
roanster007 ac084ddb0e mention_dataset: Add query to mention cross realm bots as pill.
Previously, cross realm bots were not displayed as mention Pills.

This is because, the data set for validating mentions considers
only the realm id which is None in case of cross realm bots.

Hence, adding an or Q object to it, to also check if
the email is a part of the cross realm bots email, in case the
realm id returns None.

Fixes #26913
2023-11-03 12:29:20 -07:00
David Rosa c486ae37d3 help: Document option to paste URL to create a named link.
- Adds a "Via paste" tab to document the new functionality.

Fixes #26892.
2023-11-02 12:44:33 -07:00
Prakhar Pratyush b0ef76bf27 topic_mentions: Set 'topic_wildcard_mentioned' flag on @topic mention.
Earlier, the 'wildcard_mentioned' flag was set for both the
stream and topic wildcard mentions.

Now, the 'topic_wildcard_mentioned' flag is set for topic
wildcard mentions, and the 'wildcard_mentioned' flag is set for
stream wildcard mentions.

We will rename the 'wildcard_mentioned' flag to
'stream_wildcard_mentioned' in a later commit.
2023-11-02 09:25:51 -07:00
Prakhar Pratyush c37871ac3a user_message: Rename unused flags and create an index.
This commit renames the two unused and historical bits of the
'fields' bitfield of the 'UserMessage' and 'ArchivedUserMessage'
tables.

* 'summarize_in_home' to 'topic_wildcard_mentioned'
* 'summarize_in_stream' to 'group_mentioned'

The 'group_mentioned' flag doesn't affect the feature,
but completing the work here helps to save future migration
and indexing efforts on the UserMessage table, as we plan to
use this flag in the future for group mentions.

The unused bits may have old data; we'll clear that in
a separate commit.

It creates the 'zerver_usermessage_any_mentioned_message_id'
index concurrently.
2023-11-02 09:25:51 -07:00
Sahil Batra bb15b2d708 users: Send "update" events when deactivating or reactivating users.
We now send "realm_user/update" (and "realm_bot/update" for bots)
events with "is_active" field when deactivating and reactivating
users, including bots.

We would want to use "remove" event for a user losing access
to another user for #10970, so it is better to use "update"
event for deactivation as we only update "is_active" field
in the user objects and the clients still have the data for
deactivated users.

Previously, we used to send "add" event for reactivation along
with complete user objects, but clients should have the data
for deactivated users as well, so an "update" event is enough
like we do when deactivating users.
2023-11-02 08:31:12 -07:00
Mateusz Mandera 6819ecee92 zilencer: Add new LoggingCountStat mobile_pushes_forwarded.
This one counts actual successful deliveries.
2023-11-01 17:26:10 -07:00
Sahil Batra 47c8e369cf register: Pass the configuration objects for group settings in response.
This commit adds code to pass configuration objects for group
permission settings in register response to clients such that
we do need to duplicate that data in clients and can avoid
future bugs due to inconsistency.

The "server_supported_permission_settings" field is included
in the response if "realm" is present in "fetch_event_types",
as this is what we do for other server-related fields.
2023-11-01 10:42:56 -07:00
Sahil Batra e458b73a01 user_groups: Move constants for system group names to a new class.
This commit moves constants for system group names to a new
"SystemGroups" class so that we can use these group names
in multiple classes in models.py without worrying about the
order of defining them.
2023-11-01 10:42:56 -07:00
Sahil Batra 2c43c2a4db user_groups: Pass config object to access_user_group_for_setting.
We now pass the complete configuration object for a setting to
access_user_group_for_setting instead of passing the configuration
object's fields as different variables.
2023-11-01 10:42:56 -07:00
Sahil Batra 08d41f9eb3 settings: Rename permissions_configuration variable.
This commit renames permissions_configuration variable to
permission_configuration since the object contains config for
a single permission setting and thus permission_configuration
seems like a better name.
2023-11-01 10:42:56 -07:00
Sahil Batra 9a6cf82adc streams: Fix sending stream-related events to guests.
Previous behavior-
- Guest did not receive stream creation events for new
web-public streams.
- Guest did not receive peer_add and peer_remove events
for web-public and subscribed public streams.

This commit fixes the behavior to be -
- Guests now receive stream creation events for new
web-public streams.
- Guest now receive peer_add and peer_remove events for
web-public and subscribed public streams.
2023-10-31 10:54:21 -07:00
Chandan Yadav 951e31a154 user_status: Remove `status` field from `UserInfoDict` type. 2023-10-30 11:10:47 -07:00
Hemant Umre d27a50852e onboarding: Internationalize the introductory messages in streams.
We use `Realm.default_language` value, which is set by selecting
the 'Organization language', to internationalize the introductory
messages of the initial streams.

Fixes #25729.
2023-10-26 16:27:35 -07:00
Hemant Umre ac1f711fef registration: Set the organization language at creation time.
In this commit, we add a new dropdown 'Organization language' on
the `/new` and `/realm/register` pages. This dropdown allows setting
the language of the organization during its creation. This allows
messages from Welcome Bot and introductory messages in streams to be
internationalized.

Fixes a part of #25729.
2023-10-26 16:27:35 -07:00
Lauryn Menard 02d6b3e16d onboarding-emails: Add new onboarding email for organization creator.
Adds a new onboarding email `onboarding_team_to_zulip` for the user
who created the new Zulip organization.

Co-authored by: Alya Abbott <alya@zulip.com>
2023-10-25 13:51:03 -07:00
Mateusz Mandera bb63aca8bb send_analytics_to_push_bouncer: Fix a stale comment.
The comment has drifted away from where it should be placed within the
code and also talks about RealmCounts specifically, while we have other
object types that this equally applies to.
2023-10-25 11:09:49 -07:00
Mateusz Mandera 986f2fd962 send_analytics_to_remote_server: Rename to ..._to_push_bouncer.
The former name is kind of misleading - this function is for the remote
server to send analytics to the push bouncer. Under our usual
terminology, a "remote server" is a self-hosted Zulip server. So data is
sent FROM not TO a remote server.
2023-10-25 11:09:49 -07:00
David Rosa 447b853ef1 help: Rename finding-a-topic-to-read -> finding-a-conversation-to-read
- Renames filenames to match page title.

Fixes part of #27191.
2023-10-23 16:52:41 -07:00
David Rosa d32a0601cd help: Rename help/reading-topics > help/reading-conversations.
- Renames filenames to match page title.

Fixes part of #27191.
2023-10-23 16:52:41 -07:00
ecxtacy ad63324dce help: Improve user stream management docs.
Documents the procedure to subscribe / unsubscribe a user via their
profile and general stream settings.

Both methods are separated into tabs in the documentation.

Fixes #26902
2023-10-23 14:17:21 -07:00
Esther Anierobi e957decd12
email: Drop support for SEND_MISSED_MESSAGE_EMAILS_AS_USER. (#27223)
Originally, this was how the notification emails worked, but that was changed
in 797a7ef97b, with this old behavior 
available as an option.

The footer and from address of emails that are sent when this
setting is set to True are confusing, especially when more people
are involved in a stream and since we have changed the way we send
emails, it should be removed. It’s also not widely used.

Fixes #26609.
2023-10-20 16:38:43 -07:00
Sahil Batra 760d793d6b settings: Rename "default" views to "home" views in settings and docs.
This commit renames "default" views to "home" views in the setting
labels, keyboard shortcuts list, help documentation and its urls.

This commit does not do changes in variable and class names, setting
field in database, API docs and changelog.

Fixes part of #27251.
2023-10-18 11:25:20 -07:00
Prakhar Pratyush 17a0304309 send_message: Add an optional parameter in the success response.
Add an optional `automatic_new_visibility_policy` enum field
in the success response to indicate the new visibility policy
value due to the `automatically_follow_topics_policy` and
`automatically_unmute_topics_in_muted_streams_policy` user settings
during the send message action.

Only present if there is a change in the visibility policy.
2023-10-17 15:38:16 -07:00
Mateusz Mandera 7604c7935c push_notifs: Improve error responses from /test_notification endpoint. 2023-10-12 18:15:00 -07:00
Mateusz Mandera d43be2b7c4 push_notifs: Add endpoint for sending a test notification.
Fixes #23997
2023-10-12 18:15:00 -07:00