Commit Graph

17331 Commits

Author SHA1 Message Date
Anders Kaseorg 7b4a74cc4d codespell: Fix typos caught by codespell.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-09 11:55:15 -07:00
Alya Abbott f214ba7acc integrations docs: Clarify instructions for direct messages from Zapier. 2023-10-09 11:15:25 -07:00
Prakhar Pratyush 02f3f73b0c recipient_info: Include 'sender_id' in 'topic_participant_user_ids'.
This commit includes the message's sender id in the
'topic_participant_user_ids' set.

The 'participants_for_topic' function doesn't include the sender_id,
if the user is sending their first message in the topic, because
'participants_for_topic' queries the 'Message' table, but the message
is actually sent at a later stage in the codepath, resulting in
missing the sender_id in this case.

This is needed to set the 'wildcard_mentioned' flag for the sender's
user message in the case of topic wildcard mentions.

This doesn't lead to sending email and push notifications to the
sender because we have a check to skip notifications if the user
to receive notifications is the sender itself.

This should have been included in c0c30bc.
2023-10-06 11:38:37 -07:00
Simon Michalke b6a25840a1 zerver: Implement ldap group synchronization.
Fixes #9957.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2023-10-05 16:12:52 -07:00
Tim Abbott 2bfbbf0035 migrations: Fix initial value for automatic policy migrations.
Migration 0476 was incorrectly not updated for changes in the
defaults -- we intended the "never" value.
2023-10-05 16:11:25 -07:00
Alex Vandiver db203cfc2d newrelic: ids are uuids, not integers.
The previous fixtures were not generated from New Relic directly, so
incorrectly implied that `id` was an integer.  See the examples for
`issueId` in their documentation[^1].

[^1]: https://docs.newrelic.com/docs/alerts-applied-intelligence/notifications/message-templates/
2023-10-05 09:44:51 -07:00
Anders Kaseorg f99cce91bf middleware: Send got_request_exception signal for JSON 500 errors.
This is ordinarily emitted by Django at
  https://github.com/django/django/blob/4.2.6/django/core/handlers/exception.py#L139
and received by Sentry at
  https://github.com/getsentry/sentry-python/blob/1.31.0/sentry_sdk/integrations/django/__init__.py#L166

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-04 17:25:20 -07:00
Prakhar Pratyush 58568a60d6 settings: Add automatically follow and unmute topics policy settings.
This commit adds two user settings, named
* `automatically_follow_topics_policy`
* `automatically_unmute_topics_in_muted_streams_policy`

The settings control the user's preference on which topics they
will automatically 'follow' or 'unmute in muted streams'.

The policies offer four options:
1. Topics I participate in
2. Topics I send a message to
3. Topics I start
4. Never (default)

There is no support for configuring the settings through the UI yet.
2023-10-04 13:04:29 -07:00
Prakhar Pratyush c349d1137c sent_by_human: Add "test suite" to the set of Zulip UI-style clients.
Earlier, when we used 'self.send_message()' in the backend tests,
the sent message was not marked as read for the sender.

Reason: To set the read flag, we have to check if
'message.sent_by_human()'. It returns False because the
'sending_client' for tests is "test suite" and the 'sent_by_human'
function doesn't enlist the "test suite" client name as a human client.

This commit adds "test suite" to that list.

Also fixes a bug in when apply_unread_message_event was called that
was revealed by this change.
2023-10-04 13:04:29 -07:00
Prakhar Pratyush 68abc6af21 unread_msgs: Fix 'raw_unread_messages["muted_stream_ids"]' not set.
The raw_unread_messages["muted_stream_ids"] was not properly
updated with the calculated value.

The bug was introduced in 72e0084692.
2023-10-04 13:04:29 -07:00
Alya Abbott 456f3a0d94 integrations: Update Zapier documentation. 2023-10-03 14:21:36 -07:00
Lauryn Menard a1de07ba98 demo-orgs: Set organization type to "unspecified".
Instead of having "business" as the default organization type
for demo organizations in the dev environment, we set it to
"unspecified". This way a more generic zulip guide email will
be sent as part of the onboarding process for users invited
to try out the demo organization if the owner has not yet
updated the organization type.
2023-10-03 09:11:41 -07:00
Lauryn Menard f2ee4e8288 tests: Update events and schemas for drafts events.
Updates the testing for draft event schemas to be fully checked by
`zerver/tests/test_events.py` and `tools/check-schema`.

Also, corrects the type for the timestamp field in Draft objects
in the OpenAPI documentation.
2023-10-03 09:05:22 -07:00
Lauryn Menard ca5d1c1265 tests: Update events and schemas for scheduled message events.
Updates the testing for scheduled message event schemas to be fully
checked by `zerver/tests/test_events.py` and `tools/check-schema`.

Adds the missing 'failed' field to the scheduled message events
in `web/tests/lib/events.js` as well.
2023-10-03 09:05:22 -07:00
Chris Bobbe ad7dcc7d83 api docs: Use enum for reaction_type.
It's not currently rendered in the docs, but good to have this in
place anyway:
  https://github.com/zulip/zulip/pull/21005#discussion_r797823366
2023-10-03 08:13:07 -07:00
Aman Agrawal 9042cd989b base: Play videos in new tab instead of downloading them.
We add `Content-Disposition: inline` header to commonly supported
video MIME types so that when we `Open` them in lightbox, they
play in new tab.

This will require a follow-up database migration to apply to
previously uploaded videos.
2023-10-02 22:39:39 -07:00
Aman Agrawal 8ef52d55d3 markdown: Add support for inline video thumbnails. 2023-10-02 22:39:02 -07:00
Aman Agrawal 75a1a74adf hello: Redesign page.
Fixes #24082
2023-10-02 22:30:03 -07:00
Satyam Bansal 1e2f6a06a9 events: Exclude legacy integrations from realm_incoming_webhook_bots.
This excludes the legacy webhook from the
"realm_incoming_webhook_bots" object as those do not have the same URL
format as modern webhook integrations.
2023-10-02 12:29:28 -07:00
Satyam Bansal 02bc17e5dc api-docs: Minor grammar fix. 2023-10-02 12:26:44 -07:00
Tim Abbott d4d5cd1daf test_realm: Fix incorrect comparison of sort() return values.
Fixes #22105.
2023-10-01 21:49:48 -07:00
Pedro Madriz 0ef8ec698a Add support for importing guest users from Mattermost.
This change adds support for importing guest users from a Mattermost
export file into Zulip. The function now checks the user's teams and
roles to determine whether the user is a guest on the team, and sets
the user's role accordingly. This ensures that the imported user data
includes the correct role for each user.

Fixes #23720.
2023-10-01 17:13:23 -07:00
Tim Abbott c6fe799369 i18n: Fix default language for users created via API/LDAP.
This fixes a regression introduced in
9954db4b59, where the realm's default
language would be ignored for users created via API/LDAP/SAML,
resulting in all such users having English as their default language.

The API/LDAP/SAML account creation code paths don't have a request,
and thus cannot pull default language from the user's browser.

We have the `realm.default_language` field intended for this use case,
but it was not being passed through the system.

Rather than pass `realm.default_language` through from each caller, we
make the low-level user creation code set this field, as that seems
more robust to the creation of future callers.
2023-10-01 21:10:13 +02:00
Mateusz Mandera 7669fe8446 i18n: Tweak args/kwargs for get_default_language_for_new_user.
Making request a mandatory kwarg avoids confusion about the meaning of
parameters, especially with `request` acquiring the ability to be None
in the upcoming next commit.
2023-10-01 21:10:13 +02:00
Alex Vandiver c5373e9934 users: Prevent mirror dummy users from being "reactivated."
These do not represent users who were ever active, so they should not
be able to be reactivated.

See also #26703.
2023-10-01 09:29:23 -07:00
Mateusz Mandera 11390b70be tests: Eliminate instances of time_machine.travel without tick=False.
None of these tests seem to want to have tick=True, which is the
default. Letting the clock tick without a reason introduces the
possibility of nondeterministic test failures depending on the execution
time.
2023-10-01 09:28:42 -07:00
Alex Vandiver 723a4c2fff push_notifications: Send timestamp with only second granularity.
This reverts b8581e2895.  The mobile
client on Android parses this field using:

```kotlin
                timeMs = data.require("time").parseLong("time") * 1000
```

This throws an error if value is not `long` (i.e. an integer),
resulting in dropped notifications on Android from servers which had
deployed b8581e2895.

Switch back to sending an integer, but keep the behaviour from
fd6091ad17 where we send the timestamp in the payload of both
Android and Apple push notifications.
2023-09-29 14:33:44 -07:00
Alex Vandiver c3cf1cd9d8 data_import: Skip Slack "Posts" which we don't otherwise support. 2023-09-29 12:34:07 -07:00
Alex Vandiver 325ff11ecf soft_deactivation: Stop computing "new messages" in Python.
Rather than fetch all UserMessage rows for all streams, and subtract
those out in Python-space from the list of all Message rows the user
may have received -- do this via a "NOT EXISTS" subquery.  This is
much better indexed (performing in fractions of milliseconds rather
than hundreds), and also consumes much less memory.
2023-09-29 10:26:04 -07:00
Alex Vandiver f3cd1c4b08 delete_old_unclaimed_attachments: Update docs on default max age.
42f1cb3444 updated the default up, from 1 week to 5 weeks, but did
not adjust the documentation.
2023-09-29 09:49:03 -07:00
Tim Abbott ef5fee4778 user_groups: Improve bulk membership change logic.
Adds support for bulk-adjusting a single user's membership in multiple
user groups in a single transaction in the low-level actions
functions, for future use by work on #9957.
2023-09-29 15:08:02 +02:00
Lauryn Menard 5774486c12 api-docs: Correct error documentation for api/deactivate-own-user.
In commit 3e369bcf9, the `code` field for api/deactivate-own-user
was incorrectly documented as "BAD_REQUEST", which is the code for
the similar error returned by api/deactivate-user.

Corrects the error code to be "CANNOT_DEACTIVATE_LAST_USER" and
adds documentation for the two other fields returned by this
error response.

Note that the descriptions for the fields are added in the error
response schema will not be rendered in our current documentation.

They are rendered in other third-party tools and are therefore
good to have in our OpenAPI documentation. The description that
will be rendered in our documentation is the general error response
schema description and that is also updated for details about the
extra fields in this error response.
2023-09-28 08:39:29 -07:00
Lauryn Menard 84f1ee84ff email-notifications: Add helper function to TestFollowupEmailDelay.
Adds a helper function to main test in TestFollowupEmailDelay so
that adding additional onboarding emails to the schedule is clearer.
2023-09-27 17:38:05 -07:00
Mateusz Mandera ac14a8bcf5 typed_endpoint: Rename WebhookPayload to JsonBodyPayload.
This kind of payload that's loaded from json in the body of the request
is not only used for webhooks, but also in the push bouncer, and may get
used elsewhere too - so a general name is better.
2023-09-27 14:21:42 -07:00
Tim Abbott 9e3133e820 openapi: Add inline code block around count parameter. 2023-09-27 14:02:27 -07:00
Prakhar Pratyush 49092dfa79 unread_msgs: Fix all unreads in muted stream being treated as muted.
Earlier, 'is_row_muted' returned 'true' if the message was in
a muted stream or muted topic.

If the message is in an unmuted or followed topic in a muted
stream, such topics should be treated as not muted topics
in an unmuted stream.

This commit fixes the incorrect behavior.

Now, for wildcard mentions, 'unread_msgs.mentions' exclude
the IDs in muted streams only if the message is in default or
muted topic.

Also, 'unread_msgs.count' takes into account the unreads in unmuted
or followed topics in muted streams too.

Documents that this bug was fixed in the API changelog.
2023-09-27 13:11:20 -07:00
Prakhar Pratyush a18a526427 muted_stream_ids: Use set for O(1) search operation.
Update 'get_muted_stream_ids' to return a set of IDs
instead of a list.

This will help to avoid linear time search operations later
while using 'if stream_id in muted_streams_ids'.
2023-09-27 13:11:20 -07:00
Prakhar Pratyush faa98317bc user_topics: Refactor build_topic_mute_checker to support each policy.
This prep commit renames the 'build_topic_mute_checker' function
to 'build_get_topic_visibility_policy' and updates it to support
all the visibility policies.

The function prefetches the visibility policies the user has
configured for various topics and prepares a dict named
'topic_to_visibility_policy' to be used later on.
2023-09-27 13:11:20 -07:00
Prakhar Pratyush b5232c56ad test_unread_msgs: Update comment regarding unread message count.
A comment was added in f797604 to convey that the unread count
at that time doesn't exclude the unreads in muted topics.

848c080 added the support to exclude the muted topic;
however, the comment was not updated.

This commit updates the comment to reflect the current behavior.
2023-09-27 13:11:20 -07:00
Tim Abbott efb3059fdb models: Add more comments documenting AbstractMessage. 2023-09-27 11:49:04 -07:00
Deepayan Mukherjee 79916b9c24
onboarding: Finish renaming display settings -> preferences.
Fixes part of #26874.
2023-09-27 11:30:11 -07:00
Mateusz Mandera a095c34503 saml: Add OneLogin_Saml2_ValidationError to caught parsing exceptions.
This is an exception that we should be generally catching like the
others, which will give our standard /login/ redirect and proper logging
- as opposed to a 500 if we don't catch.

Addresses directly a bug we occurred in the wild, where a SAMLResponse
was submitted without issuers specified in a valid way, causing this
exception. The added test tests this specific type of scenario.
2023-09-27 10:32:18 -07:00
Alex Vandiver e1aca65d7d narrow: Add usermessage "private" flag limit to direct message queries.
This flag exists to accelerate searches -- use it.
2023-09-27 10:22:42 -07:00
Alex Vandiver 2405b23ad0 narrow: Add realm_id limits for direct message queries.
These queries benefit from the increased specificity of using the
realm / recipient / sender indexes.  The argument from 11a1cb9630
does not apply in these cases, since there are only 2 usermessage rows
for each matching message row for DMs, and few more than that for
huddles.
2023-09-27 10:22:42 -07:00
Alex Vandiver d55240e543 topic: Add comments calling out case-sensitive index usage. 2023-09-27 10:22:42 -07:00
Alex Vandiver 436e9c8a0c topic: Add realm limits to topic history queries. 2023-09-27 10:22:42 -07:00
Alex Vandiver ae822103a7 message: Add realm limit to private-message summary query.
This query has two halves; messages set by the user, and messages
received by the user.  The former uses the already-specific
usermessage privatemessage flag index; the latter relies on the
recipient index on messages.

Add the realm_id to the latter half, so that the recipient_id is
paired with the realm_id.
2023-09-27 10:22:42 -07:00
Alex Vandiver d28d347c35 narrow: Remove an out-of-date comment.
a9b3a9c673 changed this to use an explicit map, instead of getattr
magic.
2023-09-27 10:22:42 -07:00
Hardik Dharmani 6c1f993855 user_preferences: Change option text for "Show unread counts for".
This commit updates the text for a dropdown option `Unmuted streams`
to `Unmuted streams and topics` for `Show unread counts for` user
preference settings for better clarity.
2023-09-26 15:10:51 -07:00
Lauryn Menard e5da17459d emails: Update out-of-date references to day1/day2 onboarding emails. 2023-09-26 11:46:25 -07:00