Commit Graph

17433 Commits

Author SHA1 Message Date
Mateusz Mandera 7604c7935c push_notifs: Improve error responses from /test_notification endpoint. 2023-10-12 18:15:00 -07:00
Mateusz Mandera d43be2b7c4 push_notifs: Add endpoint for sending a test notification.
Fixes #23997
2023-10-12 18:15:00 -07:00
Alya Abbott b1e0fee101 help: Add keyboard documentation for default views. 2023-10-12 12:58:20 -07:00
David Rosa 69b8dbeb28 help: Document Inbox view.
- Adds desktop/web instructions.
- Adds #inbox relative link for logged-in users.
- Moves Inbox up in the left sidebar just under "Reading strategies".
- Moves Inbox article content to Markdown include.
- Adds "From the Inbox view" section to "Finding a topic to read",
  "Getting started with Zulip", and "Reading strategies".
- Documents Inbox as a new option for the default web app view.
- Removes unused Markdown link.
- Tweaks subheading to better match help center patterns.
- Add Inbox option in "Configure default settings for new users".
- Adds new tabbed section and instructions for marking messages as
  read and reading topics via the Inbox view.

Fixes #26903.

Co-authored-by: Alya Abbott <alya@zulip.com>
2023-10-12 12:58:20 -07:00
Anders Kaseorg a50eb2e809 mypy: Enable new error explicit-override.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-12 12:28:41 -07:00
Anders Kaseorg e245f51974 test_helpers: Remove unnecessary HostRequestMock.body override.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-12 12:28:41 -07:00
lapaz 417b29638c messages: Add support to allow bot-owners to delete messages.
This commit adds support to allow bot-owners to delete messages
sent by their bots if they are allowed to delete their own messages
as per "delete_own_message_policy" setting and the message delete
time limit has not passed.
2023-10-12 12:13:39 -07:00
Sahil Batra c51c1d5135 realm: Add setting to control adding "(guest)" indicator to name.
This commit adds new realm setting to control adding "(guest)"
indicator to names of guest users in the UI.

Fixes part of #26700.
2023-10-12 12:06:10 -07:00
Tim Abbott 07e6bcda7c tests: Fix missing setUp calls in ZulipTestCase subclasses.
This may save us from bugs like the ones fixed in
78683c1b9c in the future.
2023-10-12 10:28:04 -07:00
Alex Vandiver 869f96e31b decorator: Drop the stack_info on most WebhookErrors.
`stack_info` shows the stack between where the error was raised and
where it was captured -- which is not interesting when we
intentionally raised it, and know where it will be captured.

Omit the `stack_info` when it will just fill the logs with
uninteresting data.
2023-10-12 10:06:31 -07:00
Alex Vandiver 05d8c9d49e middleware: Don't double-log AnomalousWebhookPayloadError.
These are `WebhookError`s which have a 500 status code; we have logged
them already in the webhook decorator, so just return the content directly.
2023-10-12 10:06:31 -07:00
Alex Vandiver 8a6f96acad decorator: Stop stringifying exceptions unnecessarily. 2023-10-12 10:06:31 -07:00
Alex Vandiver 42b6b03ee7 decorator: Import notify_bot_owner_about_invalid_json.
This no longer causes an import cycle.
2023-10-12 10:06:31 -07:00
Alex Vandiver ea46478686 decorator: Reorganize and comment to clarify behaviour.
This is not a behaviour change, merely a hopefully cleaner way of
expressing the same logic.
2023-10-12 10:06:31 -07:00
Alex Vandiver 0a2ceda199 decorator: Let UnsupportedWebhookEventTypeError go through the logger.
It is not clear why 84723654c8 added these lines, are they are not
related to status codes.

Using an explicit `capture_exception` causes the exception in Sentry
to not have a `logger` field, which is quite useful for filtering.
2023-10-12 10:06:31 -07:00
Alex Vandiver 1e09de1da6 webhooks: Pass helpful strings to UnsupportedWebhookEventTypeError.
This helps understand _what_ event type was not supported.
2023-10-12 10:06:31 -07:00
Alex Vandiver 69326930ff freshping: Check state is supported before using it. 2023-10-12 10:06:31 -07:00
Alex Vandiver 204535926f transifex: Use "event" parameter.
Transifex's webhook documentation[^1] describes an `event` parameter
which is used to distinguish which event type was received.  Dispatch
based on that, and pass that value to UnsupportedWebhookEventTypeError
if need be.

[^1]: https://developers.transifex.com/docs/webhooks
2023-10-12 10:06:31 -07:00
Alex Vandiver c4085d1b8a statuspage: Raise AnomalousWebhookPayloadError on other events.
Their documentation only describes these two event types.
2023-10-12 10:06:31 -07:00
Alex Vandiver bda6a4b7e4 webhooks: Make MissingHTTPEventHeaderError a WebhookError.
Specifically, an AnomalousWebhookPayloadError.
2023-10-12 10:06:31 -07:00
Alex Vandiver 558a47af82 webhooks: All validate_extract_webhook_http_header are fatal.
85f453998e removed the one and only callsite which passed
`fatal=False`; simplify the type for this function.
2023-10-12 10:06:31 -07:00
Aman Agrawal b8b7938eb9 portico: Remove orgs who haven't set their own description. 2023-10-12 09:58:47 -07:00
Prakhar Pratyush c520a96354 typing_indicator: Add a 'stream_id' parameter to 'POST /typing'.
This commit adds a 'stream_id' parameter to the 'POST /typing'
endpoint.

Now, 'to' is used only for "direct" type. In the case of
"stream" type, 'stream_id' and 'topic' are used.
2023-10-12 09:53:09 -07:00
Prakhar Pratyush 78683c1b9c test_event_queue: Add missing "super call" in 'MissedMessageHookTest'.
Earlier, 'MissedMessageHookTest' didn't have 'super().setUp()'
and 'super().tearDown()' in the overrided methods 'setUp' and
'tearDown', respectively, that resulted in cached objects being
used between tests and hence flaky test failures.

This commit adds 'super().setUp()' and 'super().tearDown()'.
2023-10-12 09:46:13 -07:00
Esther Anierobi b2ea3125b2 exports: Improve notifications about completed data exports.
Change the url in the notification message to point to the settings
interface rather than linking to the export directly.

This is a much better user experience in the case that the export has
been deleted since the time the export was requested.

Fixes: #26923.
2023-10-11 17:42:32 -07:00
Alex Vandiver 7c80cbbc77 registration: On Zulip Cloud, show a 500 instead of a config page. 2023-10-11 17:13:01 -07:00
Alex Vandiver db38d43473 registration: Provide a better exception message. 2023-10-11 17:13:01 -07:00
Alex Vandiver e45f74dafb registration: Show SMTP failure page correctly.
232eb8b7cf changed how these pages work, to render inline instead of
serving from a URL, but did not update the SMTP use case; this made
SMTP failures redirect to a 404.
2023-10-11 17:13:01 -07:00
Alex Vandiver 2536dce136 test_helpers: Exempt testing-only endpoint from URL coverage. 2023-10-11 17:13:01 -07:00
Alex Vandiver d98aff26c9 config_error: Return status code 500. 2023-10-11 17:13:01 -07:00
Alex Vandiver a21971ceae config_error: Split error pages into their own pages.
This is much easier to maintain than a giant set of if statements.
2023-10-11 17:13:01 -07:00
Alex Vandiver 5506dffa36 config_error: Coalesce redundant dicts. 2023-10-11 17:13:01 -07:00
Alex Vandiver 536aa32255 views: Extract config_error view function. 2023-10-11 17:13:01 -07:00
Alex Vandiver 50c80c6ff9 test_signup: Use verify_signup helper when possible. 2023-10-11 16:08:43 -07:00
Alex Vandiver e8405c6f7b test_signup: Fix docstring on a test. 2023-10-11 16:08:43 -07:00
Alex Vandiver 2f61a70102 test_signup: Use verify_signup helper. 2023-10-11 16:08:43 -07:00
Alex Vandiver 2952595234 test_signup: Clean up test name and docstring. 2023-10-11 16:08:43 -07:00
Alex Vandiver 89b7f9cbd0 registration: Handle race conditions with duplicate emails.
Two registration requests for the same email address can race,
leading to an IntegrityError when making the second user.

Catch this and redirect them to the login page for their existing
email.
2023-10-11 16:08:43 -07:00
Alex Vandiver b49938265f backup: Only pass --host and --port if non-empty.
This works around the `/usr/bin/pg_dump` failure described in the
previous commit.  Since we are now calling the appropriately-versioned
`pg_dump` binary directly, it is no longer "necessary", but is added
as a defense-in-depth.
2023-10-11 11:50:56 -07:00
Alex Vandiver 3a8b4b0205 backup: Call the pg_dump binary whose version we are running.
`/usr/bin/pg_dump` on Ubuntu and Debian is actually a tool which
attempts to choose which `pg_dump` binary from all of the
`postgresql-client-*` packages that are installed to run.  However,
its logic is confused by passing empty `--host` and `--port` options
-- instead of looking at the running server instance on the server, it
instead assumes some remote host and chooses the highest versioned
`pg_dump` which is installed.

Because Zulip writes binary database backups, they are sensitive to
the version of the client `pg_dump` binary is used -- and the output
may not be backwards compatible.  Using a PostgreSQL 16 `pg_dump`
writes archive format 1.15, which cannot be read by a PostgreSQL 15
`pg_restore`.

Zulip does not currently support PostgreSQL 16 as a server.  This
means that backups on servers with `postgresql-client-16` installed
did not successfully round-trip Zulip backups -- their backups are
written using PostgreSQL 16's client, and the `pg_restore` chosen on
restore was correctly chosen as the one whose version matched the
server (PostgreSQL 15 or below), and thus did not understand the new
archive format.

Existing `./manage.py backups` taken since `postgresql-client-16` were
installed are thus not directly usable by the `restore-backup` script.
They are not useless, however, since they can theoretically be
converted into a format readable by PostgreSQL 15 -- by importing into
a PostgreSQL 16 instance, and re-dumping with a PostgreSQL 15
`pg_dump`.

Fix this issue by hard-coding path to the binary whose version matches
the version of the server we are connected to.  This may theoretically
fail if we are connected to a remote PostgreSQL instance and we do not
have a `postgresql-client` package locally installed which matches the
remote PostgreSQL server's version.  However, choosing a matching
version is the only way to ensure that it will be able to be imported
cleanly -- and it is preferable that we fail the backup process rather
than write backups that we cannot easily restore from.

Fixes: #27160.
2023-10-11 11:50:56 -07:00
Alex Vandiver 9d46b41e25 backup: Use simpler api for server version. 2023-10-11 11:50:56 -07:00
Alex Vandiver cdc3413a4f sentry: Add a circuit-breaker around attempts to report to Sentry.
The goal is to reduce load on Sentry if the service is timing out, and
to reduce uwsgi load from long requests.  This circuit-breaker is
per-Django-process, so may require more than 2 failures overall before
it trips, and may also "partially" trip for some (but not all)
workers.  Since all of this is best-effort, this is fine.

Because this is only for load reduction, we only circuit-breaker on
timeouts, and not unexpected HTTP response codes or the like.

See also #26229, which would move all browser-submitted Sentry
reporting into a single process, which would allow circuit-breaking to
be more effective.
2023-10-11 09:44:18 -07:00
Alex Vandiver 9519945dc0 sentry: Do not report errors tlaking to Sentry, to Sentry.
This prevents failure to submit a client-side Sentry trace from
turning into a server-side client trace.  If Sentry is down, we merely
log the error to our error logs and carry on.
2023-10-11 09:44:18 -07:00
Dinesh fccdc054a1 typing_notifications: Frontend work to display notifs for streams.
Display/hide "X is typing" notification
on receiving start/stop typing event for streams.

Co-authored-by: Prakhar Pratyush <prakhar841301@gmail.com>
2023-10-10 17:36:55 -07:00
Prakhar Pratyush e23f3ac89b typing_indicator: Update the 'to' parameter to accept stream ID.
When the `type` of the message being composed is "stream",
this commit updates the `to` parameter to accept the ID of
the stream in which the message is being typed.

Earlier, it accepted a single-element list containing the ID
of the stream.

Sending the element instead of a list containing the single element
makes more sense.
2023-10-10 17:15:28 -07:00
Prakhar Pratyush 8b12cc606a request: Extract out methods from 'scheduled_messages' to reuse.
This is a prep commit that extracts the following two methods
from '/actions/scheduled_messages' to reuse in the next commit.
* extract_stream_id
* extract_direct_message_recipient_ids

The 'to' parameter for 'POST /typing' will follow the same pattern
in the next commit as we currently have for the 'to' parameter in
'POST /scheduled_messages', so we can reuse these functions.
2023-10-10 17:15:28 -07:00
Prakhar Pratyush 39ec5b9f66 test_typing: Replace 'private' with 'direct'.
This commit replaces the keyword 'private' with 'direct'.

'pm' has been changed to 'direct message' to be more verbose.
2023-10-10 17:15:28 -07:00
Prakhar Pratyush 8d8b6ef156 typing_indicator: Remove the "private" compatibility support.
This commit removes the compatibility support for "private"
being a valid value for the 'type' parameter in 'POST /typing'.

"direct" and "stream" are the only valid values.
2023-10-10 17:15:28 -07:00
Prakhar Pratyush a22f637769 typing_indicator: Replace private with direct in 'message_type' field.
This commit replaces the value `private` with `direct` in the
`message_type` field for the `typing` events sent when a user
starts or stops typing a message.
2023-10-10 17:15:28 -07:00
Anders Kaseorg 835ee69c80 docs: Fix grammar errors found by mwic.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-09 13:24:09 -07:00
Anders Kaseorg f338c3af07 rate_limiter: Fix PIE790 Unnecessary `pass` statement.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-09 11:55:16 -07:00
Anders Kaseorg 4cb2eded68 typos: Fix typos caught by typos.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-09 11:55:16 -07:00
Anders Kaseorg 7b4a74cc4d codespell: Fix typos caught by codespell.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-09 11:55:15 -07:00
Alya Abbott f214ba7acc integrations docs: Clarify instructions for direct messages from Zapier. 2023-10-09 11:15:25 -07:00
Prakhar Pratyush 02f3f73b0c recipient_info: Include 'sender_id' in 'topic_participant_user_ids'.
This commit includes the message's sender id in the
'topic_participant_user_ids' set.

The 'participants_for_topic' function doesn't include the sender_id,
if the user is sending their first message in the topic, because
'participants_for_topic' queries the 'Message' table, but the message
is actually sent at a later stage in the codepath, resulting in
missing the sender_id in this case.

This is needed to set the 'wildcard_mentioned' flag for the sender's
user message in the case of topic wildcard mentions.

This doesn't lead to sending email and push notifications to the
sender because we have a check to skip notifications if the user
to receive notifications is the sender itself.

This should have been included in c0c30bc.
2023-10-06 11:38:37 -07:00
Simon Michalke b6a25840a1 zerver: Implement ldap group synchronization.
Fixes #9957.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2023-10-05 16:12:52 -07:00
Tim Abbott 2bfbbf0035 migrations: Fix initial value for automatic policy migrations.
Migration 0476 was incorrectly not updated for changes in the
defaults -- we intended the "never" value.
2023-10-05 16:11:25 -07:00
Alex Vandiver db203cfc2d newrelic: ids are uuids, not integers.
The previous fixtures were not generated from New Relic directly, so
incorrectly implied that `id` was an integer.  See the examples for
`issueId` in their documentation[^1].

[^1]: https://docs.newrelic.com/docs/alerts-applied-intelligence/notifications/message-templates/
2023-10-05 09:44:51 -07:00
Anders Kaseorg f99cce91bf middleware: Send got_request_exception signal for JSON 500 errors.
This is ordinarily emitted by Django at
  https://github.com/django/django/blob/4.2.6/django/core/handlers/exception.py#L139
and received by Sentry at
  https://github.com/getsentry/sentry-python/blob/1.31.0/sentry_sdk/integrations/django/__init__.py#L166

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-04 17:25:20 -07:00
Prakhar Pratyush 58568a60d6 settings: Add automatically follow and unmute topics policy settings.
This commit adds two user settings, named
* `automatically_follow_topics_policy`
* `automatically_unmute_topics_in_muted_streams_policy`

The settings control the user's preference on which topics they
will automatically 'follow' or 'unmute in muted streams'.

The policies offer four options:
1. Topics I participate in
2. Topics I send a message to
3. Topics I start
4. Never (default)

There is no support for configuring the settings through the UI yet.
2023-10-04 13:04:29 -07:00
Prakhar Pratyush c349d1137c sent_by_human: Add "test suite" to the set of Zulip UI-style clients.
Earlier, when we used 'self.send_message()' in the backend tests,
the sent message was not marked as read for the sender.

Reason: To set the read flag, we have to check if
'message.sent_by_human()'. It returns False because the
'sending_client' for tests is "test suite" and the 'sent_by_human'
function doesn't enlist the "test suite" client name as a human client.

This commit adds "test suite" to that list.

Also fixes a bug in when apply_unread_message_event was called that
was revealed by this change.
2023-10-04 13:04:29 -07:00
Prakhar Pratyush 68abc6af21 unread_msgs: Fix 'raw_unread_messages["muted_stream_ids"]' not set.
The raw_unread_messages["muted_stream_ids"] was not properly
updated with the calculated value.

The bug was introduced in 72e0084692.
2023-10-04 13:04:29 -07:00
Alya Abbott 456f3a0d94 integrations: Update Zapier documentation. 2023-10-03 14:21:36 -07:00
Lauryn Menard a1de07ba98 demo-orgs: Set organization type to "unspecified".
Instead of having "business" as the default organization type
for demo organizations in the dev environment, we set it to
"unspecified". This way a more generic zulip guide email will
be sent as part of the onboarding process for users invited
to try out the demo organization if the owner has not yet
updated the organization type.
2023-10-03 09:11:41 -07:00
Lauryn Menard f2ee4e8288 tests: Update events and schemas for drafts events.
Updates the testing for draft event schemas to be fully checked by
`zerver/tests/test_events.py` and `tools/check-schema`.

Also, corrects the type for the timestamp field in Draft objects
in the OpenAPI documentation.
2023-10-03 09:05:22 -07:00
Lauryn Menard ca5d1c1265 tests: Update events and schemas for scheduled message events.
Updates the testing for scheduled message event schemas to be fully
checked by `zerver/tests/test_events.py` and `tools/check-schema`.

Adds the missing 'failed' field to the scheduled message events
in `web/tests/lib/events.js` as well.
2023-10-03 09:05:22 -07:00
Chris Bobbe ad7dcc7d83 api docs: Use enum for reaction_type.
It's not currently rendered in the docs, but good to have this in
place anyway:
  https://github.com/zulip/zulip/pull/21005#discussion_r797823366
2023-10-03 08:13:07 -07:00
Aman Agrawal 9042cd989b base: Play videos in new tab instead of downloading them.
We add `Content-Disposition: inline` header to commonly supported
video MIME types so that when we `Open` them in lightbox, they
play in new tab.

This will require a follow-up database migration to apply to
previously uploaded videos.
2023-10-02 22:39:39 -07:00
Aman Agrawal 8ef52d55d3 markdown: Add support for inline video thumbnails. 2023-10-02 22:39:02 -07:00
Aman Agrawal 75a1a74adf hello: Redesign page.
Fixes #24082
2023-10-02 22:30:03 -07:00
Satyam Bansal 1e2f6a06a9 events: Exclude legacy integrations from realm_incoming_webhook_bots.
This excludes the legacy webhook from the
"realm_incoming_webhook_bots" object as those do not have the same URL
format as modern webhook integrations.
2023-10-02 12:29:28 -07:00
Satyam Bansal 02bc17e5dc api-docs: Minor grammar fix. 2023-10-02 12:26:44 -07:00
Tim Abbott d4d5cd1daf test_realm: Fix incorrect comparison of sort() return values.
Fixes #22105.
2023-10-01 21:49:48 -07:00
Pedro Madriz 0ef8ec698a Add support for importing guest users from Mattermost.
This change adds support for importing guest users from a Mattermost
export file into Zulip. The function now checks the user's teams and
roles to determine whether the user is a guest on the team, and sets
the user's role accordingly. This ensures that the imported user data
includes the correct role for each user.

Fixes #23720.
2023-10-01 17:13:23 -07:00
Tim Abbott c6fe799369 i18n: Fix default language for users created via API/LDAP.
This fixes a regression introduced in
9954db4b59, where the realm's default
language would be ignored for users created via API/LDAP/SAML,
resulting in all such users having English as their default language.

The API/LDAP/SAML account creation code paths don't have a request,
and thus cannot pull default language from the user's browser.

We have the `realm.default_language` field intended for this use case,
but it was not being passed through the system.

Rather than pass `realm.default_language` through from each caller, we
make the low-level user creation code set this field, as that seems
more robust to the creation of future callers.
2023-10-01 21:10:13 +02:00
Mateusz Mandera 7669fe8446 i18n: Tweak args/kwargs for get_default_language_for_new_user.
Making request a mandatory kwarg avoids confusion about the meaning of
parameters, especially with `request` acquiring the ability to be None
in the upcoming next commit.
2023-10-01 21:10:13 +02:00
Alex Vandiver c5373e9934 users: Prevent mirror dummy users from being "reactivated."
These do not represent users who were ever active, so they should not
be able to be reactivated.

See also #26703.
2023-10-01 09:29:23 -07:00
Mateusz Mandera 11390b70be tests: Eliminate instances of time_machine.travel without tick=False.
None of these tests seem to want to have tick=True, which is the
default. Letting the clock tick without a reason introduces the
possibility of nondeterministic test failures depending on the execution
time.
2023-10-01 09:28:42 -07:00
Alex Vandiver 723a4c2fff push_notifications: Send timestamp with only second granularity.
This reverts b8581e2895.  The mobile
client on Android parses this field using:

```kotlin
                timeMs = data.require("time").parseLong("time") * 1000
```

This throws an error if value is not `long` (i.e. an integer),
resulting in dropped notifications on Android from servers which had
deployed b8581e2895.

Switch back to sending an integer, but keep the behaviour from
fd6091ad17 where we send the timestamp in the payload of both
Android and Apple push notifications.
2023-09-29 14:33:44 -07:00
Alex Vandiver c3cf1cd9d8 data_import: Skip Slack "Posts" which we don't otherwise support. 2023-09-29 12:34:07 -07:00
Alex Vandiver 325ff11ecf soft_deactivation: Stop computing "new messages" in Python.
Rather than fetch all UserMessage rows for all streams, and subtract
those out in Python-space from the list of all Message rows the user
may have received -- do this via a "NOT EXISTS" subquery.  This is
much better indexed (performing in fractions of milliseconds rather
than hundreds), and also consumes much less memory.
2023-09-29 10:26:04 -07:00
Alex Vandiver f3cd1c4b08 delete_old_unclaimed_attachments: Update docs on default max age.
42f1cb3444 updated the default up, from 1 week to 5 weeks, but did
not adjust the documentation.
2023-09-29 09:49:03 -07:00
Tim Abbott ef5fee4778 user_groups: Improve bulk membership change logic.
Adds support for bulk-adjusting a single user's membership in multiple
user groups in a single transaction in the low-level actions
functions, for future use by work on #9957.
2023-09-29 15:08:02 +02:00
Lauryn Menard 5774486c12 api-docs: Correct error documentation for api/deactivate-own-user.
In commit 3e369bcf9, the `code` field for api/deactivate-own-user
was incorrectly documented as "BAD_REQUEST", which is the code for
the similar error returned by api/deactivate-user.

Corrects the error code to be "CANNOT_DEACTIVATE_LAST_USER" and
adds documentation for the two other fields returned by this
error response.

Note that the descriptions for the fields are added in the error
response schema will not be rendered in our current documentation.

They are rendered in other third-party tools and are therefore
good to have in our OpenAPI documentation. The description that
will be rendered in our documentation is the general error response
schema description and that is also updated for details about the
extra fields in this error response.
2023-09-28 08:39:29 -07:00
Lauryn Menard 84f1ee84ff email-notifications: Add helper function to TestFollowupEmailDelay.
Adds a helper function to main test in TestFollowupEmailDelay so
that adding additional onboarding emails to the schedule is clearer.
2023-09-27 17:38:05 -07:00
Mateusz Mandera ac14a8bcf5 typed_endpoint: Rename WebhookPayload to JsonBodyPayload.
This kind of payload that's loaded from json in the body of the request
is not only used for webhooks, but also in the push bouncer, and may get
used elsewhere too - so a general name is better.
2023-09-27 14:21:42 -07:00
Tim Abbott 9e3133e820 openapi: Add inline code block around count parameter. 2023-09-27 14:02:27 -07:00
Prakhar Pratyush 49092dfa79 unread_msgs: Fix all unreads in muted stream being treated as muted.
Earlier, 'is_row_muted' returned 'true' if the message was in
a muted stream or muted topic.

If the message is in an unmuted or followed topic in a muted
stream, such topics should be treated as not muted topics
in an unmuted stream.

This commit fixes the incorrect behavior.

Now, for wildcard mentions, 'unread_msgs.mentions' exclude
the IDs in muted streams only if the message is in default or
muted topic.

Also, 'unread_msgs.count' takes into account the unreads in unmuted
or followed topics in muted streams too.

Documents that this bug was fixed in the API changelog.
2023-09-27 13:11:20 -07:00
Prakhar Pratyush a18a526427 muted_stream_ids: Use set for O(1) search operation.
Update 'get_muted_stream_ids' to return a set of IDs
instead of a list.

This will help to avoid linear time search operations later
while using 'if stream_id in muted_streams_ids'.
2023-09-27 13:11:20 -07:00
Prakhar Pratyush faa98317bc user_topics: Refactor build_topic_mute_checker to support each policy.
This prep commit renames the 'build_topic_mute_checker' function
to 'build_get_topic_visibility_policy' and updates it to support
all the visibility policies.

The function prefetches the visibility policies the user has
configured for various topics and prepares a dict named
'topic_to_visibility_policy' to be used later on.
2023-09-27 13:11:20 -07:00
Prakhar Pratyush b5232c56ad test_unread_msgs: Update comment regarding unread message count.
A comment was added in f797604 to convey that the unread count
at that time doesn't exclude the unreads in muted topics.

848c080 added the support to exclude the muted topic;
however, the comment was not updated.

This commit updates the comment to reflect the current behavior.
2023-09-27 13:11:20 -07:00
Tim Abbott efb3059fdb models: Add more comments documenting AbstractMessage. 2023-09-27 11:49:04 -07:00
Deepayan Mukherjee 79916b9c24
onboarding: Finish renaming display settings -> preferences.
Fixes part of #26874.
2023-09-27 11:30:11 -07:00
Mateusz Mandera a095c34503 saml: Add OneLogin_Saml2_ValidationError to caught parsing exceptions.
This is an exception that we should be generally catching like the
others, which will give our standard /login/ redirect and proper logging
- as opposed to a 500 if we don't catch.

Addresses directly a bug we occurred in the wild, where a SAMLResponse
was submitted without issuers specified in a valid way, causing this
exception. The added test tests this specific type of scenario.
2023-09-27 10:32:18 -07:00
Alex Vandiver e1aca65d7d narrow: Add usermessage "private" flag limit to direct message queries.
This flag exists to accelerate searches -- use it.
2023-09-27 10:22:42 -07:00
Alex Vandiver 2405b23ad0 narrow: Add realm_id limits for direct message queries.
These queries benefit from the increased specificity of using the
realm / recipient / sender indexes.  The argument from 11a1cb9630
does not apply in these cases, since there are only 2 usermessage rows
for each matching message row for DMs, and few more than that for
huddles.
2023-09-27 10:22:42 -07:00
Alex Vandiver d55240e543 topic: Add comments calling out case-sensitive index usage. 2023-09-27 10:22:42 -07:00
Alex Vandiver 436e9c8a0c topic: Add realm limits to topic history queries. 2023-09-27 10:22:42 -07:00
Alex Vandiver ae822103a7 message: Add realm limit to private-message summary query.
This query has two halves; messages set by the user, and messages
received by the user.  The former uses the already-specific
usermessage privatemessage flag index; the latter relies on the
recipient index on messages.

Add the realm_id to the latter half, so that the recipient_id is
paired with the realm_id.
2023-09-27 10:22:42 -07:00
Alex Vandiver d28d347c35 narrow: Remove an out-of-date comment.
a9b3a9c673 changed this to use an explicit map, instead of getattr
magic.
2023-09-27 10:22:42 -07:00