Commit Graph

17613 Commits

Author SHA1 Message Date
Tim Abbott 0d525cf644 integrations: Recommend new integration URL tool.
Fixes part of #25976.
2023-11-08 14:19:05 -08:00
David Rosa 0c4ef58acd help: Document new help menu.
Updates help center docs to reflect the new dedicated help menu.

- Adds `keyboard_tip` to document the `?` keyboard shortcut.

Fixes #27384.
2023-11-08 14:17:13 -08:00
Alex Vandiver acb4e9e783 cache_helpers: Show item and DB query counts. 2023-11-08 11:00:00 -08:00
roanster007 dc492867af user_mention: Fix mentions of deactivated users.
Previously, when a deactivated user was mentioned, he wasn't
rendered as a Pill. This is because the dataset for validating mentions
only included active users, which is fixed by removing that filter.

To allow only silent mentions of them, an extra is_active property
added to FullNameInfo class, which is populated from the query,
which tells if user is deactivated. This is used to convert any
mentions of them to silent mentions in the backend markdown.

Fixes #26857
2023-11-08 09:48:31 -08:00
Sahil Batra 617d2d509c users: Update format_user_row to return a TypedDict.
This commit updates format_user_row to return a TypedDict.

This commit is a prep commit for feature of restricting user
access such that code can be easy to read and understand when
we add that feature.
2023-11-08 09:22:26 -08:00
Sahil Batra 2ab71fb67d users: Update user_profile_to_user_row to return a TypedDict.
This commit updates user_profile_to_user_row to return a TypedDict
and also updates the return type of get_realm_user_dicts to be a
TypedDict.

This commit is a prep commit for feature of restricting user
access such that code can be easy to read and understand when
we add that feature.
2023-11-08 09:22:26 -08:00
Sahil Batra deb8431a47 users: Rename get_raw_user_data to get_users_for_api.
This is a prep commit for adding feature of restricting
user access to guests such that we can keep the code
easy to read and understand when that feature is added.
2023-11-08 09:22:26 -08:00
Greg Price 7a0ebc0d18 push_notifs: Log an error if ios_app_id missing on any APNs token. 2023-11-07 16:19:42 -08:00
Greg Price f109e3b598 push_notifs: Backfill ios_app_id on bouncer. 2023-11-07 16:19:42 -08:00
Greg Price be2a9a03d0 push_notifs: Check app ID has a plausible shape at bouncer. 2023-11-07 16:19:42 -08:00
Greg Price 1b2178f558 push_notifs: Require ios_app_id on register at bouncer, too.
We'll need this information in order to properly direct APNs
notifications.  Happily, the Zulip server always sends it when
registering an APNs token; and it appears it always has done so
since the commit:

cddee49e7 Add support infrastructure for push notification bouncer service.

back in 2016.  So there's no compatibility issue from requiring it.
2023-11-07 16:19:42 -08:00
Greg Price 9c2d53bcef push_notifs: Stop dropping ios_app_id on floor at bouncer.
This missing `REQ` call has meant we just drop this parameter:
even though the remote Zulip server passes it (for all APNs tokens),
we never notice and never store it.  Fix that.
2023-11-07 16:19:42 -08:00
Greg Price 9072ddda05 push_notifs: Remove remaining references to ZULIP_IOS_APP_ID setting. 2023-11-07 16:19:42 -08:00
Greg Price a9796ec503 push_notifs: Check app ID has a plausible shape. 2023-11-07 16:19:42 -08:00
Greg Price fb3af7fbcb push_notifs: Make appid required on add_apns_device_token.
We're going to need to use this information, so we shouldn't just
assume a value; the client should tell us the actual value.

Conveniently, the Zulip mobile app does already pass this parameter
and has since forever.  So we can just start requiring it, with no
compatibility constraint.
2023-11-07 16:19:42 -08:00
Greg Price 5d0d35d3f5 push_notifs: Always supply appid to add_apns_device_token in tests.
We already always pass this parameter from the mobile client,
so this makes the tests more realistic already.  And we'll shortly
be making this parameter required.
2023-11-07 16:19:42 -08:00
Greg Price 3893a70888 push_notifs: Cut spurious token_kind arguments in some test data.
These endpoints distinguish the token kinds by having different
URL paths, and don't take any argument by this name.
2023-11-07 16:19:42 -08:00
Alya Abbott 8fd631bf06 integrations-docs: Update intro content for Slack incoming webhooks. 2023-11-07 12:59:24 -08:00
Lauryn Menard 1618060a3e integrations-docs: Update both Slack integration articles.
Updates the Slack integration page to not describe adding a stream
or topic parameter to the URL query since that's not supported by
the current integration implementation.

Updates the Slack-compatible webhook integration page to have the
extra notes about the integration at the top of the page. Also,
removes the reference to a screenshot of the webhook since there
isn't one.
2023-11-07 12:59:24 -08:00
Prakhar Pratyush 94679d590f email_notification: Include prior message context only when mentioned.
Earlier, email message notifications included prior messages sent
to the same topic for context. This is more confusing than helpful
for messages that the user is likely to have received notifications
for all the prior messages in the conversation already (or read them
in the Zulip UI).

Now, we include prior context only when the user is mentioned via
personal, group, stream or topic wildcard mention.

Fixes #27479.
2023-11-07 11:44:38 -08:00
Prakhar Pratyush c0f445294c test_message_notif_email: Verify email has multiple stream messages.
This commit improves the test to explicitly verify that multiple
messages that were sent in quick succession to a topic are included
in the email body when we have email notifications enabled for a
given stream.

Earlier, the test was only verifying the email subject and the fact
that only one email was sent.

It is important to verify the fact that all the messages sent to a
topic in quick succession should be included in the email body.
2023-11-07 11:44:38 -08:00
Lauryn Menard fff2c9e47b integrations-docs: Update GitHub doc for new URL instructions. 2023-11-07 11:21:05 -08:00
Prakhar Pratyush e6e156709a typing_notifications: Don't notify long_term_idle subscribers.
The event for stream typing notifications is no longer sent
to the long_term_idle subscribers of the stream.

This helps to reduce the tornado's work of parsing super-long
JSON-encoded lists of user IDs in large streams. Now the lists
are shorter.
2023-11-07 09:30:27 -08:00
Aman Agrawal be6f467f42 home: Add `sponsorship_pending` page_param.
This will be used in gear menu to inform admin of their
sponsorship application status.

This includes some additional tweaks for when to show
billing and plans to users.
2023-11-06 14:55:14 -08:00
Aman Agrawal 00df9c6c18 urls: Show sponsorship page on a new URL.
Fixes #27377

Replaced #sponsorship links with sponsorship/ links and in other
places liking gear menu.
2023-11-06 14:55:14 -08:00
David Rosa 56f725e39b help: Document ability to drag-and-drop anywhere to upload a file.
- Replaces the "Via Markdown" tab with "Via drag-and-drop", and
modifies the instructions to explain that you can drag and drop
anywhere in the app, whether or not the compose box is open.
- Adds "Via paste" tab for the copy-pasting instructions.

Fixes #26894.
2023-11-04 12:49:54 -07:00
Aman Agrawal ddab0e6cd2 test_user_groups: Fix flaky tests.
There order of group ids doesn't matter here and thus the
compared values can have the ids in different order and test
should still pass. So, using `set` for comparing unordered
lists seems like the right fix here.
2023-11-04 12:46:06 -07:00
roanster007 ac084ddb0e mention_dataset: Add query to mention cross realm bots as pill.
Previously, cross realm bots were not displayed as mention Pills.

This is because, the data set for validating mentions considers
only the realm id which is None in case of cross realm bots.

Hence, adding an or Q object to it, to also check if
the email is a part of the cross realm bots email, in case the
realm id returns None.

Fixes #26913
2023-11-03 12:29:20 -07:00
Aman Agrawal 9ebb3f06b3 populate_billing_realms: Add script to populate different realms.
These realms are based on different cases of realms which can
exist while billing them and hence are useful for testing.
2023-11-03 08:44:23 -07:00
David Rosa c486ae37d3 help: Document option to paste URL to create a named link.
- Adds a "Via paste" tab to document the new functionality.

Fixes #26892.
2023-11-02 12:44:33 -07:00
Prakhar Pratyush b0ef76bf27 topic_mentions: Set 'topic_wildcard_mentioned' flag on @topic mention.
Earlier, the 'wildcard_mentioned' flag was set for both the
stream and topic wildcard mentions.

Now, the 'topic_wildcard_mentioned' flag is set for topic
wildcard mentions, and the 'wildcard_mentioned' flag is set for
stream wildcard mentions.

We will rename the 'wildcard_mentioned' flag to
'stream_wildcard_mentioned' in a later commit.
2023-11-02 09:25:51 -07:00
Prakhar Pratyush c37871ac3a user_message: Rename unused flags and create an index.
This commit renames the two unused and historical bits of the
'fields' bitfield of the 'UserMessage' and 'ArchivedUserMessage'
tables.

* 'summarize_in_home' to 'topic_wildcard_mentioned'
* 'summarize_in_stream' to 'group_mentioned'

The 'group_mentioned' flag doesn't affect the feature,
but completing the work here helps to save future migration
and indexing efforts on the UserMessage table, as we plan to
use this flag in the future for group mentions.

The unused bits may have old data; we'll clear that in
a separate commit.

It creates the 'zerver_usermessage_any_mentioned_message_id'
index concurrently.
2023-11-02 09:25:51 -07:00
Alex Vandiver 5a44a6624b gitlab: Description can be none. 2023-11-02 09:24:31 -07:00
Sahil Batra bb15b2d708 users: Send "update" events when deactivating or reactivating users.
We now send "realm_user/update" (and "realm_bot/update" for bots)
events with "is_active" field when deactivating and reactivating
users, including bots.

We would want to use "remove" event for a user losing access
to another user for #10970, so it is better to use "update"
event for deactivation as we only update "is_active" field
in the user objects and the clients still have the data for
deactivated users.

Previously, we used to send "add" event for reactivation along
with complete user objects, but clients should have the data
for deactivated users as well, so an "update" event is enough
like we do when deactivating users.
2023-11-02 08:31:12 -07:00
Mateusz Mandera 6819ecee92 zilencer: Add new LoggingCountStat mobile_pushes_forwarded.
This one counts actual successful deliveries.
2023-11-01 17:26:10 -07:00
Mateusz Mandera b7117d51b2 zilencer: Don't allow syncing mobile_pushes_received::day count. 2023-11-01 17:26:10 -07:00
Mateusz Mandera 7eb24ff237 send_test_push_notification_api: Fixes to the API endpoint doc.
Fixes misspellings and bad wording in the original zulip.yaml doc for
this endpoint.
2023-11-01 16:57:04 -07:00
Alex Vandiver 779268c3a4 sentry: Circuit-break on 429s and 500s from Sentry. 2023-11-01 16:16:37 -07:00
Alex Vandiver 631a1def13 portico: Skip footer and adjust header on invalid realm. 2023-11-01 11:24:12 -07:00
Alex Vandiver 54605e54b6 tests: Switch test_i18n fetch helper to use client_foo methods.
38f2a2f475 updated the comment but not the code.  Using
`self.client.post` instead of `self.client_post` means that we do not
set the host headers correctly.
2023-11-01 11:24:12 -07:00
Sahil Batra 47c8e369cf register: Pass the configuration objects for group settings in response.
This commit adds code to pass configuration objects for group
permission settings in register response to clients such that
we do need to duplicate that data in clients and can avoid
future bugs due to inconsistency.

The "server_supported_permission_settings" field is included
in the response if "realm" is present in "fetch_event_types",
as this is what we do for other server-related fields.
2023-11-01 10:42:56 -07:00
Sahil Batra e458b73a01 user_groups: Move constants for system group names to a new class.
This commit moves constants for system group names to a new
"SystemGroups" class so that we can use these group names
in multiple classes in models.py without worrying about the
order of defining them.
2023-11-01 10:42:56 -07:00
Sahil Batra 2c43c2a4db user_groups: Pass config object to access_user_group_for_setting.
We now pass the complete configuration object for a setting to
access_user_group_for_setting instead of passing the configuration
object's fields as different variables.
2023-11-01 10:42:56 -07:00
Sahil Batra 08d41f9eb3 settings: Rename permissions_configuration variable.
This commit renames permissions_configuration variable to
permission_configuration since the object contains config for
a single permission setting and thus permission_configuration
seems like a better name.
2023-11-01 10:42:56 -07:00
Lauryn Menard 6671a4a3f5 api-docs: Update description of update_message_flags op: add event.
Updates the description of update_message_flags op: add event for
details about actions that send/trigger the event and other details
that are useful for client implementations.

Also, links to the above updates in the op: remove variant of the
update_message_flags event.
2023-10-31 14:28:30 -07:00
Lauryn Menard afa5119303 api-docs: Update delete_message event for moved messages case. 2023-10-31 14:28:30 -07:00
Sahil Batra 9a6cf82adc streams: Fix sending stream-related events to guests.
Previous behavior-
- Guest did not receive stream creation events for new
web-public streams.
- Guest did not receive peer_add and peer_remove events
for web-public and subscribed public streams.

This commit fixes the behavior to be -
- Guests now receive stream creation events for new
web-public streams.
- Guest now receive peer_add and peer_remove events for
web-public and subscribed public streams.
2023-10-31 10:54:21 -07:00
Sahil Batra 71b8f49614 streams: Return early if there is no change in subscriptions.
This commit updates code in bulk_remove_subscriptions and
bulk_add_subscriptions to return early if there are no
subscribers to remove or add to the streams.

This change helps us in avoiding unnecessary queries like the
one used to get subscribers list of streams, which is then used
to send events but we would not send any events if no subscribers
are added or removed and some more similar queries.
2023-10-31 10:54:21 -07:00
Sahil Batra cdd15b4a69 test_events: Fix typo in comments. 2023-10-31 10:54:21 -07:00
Chandan Yadav 951e31a154 user_status: Remove `status` field from `UserInfoDict` type. 2023-10-30 11:10:47 -07:00
Anders Kaseorg 14b7d097a0 compilemessages: Normalize language names to NFC.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-26 16:27:35 -07:00
Hemant Umre d27a50852e onboarding: Internationalize the introductory messages in streams.
We use `Realm.default_language` value, which is set by selecting
the 'Organization language', to internationalize the introductory
messages of the initial streams.

Fixes #25729.
2023-10-26 16:27:35 -07:00
Hemant Umre ac1f711fef registration: Set the organization language at creation time.
In this commit, we add a new dropdown 'Organization language' on
the `/new` and `/realm/register` pages. This dropdown allows setting
the language of the organization during its creation. This allows
messages from Welcome Bot and introductory messages in streams to be
internationalized.

Fixes a part of #25729.
2023-10-26 16:27:35 -07:00
Sahil Batra c9fccf476a settings: Rename default_view and escape_navigates_to_default_view.
This commit renames default_view and escape_navigates_to_default_view
settings to web_home_view and web_escape_navigates_to_home_view in
database and API to match with our recent renaming of user facing
strings related to this.

We also rename the variables, functions, comments in code and class
names and IDs for elements related to this.
2023-10-26 11:00:12 -07:00
Lauryn Menard 02d6b3e16d onboarding-emails: Add new onboarding email for organization creator.
Adds a new onboarding email `onboarding_team_to_zulip` for the user
who created the new Zulip organization.

Co-authored by: Alya Abbott <alya@zulip.com>
2023-10-25 13:51:03 -07:00
Lauryn Menard f755fd1a3a api-docs: Check for deprecated at top level of endpoint docs.
Previously, we had checked that deprecated parameters and return
values had been marked as `deprecated: true` in the OpenAPI
documentation and had a description with a deprecated note.

Here we extend that check at the top level to deprecated endpoints.

The backend test that catches a failed assertion for this check
is `test_api_doc_endpoints` in zerver/tests/test_docs.py as that
test checks for a success response all pages linked the sidebar of
the API documentation.
2023-10-25 12:21:53 -07:00
Lauryn Menard 94088a7709 api-docs: Add deprecated true to /api/mute-topic endpoint. 2023-10-25 12:21:53 -07:00
Lauryn Menard 8905d918ae api-docs: Document bulk mark as read endpoints as deprecated. 2023-10-25 12:21:53 -07:00
Mateusz Mandera bb63aca8bb send_analytics_to_push_bouncer: Fix a stale comment.
The comment has drifted away from where it should be placed within the
code and also talks about RealmCounts specifically, while we have other
object types that this equally applies to.
2023-10-25 11:09:49 -07:00
Mateusz Mandera 986f2fd962 send_analytics_to_remote_server: Rename to ..._to_push_bouncer.
The former name is kind of misleading - this function is for the remote
server to send analytics to the push bouncer. Under our usual
terminology, a "remote server" is a self-hosted Zulip server. So data is
sent FROM not TO a remote server.
2023-10-25 11:09:49 -07:00
Prakhar Pratyush 3f1cc1b69d api_docs: Update the docs for the `POST /messages` response.
This commit updates the API docs for the optional parameter
'automatic_new_visibility_policy' in the `POST /messages` response.

Changes include:
* Explicitly mention that the new visibility policy is still
sent as `user_topic` event.
* Adds a link as a way to understand the meaning of the enum
values '2' and '3'.

Co-authored-by: Tim Abbott <tabbott@zulip.com>
2023-10-25 10:28:15 -07:00
Prakhar Pratyush b0d588bd98 models: Set defaults for the user_topic policies.
Now that we're enabling the feature in the UI, we should set
these to the planned long-term defaults for these settings.

Also, this commit cleans up the '0476' and '0477' migration
files related to user_topic policies.

'0476' sets 'null=True'
'0477' is noop
'0482' sets the default values and performs backfilling.

Co-authored-by: Tim Abbott <tabbott@zulip.com>
2023-10-25 02:21:03 +00:00
Lauryn Menard a8e3606c2c api-docs: Note that unread_message_ids are sorted in unread_msgs data. 2023-10-24 10:47:14 -07:00
Sayyed Arib Hussain 7e38d06130
create_user: Use silent mentions in user join notices.
This provides more information (email, etc.) when clicked on
while being better adapted to email_address_visibility existing.

Fixes #27243.
2023-10-24 09:58:55 -07:00
David Rosa 447b853ef1 help: Rename finding-a-topic-to-read -> finding-a-conversation-to-read
- Renames filenames to match page title.

Fixes part of #27191.
2023-10-23 16:52:41 -07:00
David Rosa d32a0601cd help: Rename help/reading-topics > help/reading-conversations.
- Renames filenames to match page title.

Fixes part of #27191.
2023-10-23 16:52:41 -07:00
ecxtacy ad63324dce help: Improve user stream management docs.
Documents the procedure to subscribe / unsubscribe a user via their
profile and general stream settings.

Both methods are separated into tabs in the documentation.

Fixes #26902
2023-10-23 14:17:21 -07:00
Anders Kaseorg d2faae36c6 openapi: Remove fix_events.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-23 09:07:07 -07:00
Anders Kaseorg ce0040b69f openapi: Add missing user property to deprecated user settings events.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-23 09:07:07 -07:00
Anders Kaseorg f256a5a536 openapi: Separate EmojiReactionEvent from EmojiReaction.
EmojiReactionEvent has user.user_id while EmojiReaction has user.id.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-23 09:07:07 -07:00
Anders Kaseorg b1f6ff20ef attachments: Correct attachment_id type from string to integer.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-22 17:06:34 -07:00
Esther Anierobi e957decd12
email: Drop support for SEND_MISSED_MESSAGE_EMAILS_AS_USER. (#27223)
Originally, this was how the notification emails worked, but that was changed
in 797a7ef97b, with this old behavior 
available as an option.

The footer and from address of emails that are sent when this
setting is set to True are confusing, especially when more people
are involved in a stream and since we have changed the way we send
emails, it should be removed. It’s also not widely used.

Fixes #26609.
2023-10-20 16:38:43 -07:00
Anders Kaseorg 9ba102f3ea openapi: Remove unused display_brief_error parameter.
It’s unused since commit a881918a05
(#24979).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-20 10:05:29 -07:00
Sahil Batra 760d793d6b settings: Rename "default" views to "home" views in settings and docs.
This commit renames "default" views to "home" views in the setting
labels, keyboard shortcuts list, help documentation and its urls.

This commit does not do changes in variable and class names, setting
field in database, API docs and changelog.

Fixes part of #27251.
2023-10-18 11:25:20 -07:00
Mateusz Mandera ff81e6bf32 import_util: Remove uuid and uuid_owner_secret from realm dict. 2023-10-18 11:00:49 -07:00
Mateusz Mandera 73cc680289 realm: Add uuid and associated secret columns - backfill migrations. 2023-10-18 11:00:49 -07:00
Mateusz Mandera 3afe585922 realm: Add uuid and associated secret columns.
Thisi and the following commit follow the approach used in
3e2ad84bbe.

First migration requires a server restart - after that any new realms
will be created with the columns set.

The following migrations are in the next commit:

Second migration does a backfill for older realms and can run in the
background while the server is operating normally.

Third migration enforces null=False now that all realms have the columns
set.
2023-10-18 11:00:49 -07:00
Prakhar Pratyush 17a0304309 send_message: Add an optional parameter in the success response.
Add an optional `automatic_new_visibility_policy` enum field
in the success response to indicate the new visibility policy
value due to the `automatically_follow_topics_policy` and
`automatically_unmute_topics_in_muted_streams_policy` user settings
during the send message action.

Only present if there is a change in the visibility policy.
2023-10-17 15:38:16 -07:00
Anders Kaseorg 70af7d7c58 test_user_groups: Fix database access outside test function.
Python evaluates function parameter defaults at definition time, not
call time.  This function wouldn’t work with other realms anyway.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-16 15:46:03 -07:00
Anders Kaseorg 94934f49f9 webpack_public_path: Replace with webpack 5 publicPath: "auto" setting.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-16 10:36:27 -07:00
Mateusz Mandera 7604c7935c push_notifs: Improve error responses from /test_notification endpoint. 2023-10-12 18:15:00 -07:00
Mateusz Mandera d43be2b7c4 push_notifs: Add endpoint for sending a test notification.
Fixes #23997
2023-10-12 18:15:00 -07:00
Alya Abbott b1e0fee101 help: Add keyboard documentation for default views. 2023-10-12 12:58:20 -07:00
David Rosa 69b8dbeb28 help: Document Inbox view.
- Adds desktop/web instructions.
- Adds #inbox relative link for logged-in users.
- Moves Inbox up in the left sidebar just under "Reading strategies".
- Moves Inbox article content to Markdown include.
- Adds "From the Inbox view" section to "Finding a topic to read",
  "Getting started with Zulip", and "Reading strategies".
- Documents Inbox as a new option for the default web app view.
- Removes unused Markdown link.
- Tweaks subheading to better match help center patterns.
- Add Inbox option in "Configure default settings for new users".
- Adds new tabbed section and instructions for marking messages as
  read and reading topics via the Inbox view.

Fixes #26903.

Co-authored-by: Alya Abbott <alya@zulip.com>
2023-10-12 12:58:20 -07:00
Anders Kaseorg a50eb2e809 mypy: Enable new error explicit-override.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-12 12:28:41 -07:00
Anders Kaseorg e245f51974 test_helpers: Remove unnecessary HostRequestMock.body override.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-12 12:28:41 -07:00
lapaz 417b29638c messages: Add support to allow bot-owners to delete messages.
This commit adds support to allow bot-owners to delete messages
sent by their bots if they are allowed to delete their own messages
as per "delete_own_message_policy" setting and the message delete
time limit has not passed.
2023-10-12 12:13:39 -07:00
Sahil Batra c51c1d5135 realm: Add setting to control adding "(guest)" indicator to name.
This commit adds new realm setting to control adding "(guest)"
indicator to names of guest users in the UI.

Fixes part of #26700.
2023-10-12 12:06:10 -07:00
Tim Abbott 07e6bcda7c tests: Fix missing setUp calls in ZulipTestCase subclasses.
This may save us from bugs like the ones fixed in
78683c1b9c in the future.
2023-10-12 10:28:04 -07:00
Alex Vandiver 869f96e31b decorator: Drop the stack_info on most WebhookErrors.
`stack_info` shows the stack between where the error was raised and
where it was captured -- which is not interesting when we
intentionally raised it, and know where it will be captured.

Omit the `stack_info` when it will just fill the logs with
uninteresting data.
2023-10-12 10:06:31 -07:00
Alex Vandiver 05d8c9d49e middleware: Don't double-log AnomalousWebhookPayloadError.
These are `WebhookError`s which have a 500 status code; we have logged
them already in the webhook decorator, so just return the content directly.
2023-10-12 10:06:31 -07:00
Alex Vandiver 8a6f96acad decorator: Stop stringifying exceptions unnecessarily. 2023-10-12 10:06:31 -07:00
Alex Vandiver 42b6b03ee7 decorator: Import notify_bot_owner_about_invalid_json.
This no longer causes an import cycle.
2023-10-12 10:06:31 -07:00
Alex Vandiver ea46478686 decorator: Reorganize and comment to clarify behaviour.
This is not a behaviour change, merely a hopefully cleaner way of
expressing the same logic.
2023-10-12 10:06:31 -07:00
Alex Vandiver 0a2ceda199 decorator: Let UnsupportedWebhookEventTypeError go through the logger.
It is not clear why 84723654c8 added these lines, are they are not
related to status codes.

Using an explicit `capture_exception` causes the exception in Sentry
to not have a `logger` field, which is quite useful for filtering.
2023-10-12 10:06:31 -07:00
Alex Vandiver 1e09de1da6 webhooks: Pass helpful strings to UnsupportedWebhookEventTypeError.
This helps understand _what_ event type was not supported.
2023-10-12 10:06:31 -07:00
Alex Vandiver 69326930ff freshping: Check state is supported before using it. 2023-10-12 10:06:31 -07:00
Alex Vandiver 204535926f transifex: Use "event" parameter.
Transifex's webhook documentation[^1] describes an `event` parameter
which is used to distinguish which event type was received.  Dispatch
based on that, and pass that value to UnsupportedWebhookEventTypeError
if need be.

[^1]: https://developers.transifex.com/docs/webhooks
2023-10-12 10:06:31 -07:00
Alex Vandiver c4085d1b8a statuspage: Raise AnomalousWebhookPayloadError on other events.
Their documentation only describes these two event types.
2023-10-12 10:06:31 -07:00
Alex Vandiver bda6a4b7e4 webhooks: Make MissingHTTPEventHeaderError a WebhookError.
Specifically, an AnomalousWebhookPayloadError.
2023-10-12 10:06:31 -07:00
Alex Vandiver 558a47af82 webhooks: All validate_extract_webhook_http_header are fatal.
85f453998e removed the one and only callsite which passed
`fatal=False`; simplify the type for this function.
2023-10-12 10:06:31 -07:00
Aman Agrawal b8b7938eb9 portico: Remove orgs who haven't set their own description. 2023-10-12 09:58:47 -07:00
Prakhar Pratyush c520a96354 typing_indicator: Add a 'stream_id' parameter to 'POST /typing'.
This commit adds a 'stream_id' parameter to the 'POST /typing'
endpoint.

Now, 'to' is used only for "direct" type. In the case of
"stream" type, 'stream_id' and 'topic' are used.
2023-10-12 09:53:09 -07:00
Prakhar Pratyush 78683c1b9c test_event_queue: Add missing "super call" in 'MissedMessageHookTest'.
Earlier, 'MissedMessageHookTest' didn't have 'super().setUp()'
and 'super().tearDown()' in the overrided methods 'setUp' and
'tearDown', respectively, that resulted in cached objects being
used between tests and hence flaky test failures.

This commit adds 'super().setUp()' and 'super().tearDown()'.
2023-10-12 09:46:13 -07:00
Esther Anierobi b2ea3125b2 exports: Improve notifications about completed data exports.
Change the url in the notification message to point to the settings
interface rather than linking to the export directly.

This is a much better user experience in the case that the export has
been deleted since the time the export was requested.

Fixes: #26923.
2023-10-11 17:42:32 -07:00
Alex Vandiver 7c80cbbc77 registration: On Zulip Cloud, show a 500 instead of a config page. 2023-10-11 17:13:01 -07:00
Alex Vandiver db38d43473 registration: Provide a better exception message. 2023-10-11 17:13:01 -07:00
Alex Vandiver e45f74dafb registration: Show SMTP failure page correctly.
232eb8b7cf changed how these pages work, to render inline instead of
serving from a URL, but did not update the SMTP use case; this made
SMTP failures redirect to a 404.
2023-10-11 17:13:01 -07:00
Alex Vandiver 2536dce136 test_helpers: Exempt testing-only endpoint from URL coverage. 2023-10-11 17:13:01 -07:00
Alex Vandiver d98aff26c9 config_error: Return status code 500. 2023-10-11 17:13:01 -07:00
Alex Vandiver a21971ceae config_error: Split error pages into their own pages.
This is much easier to maintain than a giant set of if statements.
2023-10-11 17:13:01 -07:00
Alex Vandiver 5506dffa36 config_error: Coalesce redundant dicts. 2023-10-11 17:13:01 -07:00
Alex Vandiver 536aa32255 views: Extract config_error view function. 2023-10-11 17:13:01 -07:00
Alex Vandiver 50c80c6ff9 test_signup: Use verify_signup helper when possible. 2023-10-11 16:08:43 -07:00
Alex Vandiver e8405c6f7b test_signup: Fix docstring on a test. 2023-10-11 16:08:43 -07:00
Alex Vandiver 2f61a70102 test_signup: Use verify_signup helper. 2023-10-11 16:08:43 -07:00
Alex Vandiver 2952595234 test_signup: Clean up test name and docstring. 2023-10-11 16:08:43 -07:00
Alex Vandiver 89b7f9cbd0 registration: Handle race conditions with duplicate emails.
Two registration requests for the same email address can race,
leading to an IntegrityError when making the second user.

Catch this and redirect them to the login page for their existing
email.
2023-10-11 16:08:43 -07:00
Alex Vandiver b49938265f backup: Only pass --host and --port if non-empty.
This works around the `/usr/bin/pg_dump` failure described in the
previous commit.  Since we are now calling the appropriately-versioned
`pg_dump` binary directly, it is no longer "necessary", but is added
as a defense-in-depth.
2023-10-11 11:50:56 -07:00
Alex Vandiver 3a8b4b0205 backup: Call the pg_dump binary whose version we are running.
`/usr/bin/pg_dump` on Ubuntu and Debian is actually a tool which
attempts to choose which `pg_dump` binary from all of the
`postgresql-client-*` packages that are installed to run.  However,
its logic is confused by passing empty `--host` and `--port` options
-- instead of looking at the running server instance on the server, it
instead assumes some remote host and chooses the highest versioned
`pg_dump` which is installed.

Because Zulip writes binary database backups, they are sensitive to
the version of the client `pg_dump` binary is used -- and the output
may not be backwards compatible.  Using a PostgreSQL 16 `pg_dump`
writes archive format 1.15, which cannot be read by a PostgreSQL 15
`pg_restore`.

Zulip does not currently support PostgreSQL 16 as a server.  This
means that backups on servers with `postgresql-client-16` installed
did not successfully round-trip Zulip backups -- their backups are
written using PostgreSQL 16's client, and the `pg_restore` chosen on
restore was correctly chosen as the one whose version matched the
server (PostgreSQL 15 or below), and thus did not understand the new
archive format.

Existing `./manage.py backups` taken since `postgresql-client-16` were
installed are thus not directly usable by the `restore-backup` script.
They are not useless, however, since they can theoretically be
converted into a format readable by PostgreSQL 15 -- by importing into
a PostgreSQL 16 instance, and re-dumping with a PostgreSQL 15
`pg_dump`.

Fix this issue by hard-coding path to the binary whose version matches
the version of the server we are connected to.  This may theoretically
fail if we are connected to a remote PostgreSQL instance and we do not
have a `postgresql-client` package locally installed which matches the
remote PostgreSQL server's version.  However, choosing a matching
version is the only way to ensure that it will be able to be imported
cleanly -- and it is preferable that we fail the backup process rather
than write backups that we cannot easily restore from.

Fixes: #27160.
2023-10-11 11:50:56 -07:00
Alex Vandiver 9d46b41e25 backup: Use simpler api for server version. 2023-10-11 11:50:56 -07:00
Alex Vandiver cdc3413a4f sentry: Add a circuit-breaker around attempts to report to Sentry.
The goal is to reduce load on Sentry if the service is timing out, and
to reduce uwsgi load from long requests.  This circuit-breaker is
per-Django-process, so may require more than 2 failures overall before
it trips, and may also "partially" trip for some (but not all)
workers.  Since all of this is best-effort, this is fine.

Because this is only for load reduction, we only circuit-breaker on
timeouts, and not unexpected HTTP response codes or the like.

See also #26229, which would move all browser-submitted Sentry
reporting into a single process, which would allow circuit-breaking to
be more effective.
2023-10-11 09:44:18 -07:00
Alex Vandiver 9519945dc0 sentry: Do not report errors tlaking to Sentry, to Sentry.
This prevents failure to submit a client-side Sentry trace from
turning into a server-side client trace.  If Sentry is down, we merely
log the error to our error logs and carry on.
2023-10-11 09:44:18 -07:00
Dinesh fccdc054a1 typing_notifications: Frontend work to display notifs for streams.
Display/hide "X is typing" notification
on receiving start/stop typing event for streams.

Co-authored-by: Prakhar Pratyush <prakhar841301@gmail.com>
2023-10-10 17:36:55 -07:00
Prakhar Pratyush e23f3ac89b typing_indicator: Update the 'to' parameter to accept stream ID.
When the `type` of the message being composed is "stream",
this commit updates the `to` parameter to accept the ID of
the stream in which the message is being typed.

Earlier, it accepted a single-element list containing the ID
of the stream.

Sending the element instead of a list containing the single element
makes more sense.
2023-10-10 17:15:28 -07:00
Prakhar Pratyush 8b12cc606a request: Extract out methods from 'scheduled_messages' to reuse.
This is a prep commit that extracts the following two methods
from '/actions/scheduled_messages' to reuse in the next commit.
* extract_stream_id
* extract_direct_message_recipient_ids

The 'to' parameter for 'POST /typing' will follow the same pattern
in the next commit as we currently have for the 'to' parameter in
'POST /scheduled_messages', so we can reuse these functions.
2023-10-10 17:15:28 -07:00
Prakhar Pratyush 39ec5b9f66 test_typing: Replace 'private' with 'direct'.
This commit replaces the keyword 'private' with 'direct'.

'pm' has been changed to 'direct message' to be more verbose.
2023-10-10 17:15:28 -07:00
Prakhar Pratyush 8d8b6ef156 typing_indicator: Remove the "private" compatibility support.
This commit removes the compatibility support for "private"
being a valid value for the 'type' parameter in 'POST /typing'.

"direct" and "stream" are the only valid values.
2023-10-10 17:15:28 -07:00
Prakhar Pratyush a22f637769 typing_indicator: Replace private with direct in 'message_type' field.
This commit replaces the value `private` with `direct` in the
`message_type` field for the `typing` events sent when a user
starts or stops typing a message.
2023-10-10 17:15:28 -07:00
Anders Kaseorg 835ee69c80 docs: Fix grammar errors found by mwic.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-09 13:24:09 -07:00
Anders Kaseorg f338c3af07 rate_limiter: Fix PIE790 Unnecessary `pass` statement.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-09 11:55:16 -07:00
Anders Kaseorg 4cb2eded68 typos: Fix typos caught by typos.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-09 11:55:16 -07:00
Anders Kaseorg 7b4a74cc4d codespell: Fix typos caught by codespell.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-09 11:55:15 -07:00
Alya Abbott f214ba7acc integrations docs: Clarify instructions for direct messages from Zapier. 2023-10-09 11:15:25 -07:00
Prakhar Pratyush 02f3f73b0c recipient_info: Include 'sender_id' in 'topic_participant_user_ids'.
This commit includes the message's sender id in the
'topic_participant_user_ids' set.

The 'participants_for_topic' function doesn't include the sender_id,
if the user is sending their first message in the topic, because
'participants_for_topic' queries the 'Message' table, but the message
is actually sent at a later stage in the codepath, resulting in
missing the sender_id in this case.

This is needed to set the 'wildcard_mentioned' flag for the sender's
user message in the case of topic wildcard mentions.

This doesn't lead to sending email and push notifications to the
sender because we have a check to skip notifications if the user
to receive notifications is the sender itself.

This should have been included in c0c30bc.
2023-10-06 11:38:37 -07:00
Simon Michalke b6a25840a1 zerver: Implement ldap group synchronization.
Fixes #9957.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2023-10-05 16:12:52 -07:00
Tim Abbott 2bfbbf0035 migrations: Fix initial value for automatic policy migrations.
Migration 0476 was incorrectly not updated for changes in the
defaults -- we intended the "never" value.
2023-10-05 16:11:25 -07:00
Alex Vandiver db203cfc2d newrelic: ids are uuids, not integers.
The previous fixtures were not generated from New Relic directly, so
incorrectly implied that `id` was an integer.  See the examples for
`issueId` in their documentation[^1].

[^1]: https://docs.newrelic.com/docs/alerts-applied-intelligence/notifications/message-templates/
2023-10-05 09:44:51 -07:00
Anders Kaseorg f99cce91bf middleware: Send got_request_exception signal for JSON 500 errors.
This is ordinarily emitted by Django at
  https://github.com/django/django/blob/4.2.6/django/core/handlers/exception.py#L139
and received by Sentry at
  https://github.com/getsentry/sentry-python/blob/1.31.0/sentry_sdk/integrations/django/__init__.py#L166

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-04 17:25:20 -07:00
Prakhar Pratyush 58568a60d6 settings: Add automatically follow and unmute topics policy settings.
This commit adds two user settings, named
* `automatically_follow_topics_policy`
* `automatically_unmute_topics_in_muted_streams_policy`

The settings control the user's preference on which topics they
will automatically 'follow' or 'unmute in muted streams'.

The policies offer four options:
1. Topics I participate in
2. Topics I send a message to
3. Topics I start
4. Never (default)

There is no support for configuring the settings through the UI yet.
2023-10-04 13:04:29 -07:00
Prakhar Pratyush c349d1137c sent_by_human: Add "test suite" to the set of Zulip UI-style clients.
Earlier, when we used 'self.send_message()' in the backend tests,
the sent message was not marked as read for the sender.

Reason: To set the read flag, we have to check if
'message.sent_by_human()'. It returns False because the
'sending_client' for tests is "test suite" and the 'sent_by_human'
function doesn't enlist the "test suite" client name as a human client.

This commit adds "test suite" to that list.

Also fixes a bug in when apply_unread_message_event was called that
was revealed by this change.
2023-10-04 13:04:29 -07:00
Prakhar Pratyush 68abc6af21 unread_msgs: Fix 'raw_unread_messages["muted_stream_ids"]' not set.
The raw_unread_messages["muted_stream_ids"] was not properly
updated with the calculated value.

The bug was introduced in 72e0084692.
2023-10-04 13:04:29 -07:00
Alya Abbott 456f3a0d94 integrations: Update Zapier documentation. 2023-10-03 14:21:36 -07:00
Lauryn Menard a1de07ba98 demo-orgs: Set organization type to "unspecified".
Instead of having "business" as the default organization type
for demo organizations in the dev environment, we set it to
"unspecified". This way a more generic zulip guide email will
be sent as part of the onboarding process for users invited
to try out the demo organization if the owner has not yet
updated the organization type.
2023-10-03 09:11:41 -07:00
Lauryn Menard f2ee4e8288 tests: Update events and schemas for drafts events.
Updates the testing for draft event schemas to be fully checked by
`zerver/tests/test_events.py` and `tools/check-schema`.

Also, corrects the type for the timestamp field in Draft objects
in the OpenAPI documentation.
2023-10-03 09:05:22 -07:00
Lauryn Menard ca5d1c1265 tests: Update events and schemas for scheduled message events.
Updates the testing for scheduled message event schemas to be fully
checked by `zerver/tests/test_events.py` and `tools/check-schema`.

Adds the missing 'failed' field to the scheduled message events
in `web/tests/lib/events.js` as well.
2023-10-03 09:05:22 -07:00
Chris Bobbe ad7dcc7d83 api docs: Use enum for reaction_type.
It's not currently rendered in the docs, but good to have this in
place anyway:
  https://github.com/zulip/zulip/pull/21005#discussion_r797823366
2023-10-03 08:13:07 -07:00
Aman Agrawal 9042cd989b base: Play videos in new tab instead of downloading them.
We add `Content-Disposition: inline` header to commonly supported
video MIME types so that when we `Open` them in lightbox, they
play in new tab.

This will require a follow-up database migration to apply to
previously uploaded videos.
2023-10-02 22:39:39 -07:00
Aman Agrawal 8ef52d55d3 markdown: Add support for inline video thumbnails. 2023-10-02 22:39:02 -07:00
Aman Agrawal 75a1a74adf hello: Redesign page.
Fixes #24082
2023-10-02 22:30:03 -07:00
Satyam Bansal 1e2f6a06a9 events: Exclude legacy integrations from realm_incoming_webhook_bots.
This excludes the legacy webhook from the
"realm_incoming_webhook_bots" object as those do not have the same URL
format as modern webhook integrations.
2023-10-02 12:29:28 -07:00
Satyam Bansal 02bc17e5dc api-docs: Minor grammar fix. 2023-10-02 12:26:44 -07:00
Tim Abbott d4d5cd1daf test_realm: Fix incorrect comparison of sort() return values.
Fixes #22105.
2023-10-01 21:49:48 -07:00
Pedro Madriz 0ef8ec698a Add support for importing guest users from Mattermost.
This change adds support for importing guest users from a Mattermost
export file into Zulip. The function now checks the user's teams and
roles to determine whether the user is a guest on the team, and sets
the user's role accordingly. This ensures that the imported user data
includes the correct role for each user.

Fixes #23720.
2023-10-01 17:13:23 -07:00
Tim Abbott c6fe799369 i18n: Fix default language for users created via API/LDAP.
This fixes a regression introduced in
9954db4b59, where the realm's default
language would be ignored for users created via API/LDAP/SAML,
resulting in all such users having English as their default language.

The API/LDAP/SAML account creation code paths don't have a request,
and thus cannot pull default language from the user's browser.

We have the `realm.default_language` field intended for this use case,
but it was not being passed through the system.

Rather than pass `realm.default_language` through from each caller, we
make the low-level user creation code set this field, as that seems
more robust to the creation of future callers.
2023-10-01 21:10:13 +02:00
Mateusz Mandera 7669fe8446 i18n: Tweak args/kwargs for get_default_language_for_new_user.
Making request a mandatory kwarg avoids confusion about the meaning of
parameters, especially with `request` acquiring the ability to be None
in the upcoming next commit.
2023-10-01 21:10:13 +02:00
Alex Vandiver c5373e9934 users: Prevent mirror dummy users from being "reactivated."
These do not represent users who were ever active, so they should not
be able to be reactivated.

See also #26703.
2023-10-01 09:29:23 -07:00
Mateusz Mandera 11390b70be tests: Eliminate instances of time_machine.travel without tick=False.
None of these tests seem to want to have tick=True, which is the
default. Letting the clock tick without a reason introduces the
possibility of nondeterministic test failures depending on the execution
time.
2023-10-01 09:28:42 -07:00
Alex Vandiver 723a4c2fff push_notifications: Send timestamp with only second granularity.
This reverts b8581e2895.  The mobile
client on Android parses this field using:

```kotlin
                timeMs = data.require("time").parseLong("time") * 1000
```

This throws an error if value is not `long` (i.e. an integer),
resulting in dropped notifications on Android from servers which had
deployed b8581e2895.

Switch back to sending an integer, but keep the behaviour from
fd6091ad17 where we send the timestamp in the payload of both
Android and Apple push notifications.
2023-09-29 14:33:44 -07:00
Alex Vandiver c3cf1cd9d8 data_import: Skip Slack "Posts" which we don't otherwise support. 2023-09-29 12:34:07 -07:00
Alex Vandiver 325ff11ecf soft_deactivation: Stop computing "new messages" in Python.
Rather than fetch all UserMessage rows for all streams, and subtract
those out in Python-space from the list of all Message rows the user
may have received -- do this via a "NOT EXISTS" subquery.  This is
much better indexed (performing in fractions of milliseconds rather
than hundreds), and also consumes much less memory.
2023-09-29 10:26:04 -07:00
Alex Vandiver f3cd1c4b08 delete_old_unclaimed_attachments: Update docs on default max age.
42f1cb3444 updated the default up, from 1 week to 5 weeks, but did
not adjust the documentation.
2023-09-29 09:49:03 -07:00
Tim Abbott ef5fee4778 user_groups: Improve bulk membership change logic.
Adds support for bulk-adjusting a single user's membership in multiple
user groups in a single transaction in the low-level actions
functions, for future use by work on #9957.
2023-09-29 15:08:02 +02:00
Lauryn Menard 5774486c12 api-docs: Correct error documentation for api/deactivate-own-user.
In commit 3e369bcf9, the `code` field for api/deactivate-own-user
was incorrectly documented as "BAD_REQUEST", which is the code for
the similar error returned by api/deactivate-user.

Corrects the error code to be "CANNOT_DEACTIVATE_LAST_USER" and
adds documentation for the two other fields returned by this
error response.

Note that the descriptions for the fields are added in the error
response schema will not be rendered in our current documentation.

They are rendered in other third-party tools and are therefore
good to have in our OpenAPI documentation. The description that
will be rendered in our documentation is the general error response
schema description and that is also updated for details about the
extra fields in this error response.
2023-09-28 08:39:29 -07:00
Lauryn Menard 84f1ee84ff email-notifications: Add helper function to TestFollowupEmailDelay.
Adds a helper function to main test in TestFollowupEmailDelay so
that adding additional onboarding emails to the schedule is clearer.
2023-09-27 17:38:05 -07:00
Mateusz Mandera ac14a8bcf5 typed_endpoint: Rename WebhookPayload to JsonBodyPayload.
This kind of payload that's loaded from json in the body of the request
is not only used for webhooks, but also in the push bouncer, and may get
used elsewhere too - so a general name is better.
2023-09-27 14:21:42 -07:00
Tim Abbott 9e3133e820 openapi: Add inline code block around count parameter. 2023-09-27 14:02:27 -07:00
Prakhar Pratyush 49092dfa79 unread_msgs: Fix all unreads in muted stream being treated as muted.
Earlier, 'is_row_muted' returned 'true' if the message was in
a muted stream or muted topic.

If the message is in an unmuted or followed topic in a muted
stream, such topics should be treated as not muted topics
in an unmuted stream.

This commit fixes the incorrect behavior.

Now, for wildcard mentions, 'unread_msgs.mentions' exclude
the IDs in muted streams only if the message is in default or
muted topic.

Also, 'unread_msgs.count' takes into account the unreads in unmuted
or followed topics in muted streams too.

Documents that this bug was fixed in the API changelog.
2023-09-27 13:11:20 -07:00
Prakhar Pratyush a18a526427 muted_stream_ids: Use set for O(1) search operation.
Update 'get_muted_stream_ids' to return a set of IDs
instead of a list.

This will help to avoid linear time search operations later
while using 'if stream_id in muted_streams_ids'.
2023-09-27 13:11:20 -07:00
Prakhar Pratyush faa98317bc user_topics: Refactor build_topic_mute_checker to support each policy.
This prep commit renames the 'build_topic_mute_checker' function
to 'build_get_topic_visibility_policy' and updates it to support
all the visibility policies.

The function prefetches the visibility policies the user has
configured for various topics and prepares a dict named
'topic_to_visibility_policy' to be used later on.
2023-09-27 13:11:20 -07:00
Prakhar Pratyush b5232c56ad test_unread_msgs: Update comment regarding unread message count.
A comment was added in f797604 to convey that the unread count
at that time doesn't exclude the unreads in muted topics.

848c080 added the support to exclude the muted topic;
however, the comment was not updated.

This commit updates the comment to reflect the current behavior.
2023-09-27 13:11:20 -07:00
Tim Abbott efb3059fdb models: Add more comments documenting AbstractMessage. 2023-09-27 11:49:04 -07:00
Deepayan Mukherjee 79916b9c24
onboarding: Finish renaming display settings -> preferences.
Fixes part of #26874.
2023-09-27 11:30:11 -07:00
Mateusz Mandera a095c34503 saml: Add OneLogin_Saml2_ValidationError to caught parsing exceptions.
This is an exception that we should be generally catching like the
others, which will give our standard /login/ redirect and proper logging
- as opposed to a 500 if we don't catch.

Addresses directly a bug we occurred in the wild, where a SAMLResponse
was submitted without issuers specified in a valid way, causing this
exception. The added test tests this specific type of scenario.
2023-09-27 10:32:18 -07:00
Alex Vandiver e1aca65d7d narrow: Add usermessage "private" flag limit to direct message queries.
This flag exists to accelerate searches -- use it.
2023-09-27 10:22:42 -07:00
Alex Vandiver 2405b23ad0 narrow: Add realm_id limits for direct message queries.
These queries benefit from the increased specificity of using the
realm / recipient / sender indexes.  The argument from 11a1cb9630
does not apply in these cases, since there are only 2 usermessage rows
for each matching message row for DMs, and few more than that for
huddles.
2023-09-27 10:22:42 -07:00
Alex Vandiver d55240e543 topic: Add comments calling out case-sensitive index usage. 2023-09-27 10:22:42 -07:00
Alex Vandiver 436e9c8a0c topic: Add realm limits to topic history queries. 2023-09-27 10:22:42 -07:00
Alex Vandiver ae822103a7 message: Add realm limit to private-message summary query.
This query has two halves; messages set by the user, and messages
received by the user.  The former uses the already-specific
usermessage privatemessage flag index; the latter relies on the
recipient index on messages.

Add the realm_id to the latter half, so that the recipient_id is
paired with the realm_id.
2023-09-27 10:22:42 -07:00
Alex Vandiver d28d347c35 narrow: Remove an out-of-date comment.
a9b3a9c673 changed this to use an explicit map, instead of getattr
magic.
2023-09-27 10:22:42 -07:00
Hardik Dharmani 6c1f993855 user_preferences: Change option text for "Show unread counts for".
This commit updates the text for a dropdown option `Unmuted streams`
to `Unmuted streams and topics` for `Show unread counts for` user
preference settings for better clarity.
2023-09-26 15:10:51 -07:00
Lauryn Menard e5da17459d emails: Update out-of-date references to day1/day2 onboarding emails. 2023-09-26 11:46:25 -07:00
Alex Vandiver 71e297efb4 realm: Differentiate reserved realms from in-use realms.
Fixes: #23896.
2023-09-25 12:48:14 -07:00
Anders Kaseorg dd7b09d71a ruff: Fix C416 Unnecessary `list` comprehension.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-24 23:25:52 -07:00
Alex Vandiver 4cef665e98 zilencer: Log how long the remote server delayed the push notification. 2023-09-24 15:24:02 -07:00
Alex Vandiver fd6091ad17 push_notifications: Provide message timestamp in both APNS and GCM notifications. 2023-09-24 15:24:02 -07:00
Alex Vandiver b8581e2895 push_notifications: Provide subsecond granularity on the sent time. 2023-09-24 15:24:02 -07:00
Lauryn Menard 48a1cf04d0 api-docs: Update the add and remove update_message_flags events.
Clarifies that the `all` field in the `op: "add"` event is only
relevant for the `"read"` message flag, and that it will be false
for all other specified flags in theses events.

Deprecates the `all` field in the `op: "remove"` event and document
that it is false for all specified flags.

Updates the deprecated `operation` field description and makes
a few other small revisions to the event text for clarity and
accuracy.
2023-09-22 11:28:09 -07:00
Hemant Umre be653dd5b4 org_settings: Add backend for `realm_jitsi_server_url` setting.
This commit adds a `jitsi_server_url` field to the Realm model, which
will be used to save the URL of the custom Jitsi Meet server. In
the database, `None` will encode the server-level default. We can't
readily use `None` in the API, as it could be confused with "field not
sent". Therefore, we will use the string "default" for this purpose.

We have also introduced `server_jitsi_server_url` in the `/register`
API. This will be used to display the server's default Jitsi server
URL in the settings UI.

The existing `jitsi_server_url` will now be calculated as
`realm_jitsi_server_url || server_jitsi_server_url`.

Fixes a part of #17914.

Co-authored-by: Gaurav Pandey <gauravguitarrocks@gmail.com>
2023-09-21 17:39:10 -07:00
Alex Vandiver 11a1cb9630 narrow: Skip the realm_id limit on joins across to usermessage.
The unique index on `(user_id, message_id)` that is the
`zerver_usermessage` table is rather specific, and even the PostgreSQL
extended statistics are not enough for it to realize there is a
correlation between the `realm_id` in the message table and the
`user_id` in the usermessage table.  This means that adding the
`realm_id` limit when there is a join to `zerver_usermessage` flips
the query plan from a nested loop of unique usermessage index-only
scan, with an index scan of the messages pkey -- to a parallel hash
join of the messages limit with a index scan of just the user_id limit
on usermessages.  It thinks this is necessary because it thinks that
the `realm_id` limit may remove a large number of messages from the
usermessage set -- which is totally untrue.

Remove the `realm_id` limit if we have a usermessage join.
2023-09-21 13:58:34 -07:00
Mateusz Mandera 00f4f77358 zilencer: Remove /json versions of push bouncer endpoints.
These don't make sense (because authentication here is not based on
session) and aren't used.
2023-09-21 10:42:15 -07:00
David Rosa 6588190918 help: Delete "Start a new topic" page in favor of "Starting a new topic".
- Deletes page, updates references, and adds URL redirect.
2023-09-21 08:41:40 -07:00
Sahil Batra cb203fbe9a user_groups: Do not allow empty group names in backend.
We now raise error if a user tries to create a group with
empty name or tries to update a group name to be empty.
2023-09-20 15:35:26 -07:00
Lauryn Menard f3a2927e7a api-docs: Remove unused PartiallyCompleted schema.
This was removed as a potential success response in #26658, but
removing the schema was missed in those changes.
2023-09-20 10:38:01 -07:00
Lauryn Menard ef1ac399e6 api-docs: Clean up shared schemas for error responses.
Removes the JsonErrorBase and JsonError schemas as all error
responses in the API docs use the CodedErrorBase or CodedError
schemas.

Removes the AddSubscriptionsResponse schema since it's no longer
incorrectly used as a shared schema for error responses, and
instead documents the specific success response properties in the
endpoint.
2023-09-20 10:38:01 -07:00
Lauryn Menard c0a4ecd5aa api-docs: Move ignored params example to IgnoredParametersSuccess. 2023-09-20 10:38:01 -07:00
Lauryn Menard f0fe86d5e3 api-docs: Move simple success example up to JsonSuccess schema. 2023-09-20 10:38:01 -07:00
Lauryn Menard eb99efccd8 api-docs: Make "code" field required for CodedErrorBase schema. 2023-09-20 10:38:01 -07:00
Lauryn Menard 6fac83d54f api-docs: Fix RateLimitedError schema. 2023-09-20 10:38:01 -07:00
Lauryn Menard ee2f58e67a api-docs: Replace incorrect JsonError schema refs with CodedError. 2023-09-20 10:38:01 -07:00