Commit Graph

17281 Commits

Author SHA1 Message Date
Alex Vandiver 71e297efb4 realm: Differentiate reserved realms from in-use realms.
Fixes: #23896.
2023-09-25 12:48:14 -07:00
Anders Kaseorg dd7b09d71a ruff: Fix C416 Unnecessary `list` comprehension.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-24 23:25:52 -07:00
Alex Vandiver 4cef665e98 zilencer: Log how long the remote server delayed the push notification. 2023-09-24 15:24:02 -07:00
Alex Vandiver fd6091ad17 push_notifications: Provide message timestamp in both APNS and GCM notifications. 2023-09-24 15:24:02 -07:00
Alex Vandiver b8581e2895 push_notifications: Provide subsecond granularity on the sent time. 2023-09-24 15:24:02 -07:00
Lauryn Menard 48a1cf04d0 api-docs: Update the add and remove update_message_flags events.
Clarifies that the `all` field in the `op: "add"` event is only
relevant for the `"read"` message flag, and that it will be false
for all other specified flags in theses events.

Deprecates the `all` field in the `op: "remove"` event and document
that it is false for all specified flags.

Updates the deprecated `operation` field description and makes
a few other small revisions to the event text for clarity and
accuracy.
2023-09-22 11:28:09 -07:00
Hemant Umre be653dd5b4 org_settings: Add backend for `realm_jitsi_server_url` setting.
This commit adds a `jitsi_server_url` field to the Realm model, which
will be used to save the URL of the custom Jitsi Meet server. In
the database, `None` will encode the server-level default. We can't
readily use `None` in the API, as it could be confused with "field not
sent". Therefore, we will use the string "default" for this purpose.

We have also introduced `server_jitsi_server_url` in the `/register`
API. This will be used to display the server's default Jitsi server
URL in the settings UI.

The existing `jitsi_server_url` will now be calculated as
`realm_jitsi_server_url || server_jitsi_server_url`.

Fixes a part of #17914.

Co-authored-by: Gaurav Pandey <gauravguitarrocks@gmail.com>
2023-09-21 17:39:10 -07:00
Alex Vandiver 11a1cb9630 narrow: Skip the realm_id limit on joins across to usermessage.
The unique index on `(user_id, message_id)` that is the
`zerver_usermessage` table is rather specific, and even the PostgreSQL
extended statistics are not enough for it to realize there is a
correlation between the `realm_id` in the message table and the
`user_id` in the usermessage table.  This means that adding the
`realm_id` limit when there is a join to `zerver_usermessage` flips
the query plan from a nested loop of unique usermessage index-only
scan, with an index scan of the messages pkey -- to a parallel hash
join of the messages limit with a index scan of just the user_id limit
on usermessages.  It thinks this is necessary because it thinks that
the `realm_id` limit may remove a large number of messages from the
usermessage set -- which is totally untrue.

Remove the `realm_id` limit if we have a usermessage join.
2023-09-21 13:58:34 -07:00
Mateusz Mandera 00f4f77358 zilencer: Remove /json versions of push bouncer endpoints.
These don't make sense (because authentication here is not based on
session) and aren't used.
2023-09-21 10:42:15 -07:00
David Rosa 6588190918 help: Delete "Start a new topic" page in favor of "Starting a new topic".
- Deletes page, updates references, and adds URL redirect.
2023-09-21 08:41:40 -07:00
Sahil Batra cb203fbe9a user_groups: Do not allow empty group names in backend.
We now raise error if a user tries to create a group with
empty name or tries to update a group name to be empty.
2023-09-20 15:35:26 -07:00
Lauryn Menard f3a2927e7a api-docs: Remove unused PartiallyCompleted schema.
This was removed as a potential success response in #26658, but
removing the schema was missed in those changes.
2023-09-20 10:38:01 -07:00
Lauryn Menard ef1ac399e6 api-docs: Clean up shared schemas for error responses.
Removes the JsonErrorBase and JsonError schemas as all error
responses in the API docs use the CodedErrorBase or CodedError
schemas.

Removes the AddSubscriptionsResponse schema since it's no longer
incorrectly used as a shared schema for error responses, and
instead documents the specific success response properties in the
endpoint.
2023-09-20 10:38:01 -07:00
Lauryn Menard c0a4ecd5aa api-docs: Move ignored params example to IgnoredParametersSuccess. 2023-09-20 10:38:01 -07:00
Lauryn Menard f0fe86d5e3 api-docs: Move simple success example up to JsonSuccess schema. 2023-09-20 10:38:01 -07:00
Lauryn Menard eb99efccd8 api-docs: Make "code" field required for CodedErrorBase schema. 2023-09-20 10:38:01 -07:00
Lauryn Menard 6fac83d54f api-docs: Fix RateLimitedError schema. 2023-09-20 10:38:01 -07:00
Lauryn Menard ee2f58e67a api-docs: Replace incorrect JsonError schema refs with CodedError. 2023-09-20 10:38:01 -07:00
Lauryn Menard d3e66b4f4f api-docs: Add 'code' field to emoji does not exist error. 2023-09-20 10:38:01 -07:00
Lauryn Menard 28a2893a41 api-docs: Update mute/unmute user error documentation for 'code' fields. 2023-09-20 10:38:01 -07:00
Lauryn Menard c4d4dfda4d api-docs: Add shared InvalidStreamError schema.
Adds an InvalidStreamError schema for errors that return a 'msg'
field with the string: "Invalid stream ID". Updates endpoints that
have this error 'str' documented to use the shared schema.
2023-09-20 10:38:01 -07:00
Lauryn Menard 3e369bcf96 api-docs: Add 'code' field to errors for deleting only org owner. 2023-09-20 10:38:01 -07:00
Lauryn Menard cb921300ce api-docs: Add 'code' field to create user email already exists. 2023-09-20 10:38:01 -07:00
Lauryn Menard 1b14745b17 api-docs: Update InvalidMessageError and use for read receipts doc. 2023-09-20 10:38:01 -07:00
Lauryn Menard 767c6a506c api-docs: Update drafts/scheduled message ResourceNotFoundErrors.
Updates documentation of ResourceNotFoundErrors for unknown draft
and scheduled message IDs to include the 'code' field, have an
HTTP status code of 404 in the documentation, and to follow the
general description format of errors in the API documentation.
2023-09-20 10:38:01 -07:00
Lauryn Menard 03ba4df8ee api-docs: Add 'code' field to unauthorized error in delete attachments. 2023-09-20 10:38:01 -07:00
Lauryn Menard 68fcd0a299 api-docs: Add 'code' field to InvalidApiKeyError response example. 2023-09-20 10:38:01 -07:00
Mateusz Mandera 41d76969cb build_message_send_dict: Remove redundant realm arg.
Since the function asserts that the realm arg matches message.realm
anyway, having the argument is redundant here.
2023-09-20 09:55:25 -07:00
Alex Vandiver 5ee4b642ad views: Add a /health healthcheck endpoint.
This endpoint verifies that the services that Zulip needs to function
are running, and Django can talk to them.  It is designed to be used
as a readiness probe[^1] for Zulip, either by Kubernetes, or some other
reverse-proxy load-balancer in front of Zulip.  Because of this, it
limits access to only localhost and the IP addresses of configured
reverse proxies.

Tests are limited because we cannot stop running services (which would
impact other concurrent tests) and there would be extremely limited
utility to mocking the very specific methods we're calling to raising
the exceptions that we're looking for.

[^1]: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/
2023-09-20 09:53:59 -07:00
David Rosa cb123d4f15 management: Rename command reactivate_stream -> unarchive_stream. 2023-09-19 15:03:33 -07:00
Steve Howell a8f5836ee6 tests: Make soft-reactivation tests readable.
The `expected` flag was incredibly confusing, as you
couldn't tell from the calling code what you were
actually expecting to happen.

I avoid the context manager idiom in order to force
the callers to create simple helper functions, and
I de-duplicate some code in some places.

I also force the caller to explicitly soft-deactivate
the user with one simple line of code, so that the
person reading the test doesn't have to research
the side effects of the helper. (And I make it
very easy for new authors to follow the practice
going forward.)

This is also somewhat of a prep commit to avoid
the obfuscated use of refresh_from_db.
2023-09-18 16:55:06 -07:00
Steve Howell 31d2660ee2 tests: Simplify policy-related tests.
The helper here was no longer a useful abstraction.
2023-09-18 16:55:06 -07:00
Steve Howell 0e261f6ec4 tests: Wrap get_user method.
The get_user function is poorly named, but I don't want to
sweep the entire codebase yet.

It's also nice to have a test wrapper for little experiments
like profiling tests or hunting down calls to refresh_from_db.

It's possible that we would also just change the new wrapper
to more directly call Django. The `get_user` function isn't
used in a ton of real-world places, so we might want the test
code to just bypass the cache.
2023-09-18 16:55:06 -07:00
Steve Howell df43f86cbc tests: Clean up check_has_permission_policies.
I add a bunch of cute helper methods to make
the test a bit more readable.

And then I make sure to get clean objects,
which precludes the need for our callback
functions to refresh the user objects.

And finally I make sure that our validation
functions don't cause any round trips (assuming
we have fetched objects using a standard
Zulip helper, which example_user ensures.)
2023-09-18 16:55:05 -07:00
Lauryn Menard 31daef7f79 response: Remove "result: partially_completed" for success responses.
In feature levels 153 and 154, a new value of "partially_completed"
for `result` in a success (HTTP status code 200) was added for two
endpoints that process messages in batches: /api/delete-topic and
/api/mark-all-as-read.

Prior to these changes, `result` was either "success" or "error" for
all responses, which was a useful API invariant to have for clients.

So, here we remove "partially_completed" as a potential value for
"result" in a response. And instead, for the two endpoints noted
above, we return a boolean field "complete" to indicate if the
response successfully deleted/marked as read all the targeted
messages (complete: true) or if only some of the targeted messages
were processed (complete: false).

The "code" field for an error string that was also returned as part
of a partially completed response is removed in these changes as
well.

The web app does not currently use the /api/mark-all-as-read
endpoint, but it does use the /api/delete-topic endpoint, so these
changes update that to check the `complete` boolean instead of the
string value for `result`.
2023-09-18 13:18:24 -07:00
Aman Agrawal 7e11f95716 inbox: Add inbox as default view.
Fixes #26736
2023-09-18 12:17:49 -07:00
Lauryn Menard 7c618b57cf emails: Update subject for confirm_new_email to have realm host.
Updates the email subject for confirming an email change to include
the realm host.
2023-09-17 15:12:21 -07:00
Lauryn Menard ab54a3b792 api-docs: Clean up descriptions in unread_msgs register response.
For arrays of objects in return values of API endpoints, any
general description of the objects in the arrays should be
documented in the description of the array. A description at the
level of the items in the array will not be rendered in the API
documentation. Descriptions of each property of the object will
be rendered, but these are specific to the property and not the
object as a whole.

Updates the pms, streams and huddles arrays of objects included
in the unread_msgs object of the register response so that the
descriptions are at the array level in the OpenAPI documentation.
2023-09-15 13:20:03 -07:00
Lauryn Menard 0f5353486b api-docs: Document order of user IDs in huddles unread messages data.
When unread_msgs data was added to the register queue response, see
commit 4f0110e, the `user_ids_string` field in the `huddles` array
of objects with information about unread group direct messages, had
the user IDs in the string sorted numerically.

Documents that these strings include the current users's ID and are
sorted numerically and separated by commas so that the documentation
is clear for client implementations.
2023-09-15 13:20:03 -07:00
Mateusz Mandera 3e15ea3f3f scim: Add supporting for syncing the user role.
This adds support for syncing user role via the newly added "role"
attribute, which can be set to either of
['owner', 'administrator', 'moderator', 'member', 'guest'].

Removes durable=True from the atomic decorator of do_change_user_role,
as django-scim2 runs PATCH operations in an atomic block.
2023-09-15 13:15:45 -07:00
Prakhar Pratyush db8229ae32 test_events: Separate tests for send and update message events.
This is a prep commit to separate the single test
'test_stream_send_message_events' into two separate tests named
'test_stream_send_message_events' & test_stream_update_message_events'
to verify the events related to send and update message, respectively.

As a part of introducing two new user settings
* 'automatically_follow_topics_policy'
* 'automatically_unmute_topics_policy'
in the next commit, we will extend 'test_stream_send_message_events'.

This logical separation helps in avoiding a single, super-long test.
2023-09-14 17:16:36 -07:00
Prakhar Pratyush cf804200c7 tests: Remove the usage of stray desktop_icon_count_display values.
This commit removes the stray values, i.e., [1, 2, 3], used
in the tests for desktop_icon_count_display.

We use 'UserProfile.DESKTOP_ICON_COUNT_DISPLAY_CHOICES' instead.
2023-09-14 17:16:36 -07:00
Prakhar Pratyush 35c3724f22 tests: Fix the incomplete exclusion of the notification settings.
'test_change_user_settings' in 'UserDisplayActionTest' excludes
the notification settings and tests only the display settings.

The code block excluding the notification settings doesn't exclude
'modern_notification_settings'. It only excludes the
'notification_settings_legacy'.

This commit replaces 'notification_settings_legacy' with
'notification_setting_types', which consists of all the
notification settings.
2023-09-14 17:16:36 -07:00
Alex Vandiver 4f4627b79b send_email: Use a consistent order when sending custom emails to users. 2023-09-14 17:16:36 -07:00
Lauryn Menard 3cd2b6886d api-docs: Expands API changelog feature level 134 entry.
Expands API changelog feature level 134 entry and adds the related
Changes notes to the events documentation for the updates made in
commit f4fcedd: "stream op: create" and "subscription op: peer_add"
events being sent when a private stream is made public.

Those changes were made after the feature level 133 updates, but
before the feature level 134 updates, which is why 134 is the
feature level for the change that is documented for clients.
2023-09-14 09:50:41 -07:00
Lauryn Menard fdf41774db api-docs: Update subscription event for peer_add feature level 205.
In commit ada2991f1c, when a user gains access to a stream due to
a role change, in addition to sending "stream op: create" events,
"subscription op: peer_add" events are sent for streams that the
user gains access to due to their role change. Updates the API
changelog entry for feature level 205.

Updates the "subcription op: peer_add" event documentation to be
more accurate in for the general use cases of this event, which
are to provide updated subscriber information for streams that
a user has access to.
2023-09-14 09:50:41 -07:00
Lalit 2b566c778b user_settings: Add new `web_stream_unreads_count_display_policy` field.
This is a backend change that will help us support the new "Show unread counts for"
user display setting.
2023-09-13 18:45:45 -07:00
Anders Kaseorg 28597365da python: Delete superfluous parens.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-13 13:40:19 -07:00
Alex Vandiver e0d3176098 digest: Increase size of stream cache.
Since the cache is flushed when the cutoff or realm changes, the
maximum size of the cache should cap out at the number of streams in
the realm.  Raise the max cache size, now that this will not simply
lead to useless cache space for smaller servers.
2023-09-13 13:25:59 -07:00
Alex Vandiver a8a1f10f3c digest: Clear the cache once we move to a new realm / cutoff value. 2023-09-13 13:25:59 -07:00