Commit Graph

15683 Commits

Author SHA1 Message Date
Zixuan James Li 730d3ec929 typing: Fix type annotation of assert_(not_)in_success_response.
This allows us to use them with HttpResponse objects returned by
calling a view function directly.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-17 12:05:38 -07:00
Zixuan James Li db3fd6947f test_decorators: Remove errors_disallowed helper.
The said hack was removed in c00aab8ede,
and rate_limit itself has also been removed.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-17 12:05:38 -07:00
Zixuan James Li 26a518267a rate_limit: Replace rate_limit with inlined rate limit checks.
This change incorporate should_rate_limit into rate_limit_user and
rate_limit_request_by_ip. Note a slight behavior change to other callers
to rate_limit_request_by_ip is made as we now check if the client is
eligible to be exempted from rate limiting now, which was previously
only done as a part of zerver.lib.rate_limiter.rate_limit.

Now we mock zerver.lib.rate_limiter.RateLimitedUser instead of
zerver.decorator.rate_limit_user in
zerver.tests.test_decorators.RateLimitTestCase, because rate_limit_user
will always be called but rate limit only happens the should_rate_limit
check passes;

we can continue to mock zerver.lib.rate_limiter.rate_limit_ip, because the
decorated view functions call rate_limit_request_by_ip that calls
rate_limit_ip when the should_rate_limit check passes.

We need to mock zerver.decorator.rate_limit_user for SkipRateLimitingTest
now because rate_limit has been removed. We don't need to mock
RateLimitedUser in this case because we are only verifying that
the skip_rate_limiting flag works.

To ensure coverage in add_logging_data, a new test case is added to use
a web_public_view (which decorates the view function with
add_logging_data) with a new flag to check_rate_limit_public_or_user_views.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-17 12:05:38 -07:00
Zixuan James Li 2aac1dc40a rate_limit: Move rate_limit_remote_server to zilencer.auth.
This allows us to avoid importing from zilencer conditionally in
zerver.lib.rate_limiter, as we make rate limiting self-contained now.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-17 12:05:38 -07:00
Zixuan James Li f158c86ae1 test_decorators: Rework RateLimitTestCase.
- RateLimitTestCase.get_ratelimited_view is replaced by a view
function directly decorated by public_json_view.

- the META dict is initialized with "PATH_INFO": "test" because now the
tests cover the process_client codepath;

- HostRequestMock is initialized with host="zulip.testserver" to pass
the validate_account_and_subdomain check;

- check_rate_limit_public_or_user_views replaces both
test_rate_limiting_happens_in_normal_case and
test_rate_limiting_happens_by_ip_if_unauthed.

Overall, we deduplicate the test cases in this change, and make sure
that they also cover the view function decorators for authentication.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-17 12:05:38 -07:00
Zixuan James Li 0916f9eca2 test_decorators: Extract check_rate_limit_public_or_user_views.
The test setup for some of the test cases are largely similar, so it
would be cleaner to be able to reuse them.

Note that we use "check" in the name of this helper because later we
will extend it to take a flag to set whether rate limiting is expected.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-17 12:05:38 -07:00
Zixuan James Li 29bad25f83 zilencer: Only apply rate limit to remote server.
This refactors the test case alongside, since normal views accessed by
remote server do not get rate limited by remote server anymore.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-17 12:05:38 -07:00
Zixuan James Li 79e86471e7 rate_limit: Extract should_rate_limit from rate_limit.
This allows us to reapply this check for other rate limiting functions.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-17 12:05:38 -07:00
Lauryn Menard 885c3d6593 subscriptions: Send update events for `is_muted` property.
In Zulip 2.1.0, the `is_muted` stream subscription property was
added and replaced the `in_home_view` property. But the server has
still only been sending subscription update events with the
`in_home_view` property.

Updates `do_change_subscription_property` to send a subscription
update event for both `is_muted` and `in_home_view`, so that
clients can fully migrate away from using `in_home_view` allowing
us to eventually remove it completely.
2022-08-16 16:50:51 -07:00
Zixuan James Li d6c3f6626a freshping: Create a mapping for event types.
This is a follow-up to #19274. We map the supported event types to a
more suitable format for events.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-16 13:43:10 -07:00
Kartik Srivastava 098effe0a6 api: Modify "alert_words" response description in GET /events doc. 2022-08-16 13:38:41 -07:00
Kartik Srivastava 342fd72a10 api: Document /users/me/alert_words API endpoint. 2022-08-16 13:37:52 -07:00
Zixuan James Li d4b6d36740 typing: Tighten type annotation of zerver.lib.rest.
By replacing Any with object we enforce type narrowing before using the
kwargs when a more specific type is required.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-15 19:51:08 -07:00
Zixuan James Li eb88fee745 rest: Remove kwargs from rest_path.
The only caller that passes the kwargs argument is the avatar rest_path.
The application of kwargs can be rewritten with a wrapper.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-15 19:51:08 -07:00
Mateusz Mandera d48075de58 logout_all_users: Add --rotate-api-keys option to the command.
Fixes #19397.
2022-08-15 17:58:05 -07:00
Zixuan James Li fcba35036b rest: Add assertions for entries from supported methods.
Mypy considers that "Tuple[Any, ...]" is incompatible with
"Union[Tuple[Callable[..., HttpResponse], Set[str]], HttpResponse]".

handler, view_flags = entry is sufficient to suppress the error, but we
also add assertions for full measure.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-15 17:20:04 -07:00
Anders Kaseorg c0287473b9 forms: Fix 500 error on realm creation with invalid email.
Commit b945aa3443 (#22604) incorrectly
assumed that Django would run the extra EmailField validators if basic
email address validation passed.  Actually, it runs all validators
unconditionally and collects all failures.  So email_is_not_disposable
needs to catch email address parsing errors.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-08-15 17:19:37 -07:00
Adam Sah 800de1e682 search: Add test for pgroogna parts-of-URL search.
This test verifies that the PGroonga backend fixes the issues in
backend by default.
2022-08-15 17:14:17 -07:00
Zixuan James Li 5c49e4ba06 rest: Extract remote_server_path from rest_path.
This allows us to separate the zilencer paths from other JSON paths,
with explicit type annotation expecting `RemoteZulipServer` as the
second parameter of the handler using
authenticated_remote_server_view.

The test case is also updated to remove a test for a situation that no
longer occurs anymore, since we don't perform subdomain checks on
remote servers.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-13 14:53:52 -07:00
Zixuan James Li dd2fd8edda rest: Implement get_target_view_function_or_response.
As noted in the docstring, this is a temporary helper function that
separates routing for paths that support multiple HTTP methods from
`rest_dispatch` itself. We will need to replace this helper with
class-based views in the future. The helper will also be handy to
reduce duplication when splitting up `rest_dispatch` by authentication
methods.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-13 14:53:51 -07:00
Zixuan James Li af88417847 decorator: Extract validate_remote_server.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-13 14:33:59 -07:00
Zixuan James Li ac2185a2e8 decorator: Extract get_basic_credentials.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-13 14:33:59 -07:00
Sahil Batra d3468e3f78 streams: Extract code for sending events in bulk_remove_subscriptions.
This is a prep commit such that we can avoid duplicate code when we
unsubscribe bots for inaccessible private streams when changing owner
or reactivating them.
2022-08-13 14:28:30 -07:00
Sahil Batra a1f40ccda5 message_edit: Make zero invalid value for message_content_edit_time_limit_seconds.
This commit changes the code to consider zero as an invalid value for
message_content_edit_time_limit_seconds. Now to represent the setting that
user can edit the message anytime, the setting value will be "None" in
database and "unlimited" will be passed to API from clients.
2022-08-12 18:09:53 -07:00
Sahil Batra 1747ab8482 realm: Set setting value only if value has changed.
We sent the "message_content_delete_limit_seconds"
value to API when we change any setting in that
subsection and thus "do_set_realm_property" is
called even when we do not change that setting
since we handle it separately from most of the
other settings for which we use loop to call
"do_set_realm_property".

This commit changes the code to handle only
the case when the setting value is "None"
separately and rest all the values will be
changed from the "do_set_realm_property"
called in the loop as for other settings.
2022-08-12 18:09:53 -07:00
Zixuan James Li 92c4e5b75c models: Add type annotation for CTEManager.
mypy_django_plugin cannot resolve the type of the custom manager. We add
a type annotation explicitly to skip the type inferrence.

This fixes the following error:
error: Could not resolve manager type for "zerver.models.UserGroup.objects"  [django-manager-missing]

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 17:08:04 -07:00
Zixuan James Li 95d12749c0 import_realm: Check for recipient instead of recipient_id.
Technically recipient_id cannot be None when recipient exists. We
actually just want to check if the recipient exists.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 17:08:04 -07:00
Zixuan James Li 5bdf49c005 decorator: Add an isinstance check for otp auth test function.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 17:08:04 -07:00
Zixuan James Li c1f5609e59 streams: Add None check for stream.recipient_id.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 17:08:04 -07:00
Zixuan James Li 5d65ca33a8 types: Make value optional for ProfileData.
`UserProfile.profile_data` allows each element's value to be `None`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 17:08:04 -07:00
Zixuan James Li 2382f1925d export: Add an isinstance check for orig_dt.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 17:08:04 -07:00
Zixuan James Li 35e0fe8347 models: Duplicate __str__ in subclasses of AbstractUserMessage.
The `message` attirbute doesn't get defined in the class where it is used.
The cleanest solution will be using the `Self` type introduced in PEP
673 and type it as something like `ForeignKey[Union[Self, Combinable],
Self]`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 17:08:04 -07:00
Zixuan James Li 8fae87e2df zephyr: Check PERSONAL_ZMIRROR_SERVER before updating cache.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 17:08:04 -07:00
Zixuan James Li d7746c9e6c logging_util: Ensure that the logged user is authenticated.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 17:08:04 -07:00
Zixuan James Li 7acf43f475 bots: Allow changing bot owner without an acting_user.
This applies to the usecase in
zerver.tests.test_bots.BotTest.test_patch_bot_role.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 17:08:04 -07:00
Zixuan James Li e382cec015 topic: Add a None check with an assertion.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 17:08:04 -07:00
Zixuan James Li 14dd7f1692 auth: Make QueryDict immutable before assignment.
This modifies the QueryDict when it is mutable, and assign it to `.POST`
after it is turned immutable, as required by django-stubs for this
attribute.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 17:08:04 -07:00
Zixuan James Li a0ccc7b872 test_signup: Replace MagicMock with HostRequestMock.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 17:08:04 -07:00
Zixuan James Li a3ec420074 auth: Add type annotation for contexts.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 17:08:04 -07:00
Zixuan James Li 87921315be migrations: Remove noop migration operations.
Since validators do not affect the database, this migration is a noop.
Removing the migration fails check-database-compatibility. We might
eventually delete it when Django supports a cleaner method for deletion.

TODO:
Remove the type annotation when django-stubs is integrated.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 16:53:54 -07:00
Tim Abbott 9bf383dcae decorator: Reorder authenticated_json_view.
Checking authentication before rate limiting is easier to reason
about, especially since rate_limit() would check authentication
anyway.
2022-08-12 16:51:55 -07:00
Tim Abbott 60a2de21a9 decorator: Reorder public_json_view.
Doing the dispatch to authenticated_json_view first lets us avoid
messing around with the skip_rate_limiting parameter.

Since rate_limit itself checks user.is_authenticated, there's no
potential downside to doing that check first here.
2022-08-12 16:51:55 -07:00
Zixuan James Li f54ecad6cd decorator: Extract public_json_view.
This refactoring is necessary to separate the expected type annotation
for view functions with different authentication methods. Currently the
signature aren't actually check against view functions because
`rest_path` does not support type checking parameter types, but it will
become useful once we do.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 16:51:55 -07:00
Zixuan James Li 299f3442ff decorator: Refactor view decorators with ParamSpec.
`authenticated_rest_api_view` and `authenticated_json_view` essentially
remove `UserProfile` from the decorated function.

Note that `authenticated_log_and_execute_json` is removed to avoid
duplicating `ParamT` unnecessarily in the helper.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 16:51:55 -07:00
Zixuan James Li c9f54766c3 rate_limiter: Extract rate limit related functions.
This refactors rate limit related functions from `zerver.decorator` to
zerver.lib.rate_limiter.

We conditionally import `RemoteZulipServer`, `RequestNotes`, and
`RateLimitedRemoteZulipServer` to avoid circular dependency.

Most instances of importing these functions from `zerver.decorator` got
updated, with a few exceptions in `zerver.tests.test_decorators`, where
we do want to mock the rate limiting functions imported in
`zerver.decorator`. The same goes with the mocking example in the
"testing-with-django" documentation.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 16:51:55 -07:00
Zixuan James Li 232ba4866a rate_limit: Stop wrapping rate limited functions.
This refactors `rate_limit` so that we no longer use it as a decorator.
This is a workaround to https://github.com/python/mypy/issues/12909 as
`rate_limit` previous expects different parameters than its callers.

Our approach to test logging handlers also needs to be updated because
the view function is not decorated by `rate_limit`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 16:51:55 -07:00
Dinesh 48d2783559 read_receipts: Add support for displaying read receipts.
Adds an API endpoint for accessing read receipts for other users, as
well as a modal UI for displaying that information.

Enables the previously merged privacy settings UI for managing whether
a user makes read receipts data available to other users.

Documentation is pending, and we'll likely want to link to the
documentation with help_settings_link once it is complete.

Fixes #3618.

Co-authored-by: Tim Abbott <tabbott@zulip.com>
2022-08-12 13:16:35 -07:00
Tim Abbott 5bd1a85659 test_fixtures: Rebuild database when create_realm.py changes.
Now that we've split this out from the enormous actions.py, it makes
sense to include this in the set of inputs for generating the
database.
2022-08-12 13:16:35 -07:00
Sahil Batra 09b35b09c5 migrations: Add migration to set default value of enable_read_receipts.
This migration set default value of enable_read_receipts to True
for existing realms which require an invitation to join.
2022-08-12 22:26:14 +05:30
Sahil Batra 13e0311ef3 realm: Add support to change enable_read_receipts setting.
This commit adds support to change enable_read_receipts
setting through API and also adds the field to response
of "/register" endpoint so that the setting value
is available to clients.
2022-08-12 17:10:03 +05:30