Now that we can assume Python 3.6+, we can use the
email.headerregistry module to replace hacky manual email address
parsing.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
We have already checked the size of the file in `upload_file_backend`.
This is the only caller of `upload_message_image_from_request`, and
indirectly the only caller of `get_file_info`. There is no need to
retrieve this information again.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
It is not possible in the codebase to have request.user be None. But
it is possible to have it not present at all. `delattr` is more
appropriate here.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This makes the test cleaner and we don't have to overwrite the `get_host`
callable on `HttpRequest`.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
`context` as `AccessDeniedError` is incompatible with
`RequestVariableMissingError`. Mypy does not allow such redefinition.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
`DiscoverRunner.run_tests` has a return type of `int`. While
`Runner.run_tests` has a wildly different `Tuple[bool, List[str]]`.
This refactors it so that we have the correct return type, by passing
the additional information about failed tests through a side effect to directly
write the failed tests to a file.
Note that we have to make `failed_tests_path` optional as otherwise the method
signature will not be compatible with the supertype.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This reverts part of commit 1432067959
(#17047). The spooky warnings foretold by the comment don’t seem to
show up.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
`context` as `AccessDeniedError` is incompatible with
`RequestVariableMissingError`. Mypy does not allow such redefinition.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
It is really a generator of test cases from the test suite. Which should
be typed as an `Iterable` instead.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
`context_data` is only available on `SimpleTemplateResposne`, we can't
narrow `TestHttpResponse` to it because the latter is not in fact a
subtype of `HttpResponse`.
Differently, `redirect_chain` is an attribute that only appears on the
test response when the test client method is called with `follow=True`.
`TestHttpResponse` does not have that by defalut, either.
The occurence of these two cases are rare enough throughout the codebase
and we can't get around that without aggressively overloading the test client
or refactoring `_MonkeyPatchedWSGIResponse` in the upstream.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This eliminates the possibility of having `request.user` as
`RemoteZulipServer` by refactoring it as an attribute of `RequestNotes`.
So we can effectively narrow the type of `request.user` by testing
`user.is_authenticated` in most cases (except that of `SCIMClient`) in
code paths that require access to `.format_requestor_for_logs` where we
previously expect either `UserProfile` or `RemoteZulipServer` backed by
the implied polymorphism.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
In b46af40bd3,
we set this attribute because back then we might call `rate_limit_user`
on `RemoteZulipServer`.
This is no longer the case as `RemoteZulipServer` now has its own rate
limiting and we never call `rate_limit_user` without an `isinstance` check
for `UserProfile`.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This verifies that `request_for_logs` is correctly set for requests
with different types of authentication.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
`BaseNotes(str, str).get_notes` does not do anything here.
It was introduced in
53888e5a26
by unintendedly.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
We can express the same idea more simply by not passing `user` in
cases where it isn't valid for UserActivity.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Fixes some in-app and documentation references to to new user
announcements and new stream announcements that were still using
'notifications' to refer them. These were missed in the original
pass for updating this language.
A user ran into an issue while upgrading where
ContentType.objects.get(model="realmreactivationstatus",
app_label="zerver") fails due to the object being missing. The reason
for that is to be yet figured out, but the immediate solution is clear
in the sense that the migration can just quit early
if not Confirmation.objects.filter(type=REALM_REACTIVATION).exists() and
that'll effectively skip it for almost all servers (because realm
reactivations links are something that's really only useful on Zulip
Cloud).
We are no longer creating confirmation objects associated with realms
directly. This should test for `RealmReactivationStatus` instead.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Newrelic updated the payload that's sent via the webhook incoming call
causing a bug in the newrelic webhook endpoint.
This fixes the bug by updating the endpoint to respect the new format
of the payload as well as the old format. This should be updated once
the old format is EOLed.
Fixes#22338.
This makes it mandatory to narrow the type of the user to `UserProfile`
before calling this helper.
This effectively removes the `request.user` check. We do not call login_page
anywhere else without getting through the authentication middleware.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
In the test case `test_check_if_every_integration_has_logo_that_exists`,
`urlsplit(integration.logo_url).path` gets inferred as possibly bytes
because `integration.logo_url` might be `None`.
5598b49851/stdlib/urllib/parse.pyi (L166-L169)
TODO:
We might want to ensure that every integration has a `logo_url` with an
explicit assertion in `Integrations` (as noted in the comment).
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
`expected_draft_contents` would be inferred as a list of mutable
mappings that only allow `int` as the value, and thus incompatible with
the `draft_dicts[i]` to be expanded. This is fixed by adding explicit
type annotation.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This was added in d43b031a32 and was
unused when it was added. This is an error that we want to remove.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
We likely just wanted to check that `validate_password` succeeds without
any exception being raised. A simple call is sufficient to verify that,
since `validate_password` does not return anything and raises an
exception on failure.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This check was added in 495a8476be.
Now that django-stubs finds that the left operand of the `and` will
always evaluates to `True`, so it makes sense to remove it.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
`report/error` is a path where we allow anonymous user access. This has
to be correctly denoted in the type annotation of the user argument of
the view function.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This fixes the type annotations of `Set` derived from `QuerySet` objects,
and add necessary assertions.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
`HostRequestMock` has `user` default to `None`, which later gets
initialized as `AnonymousUser`. The separate initialization here is
unnecessary.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Previously, we had a function named create_add_users_to_system_user_groups
for creating system user groups and adding users to them in case when
exports do not contain these groups when importing from other services.
This commit just separates out the call to create_system_user_groups_for_realm
outside the function and the function is thus renamed to
add_users_to_system_user_group. This change is done because in further
commits we would need to update the import order and user groups will
be created before creating user profile objects.
This commit extracts whether a stream is accessible or not
in a new function such that "Subscription" object is passed
by the caller and thus we can use these functions to check
access of multiple streams in a loop without querying the
database in a loop for subscription objects.
This commit renames admin_access_required parameter of
list_to_streams function to unsubscribing_others since that
parameter is used and passed as True only when calling
the function while unsubscribing others and in further
commits we would allow non-admins too to unsubscribe others
based on can_remove_subscribers_group setting.
This commit removes the instances of using "Stream.objects.create"
in tests with make_stream function. This change will help us to
avoid adding code for things to be done after creating streams in
multiple places. We can instead just add it in make_stream function
only.
The .status value of EmailChangeStatus was not being looked
at anywhere to prevent re-use of email change confirmation links. This
is not a security issue, since the EmailChangeStatus object has a fixed
value for the new_email, while the confirmation link has expiry time of
1 day, which prevents any reasonable malicious scenarios.
We fix this by making get_object_from_key look at
confirmation.content_object.status - which applies
generally to all confirmations where the attached object has the .status
attribute. This is desired, because we never want to
successfully get_object_from_key an object that has already been used or
reused.
This makes the prereg_user.status check in check_prereg_key redundant so
it can be deleted.
Type inference does not work when the default value of `REQ` is
non-optional while `ResultT` is optional. Mypy tries to unify
`json_validator` with `Validator[int]` in `invite_users_backend` instead
of the desired `Validator[Optional[int]]` because of the presence of the
default value `settings.INVITATION_LINK_VALIDITY_MINUTES`, which is
inferred to be an `int`. Mypy does not resort to a less specific type but
instead gives up early.
This issue applies to invite_users_backend and generate_multiuse_invite_backend
in zerver.views.invite.
There might be a way that we can add an overload to get around this, but
it's probably not worth the complexity until it comes up again more frequently.
We do in fact allow `invite_expires_in_minutes` to be `None` in places
like `do_invite_users`, `invite_users_backend`, etc, and we have
`settings.INVITATION_LINK_VALIDITY_MINUTES` as the default for them. So
it makes sense to allow having an optional value for this setting. And
since there isn't a way to independently set the value of this constant,
we move it to a different place.
TODO:
This is a temporary fix that should be refactored when the bug is fixed.
The encountered mypy issue: https://github.com/python/mypy/issues/13234
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Iterating over ValidatorError does not necessarily return a tuple. This
uses the `message_dict` property on `ValidationError` instead to make
sure that we always get a `dict` (it otherwise raises an `AttributeError`
when the `dict` is not available).
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
The `RateLimited` exception can be caught by `JsonErrorHandler`, so it
is not necessary to have the try...except statement here. It is also invalid
to pass a string to initialize `RateLimited`.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
`extra_data` as a `TextField` expects a `str`, but we had been passing
`dict` instead. This is a temporary solution before #18391 to fix the
type annotation.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This adds a `check_string_in` validator to ensure that `op` is actually
valid before we finally return `json_success()`.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Fixes#21266.
We want to tie the prereg_user to the MultiUseInvite directly rather
than to the MultiUserInvite's confirmation object, because the latter is
not possible. This is because the flow is that after going through the
multiuse invite link, the PreregistrationUser is created together with a
Confirmation object, creating a confirmation link (via
create_confirmation_link) to which then the user is redirected to finish
account creation. This means that the PreregistrationUser is already
tied to a Confirmation, so that attribute is occupied.
A standard OpenAPI document has no reason to redundantly include this
information in description fields, as standard generators already
display it.
This uniformly moves the URL above the description, which seems fine.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
The shared fields of `RawUserInfoDict` and `UserInfoDict` could have
been reused if they both require all keys or none. This is unfortunately
not the case, because subclassing does not override `__total__`.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Since we in fact are using the django test client to generate a response
here, the return type should be `TestHttpResponse` instead.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
`_callback_str` was removed in Django in 1.10, and other logic relevant
to that particular attribute was removed in
32849b80ad, but not to its entirety. It
does not make sense to fall back to `_callback_str`. The
`get_callback_string` helper is no longer needed.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
On registration and login pages on self-hosted Zulip servers,
it is not helpful and confusing to show the full navigation footer
for the Zulip website. Instead, we should show a minimal footer.
Fixes#21776
Due to an incorrect authorization check in Zulip Server 5.4 and
earlier, a member of an organization could craft an API call that
grants organization administrator privileges to one of their bots.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Previously if `test_forward_address_details` failed, the file
created when setting the `forward_address` may not have been
removed, which would then cause an `EmailNotDeliveredException`
to be raised when then creating a new user in the dev environment.
Wraps the test in a try block, with a finally block for the call
to remove the file.
Before this, a link still couldn't be re-used because it would trip up
exception further down user creation codepaths, but that was still a
bug. check_prereg_key is supposed to correctly validate the key - and
trigger an error page being returned if a key (or for any other reason,
the attached PreregistrationUser object) is reused.
test_validate_email_not_already_in_realm needs to be adjusted, because
it was actually re-using a key.
This reverts commit 40fcf5a633.
This commit triggers bug that we haven't fully tracked down, where web
app clients will continually send `update_message_flags` requests,
that then send out via the events system "0 messages were marked as
read" notices, eventually leading to a load spike.
The Tornado part can likely be fixed by checking if
updated_message_ids is empty, but we need to track down the frontend
bug as well.
`_cache` is not an attribute defined on `BaseCache`, but an
implementation detail of django_bmemcache.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Two endpoints had linked markdown files that were used in
their general descriptions to add warning notes with important
information (`/create-user` and `/get-user-groups`).
This moves the warning content to be inline in the endpoint
descriptions so that the important content is in the OpenAPI
documentation and is still formatted to be rendered in a warning
block.
Deletes `can-create-users-only.md` and `api-members-only.md`
since they were only used for these two endpoint descriptions.
Also, cleans up the other instance of a inline warning block in
an endpoint description (`/fetch-api-key`).
Instead of using `request.POST` to access the `data` parameter used
in the internal `notify_tornado` path, adds `has_request_variables`
decorator and accesses `data` as a `REQ` parameter.
Expands `test_tornado_endpoint` in `test_event_system.py` for
`data` being a required parameter for this path.
Instead of using `request.POST` to access `forward_address` for
the parameter used in `set_forward_address` in `email_page`, adds
`has_request_variable` decorator and an optional `forward_address`
parameter through the `REQ` framework.
Adds an assertion that `forward_address` is not `None` for `POST`
requests.
Anders Kaseorg
Zixuan James Li
Lauryn Menard
Mateusz Mandera
rodwan-bakkar
Adam Sah
Sahil Batra
Aman Agrawal
Christopher Chong
Lauryn Menard
Tim Abbott