Commit Graph

15635 Commits

Author SHA1 Message Date
Sahil Batra 2713dd4e78 message_edit: Separate permission checks for topic edits.
We separate the permission checks for content and topic edits
by changing the can_edit_topic_or_content to can_edit_topic
and use it only for checking topic edits and check content
edits separately in check_update_message itself. There is no
change in behavior as of this PR, there will be more changes
as per #21739.

This is a prep commit for #21739. The permission checks for
them are essentially separate except the one that message
sender is allowed to edit content and topic irresepctive of
edit_topic_policy setting, and this will too be changed in
future commit and so it will be better to have these checks
separate for readability.

We can also probably create a new function for checking content
edits but currently we only check the sender is same as the use
who is editing and it does not make sense to have a separate
function for just one check. We can do so in future in case we
do some more refactoring for #21739.
2022-08-03 15:59:51 -07:00
Zixuan James Li 77eef5b1ad types: Add OIDCIdPConfigDict.
The presence of `auto_signup` in idp_settings_dict in the test case
test_social_auth_registration_auto_signup is incompatible with the
previous type annotation of SOCIAL_AUTH_OIDC_ENABLED_IDPS, where `bool`
is not allowed.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-03 15:03:15 -07:00
Zixuan James Li ae949bce04 test_auth_backends: Return an Iterable for idp names.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-03 15:03:15 -07:00
Zixuan James Li fdd0d18f8b types: Add auto_signup to SAMLIdPConfigDict.
In 3b4f8cc85b,
we added support to `auto_signup`, but this field was not defined in
`SAMLIdPConfigDict`, causing mypy type error in
`SAMLAuthenBackendTest.test_social_auth_registration_auto_signup`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-03 15:03:15 -07:00
Mateusz Mandera e379d34043 create_system_user_groups_for_realm: Tweak docstring.
The migration number is 0382. 0375 must have been the number before
rebasing and didn't get updated in the docstring.
2022-08-01 13:58:33 -07:00
Zixuan James Li 6ee0a979f3 import_util: Post-modify date fields with float values.
We construct model instances in the import tool solely for the purpose
of serializing them with the `model_to_dict` helper that returns a
dictionary. Passing `float` to these models' DateTimeField is not
accepted by the type checker. Modifying the dictionary instead avoids
this typing issue.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-01 13:58:12 -07:00
Sahil Batra f2a3236b42 actions: Use transaction.atomic for do_remove_realm_emoji. 2022-07-31 18:32:28 -07:00
Sahil Batra 9d9252ef08 realm: Create RealmAuditLog entries when removing custom emoji.
We also add acting_user parameter to do_remove_realm_emoji in
this commit, so that we can use it while creating RealmAuditLog
objects.
2022-07-31 18:32:28 -07:00
Sahil Batra 90898cc9c4 realm: Create RealmAuditLog entries when adding custom emoji. 2022-07-31 18:32:28 -07:00
Sahil Batra 1307976d34 actions: Pass realm_emoji dict to notify_realm_emoji.
We pass the realm_emoji dict to notify_realm_emoji instead
of computing it in notify_realm_emoji itself, because its
caller functions will require the dict in further commits
for creating RealmAuditLog entries.
2022-07-31 18:32:28 -07:00
Sahil Batra e64879e68a actions: Use transaction.atomic for do_remove_linkifier. 2022-07-31 18:32:28 -07:00
Sahil Batra fd6dc754d9 actions: Use transaction.atomic for do_update_linkifier. 2022-07-31 18:32:28 -07:00
Sahil Batra 34b65b9942 actions: Use transaction.atomic for do_add_linkifier.
This commit also renames the legacy event variable to
legacy_event from event because same event is sent
twice if we keep only single variable and change its
value. This was happening because the event value was
changed when the first send_event was actually executed.
2022-07-31 18:32:28 -07:00
Sahil Batra ef48047dfc realm: Create RealmAuditLog entry when removing realm linkifiers.
This commit also adds 'acting_user' parameter to do_remove_linkifier
function.

Fixes a part of #21268.
2022-07-31 18:32:28 -07:00
Sahil Batra 4d62915912 realm: Create RealmAuditLog entry when updating realm linkifiers.
This commit also adds 'acting_user' parameter to do_update_linkifier
function.

Fixes a part of #21268.
2022-07-31 18:32:28 -07:00
Sahil Batra 4f1dd0710c realm: Create RealmAuditLog entry when adding realm linkifiers.
This commit also adds 'acting_user' parameter to do_add_linkifier
function.

Fixes a part of #21268.
2022-07-31 18:32:28 -07:00
Sahil Batra ef9c52011d realm_linkifiers: Pass realm linkifiers list to notify_linkifiers.
We pass the realm_linkfiers list to notify_linkifiers instead of
computing it in notify_linkifiers itself, because its caller
functions will require the list in further commits for creating
RealmAuditLog entries.
2022-07-31 18:32:28 -07:00
Sahil Batra d2f2fbf6b9 realm: Disable enable_spectator_access on changing to limited plan.
Since we not allow enabling public access on limited plan realms,
we set the enable_spectator_access setting to False when downgrading
to a limited plan. Setting is still shown in the UI but it is
disabled.
2022-07-31 18:02:36 -07:00
Sahil Batra f4fcedd072 streams: Send stream creation event when changing stream to public.
This commit adds code to send stream creation and peer add events
when stream is changed from private to public. These events are
only sent to users who are not susbcribed to the stream and are
not realm admins as subscribers and realm admins already have
the stream data. This will update the stream data with clients
and will remove the need to reload to view the modified stream.

Fixes #22194.
2022-07-31 17:33:53 -07:00
Sahil Batra 319ff6eba2 events: Update stream objects in "unsubscribed" and "never_subscribed" lists. 2022-07-31 17:33:53 -07:00
Zixuan James Li 58d1be8085 decorator: Replace ViewFuncT with ParamSpec.
`ParamSpec` can be easily applied to many use cases of ViewFuncT with
`Concatenate` to help us get rid of the `cast` calls. This does not
include decorators that require the second argument being
`UserProfile`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-30 18:15:33 -07:00
Zixuan James Li 7cbc1ab7d4 decorator: Refactor zulip_login_required to use ParamSpec.
As we refactor this, any decorators that `zulip_login_required` depends
on are also refactored to use `ParamSpec`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-30 18:15:33 -07:00
Zixuan James Li 95394de186 decorator: Refactor require_server_admin_api with ParamSpec.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-30 18:15:33 -07:00
Zixuan James Li 9f2dfd848f message_send: Add service_queue_events to SendMessageRequest.
We should not monkey-patch message when unnecessary. Adding
`service_queue_events` to `SendMessageRequests` suits our need to type
safety here.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-30 18:03:56 -07:00
Anders Kaseorg b4b8691239 retention: Inline move_rows query arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-30 06:46:34 -07:00
Anders Kaseorg 2508b579a6 upload: Replace boto3.Session with boto3.session.Session.
boto3-stubs seems to have dropped the former for some reason.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-30 06:46:34 -07:00
Anders Kaseorg b945aa3443 python: Use a real parser for email addresses.
Now that we can assume Python 3.6+, we can use the
email.headerregistry module to replace hacky manual email address
parsing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-29 15:47:33 -07:00
Anders Kaseorg 8c2d478e6a Revert "test_runner: Monkey-patch to allow using multiprocessing in tests."
This reverts commit 73eb0aa891.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-29 15:27:09 -07:00
Anders Kaseorg fa740c0435 transfer: Migrate from multiprocessing to ProcessPoolExecutor.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-29 15:27:09 -07:00
Anders Kaseorg 25cf0319e8 import_realm: Migrate from multiprocessing to ProcessPoolExecutor.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-29 15:27:09 -07:00
Anders Kaseorg 7b4cfcddb3 import_util: Migrate from multiprocessing to ProcessPoolExecutor.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-29 15:27:09 -07:00
Zixuan James Li e68fb802f4 upload: Replace File with UploadedFile.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-29 14:09:12 -07:00
Zixuan James Li f42465319b upload: Refactor file size out of get_file_info.
We have already checked the size of the file in `upload_file_backend`.
This is the only caller of `upload_message_image_from_request`, and
indirectly the only caller of `get_file_info`. There is no need to
retrieve this information again.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-29 14:09:12 -07:00
Zixuan James Li bdee079b0d test_logging_handlers: Use delattr instead to unset request.user.
It is not possible in the codebase to have request.user be None. But
it is possible to have it not present at all. `delattr` is more
appropriate here.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-29 14:07:41 -07:00
Zixuan James Li 16b51ef437 test_logging_handlers: Mock get_host instead of replacing it.
This makes the test cleaner and we don't have to overwrite the `get_host`
callable on `HttpRequest`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-29 14:07:41 -07:00
Zixuan James Li fd8f049a8d typing: Avoid redefinition of variables with a different type.
`context` as `AccessDeniedError` is incompatible with
`RequestVariableMissingError`. Mypy does not allow such redefinition.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-29 13:59:30 -07:00
Zixuan James Li c86b3fd793 test_decorators: Call process_request as an instance method.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-29 13:59:30 -07:00
Zixuan James Li 62cbacbfe7 test_runner: Write failed tests with a side effect.
`DiscoverRunner.run_tests` has a return type of `int`. While
`Runner.run_tests` has a wildly different `Tuple[bool, List[str]]`.

This refactors it so that we have the correct return type, by passing
the additional information about failed tests through a side effect to directly
write the failed tests to a file.

Note that we have to make `failed_tests_path` optional as otherwise the method
signature will not be compatible with the supertype.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-29 13:59:30 -07:00
Zixuan James Li 8b29b37227 test_runner: Replace django.test.TestCase with unittest.TestCase.
The supertype uses unittest.TestCase. We conform to that for
type compatibility.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-29 13:59:29 -07:00
Anders Kaseorg c68c07619f test_runner: Remove _destroy_test_db monkey patch.
This reverts part of commit 1432067959
(#17047).  The spooky warnings foretold by the comment don’t seem to
show up.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-29 13:56:47 -07:00
Zixuan James Li 04d4c6a453 test_decorators: Avoid redefinition of variables with a different type.
`context` as `AccessDeniedError` is incompatible with
`RequestVariableMissingError`. Mypy does not allow such redefinition.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-28 10:26:32 -07:00
Zixuan James Li ba21925b49 test_signup: Add None check for prereg_user.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-28 10:26:32 -07:00
Zixuan James Li f3b2e2b541 test_email_mirror: Add None check for settings.SHARED_SECRET.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-28 10:26:32 -07:00
Zixuan James Li a996228ced test_auth_backends: Add None check for prereg_user.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-28 10:26:31 -07:00
Zixuan James Li c2342a5ece test_runner: Fix type annotation of get_tests_from_suite.
It is really a generator of test cases from the test suite. Which should
be typed as an `Iterable` instead.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-28 09:48:16 -07:00
Zixuan James Li 3ad9876d69 test_signup: Use getattr to access potentially not presenting fields.
`context_data` is only available on `SimpleTemplateResposne`, we can't
narrow `TestHttpResponse` to it because the latter is not in fact a
subtype of `HttpResponse`.

Differently, `redirect_chain` is an attribute that only appears on the
test response when the test client method is called with `follow=True`.
`TestHttpResponse` does not have that by defalut, either.

The occurence of these two cases are rare enough throughout the codebase
and we can't get around that without aggressively overloading the test client
or refactoring `_MonkeyPatchedWSGIResponse` in the upstream.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-28 09:48:16 -07:00
Zixuan James Li 0f416c44d0 test_rate_limiter: Define backend on RateLimiterBackendBase.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-28 09:48:16 -07:00
Zixuan James Li b02779c005 request: Refactor remote_server into RequestNotes.
This eliminates the possibility of having `request.user` as
`RemoteZulipServer` by refactoring it as an attribute of `RequestNotes`.

So we can effectively narrow the type of `request.user` by testing
`user.is_authenticated` in most cases (except that of `SCIMClient`) in
code paths that require access to `.format_requestor_for_logs` where we
previously expect either `UserProfile` or `RemoteZulipServer` backed by
the implied polymorphism.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-28 09:38:40 -07:00
Zixuan James Li 3bc78d2473 decorator: Do not set remote_server.rate_limits.
In b46af40bd3,
we set this attribute because back then we might call `rate_limit_user`
on `RemoteZulipServer`.

This is no longer the case as `RemoteZulipServer` now has its own rate
limiting and we never call `rate_limit_user` without an `isinstance` check
for `UserProfile`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-28 09:38:40 -07:00
Zixuan James Li 42fddd5a1e test_middlware: Add test coverage for request_for_logs.
This verifies that `request_for_logs` is correctly set for requests
with different types of authentication.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-28 09:38:40 -07:00
Zixuan James Li c7a0d5d273 test_helpers: Remove noop.
`BaseNotes(str, str).get_notes` does not do anything here.
It was introduced in
53888e5a26
by unintendedly.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-28 09:38:40 -07:00
Zixuan James Li 789b66ff3b decorator: Remove unnecessary flag for process_client.
We can express the same idea more simply by not passing `user` in
cases where it isn't valid for UserActivity.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-28 09:38:21 -07:00
Lauryn Menard b89e0f6771
settings: Fix references to new user and stream announcements.
Fixes some in-app and documentation references to to new user
announcements and new stream announcements that were still using
'notifications' to refer them. These were missed in the original
pass for updating this language.
2022-07-28 09:33:09 -07:00
Mateusz Mandera f04acc6040 migration: Tweak migration 0401 to exit early on most servers.
A user ran into an issue while upgrading where
ContentType.objects.get(model="realmreactivationstatus",
app_label="zerver") fails due to the object being missing. The reason
for that is to be yet figured out, but the immediate solution is clear
in the sense that the migration can just quit early
if not Confirmation.objects.filter(type=REALM_REACTIVATION).exists() and
that'll effectively skip it for almost all servers (because realm
reactivations links are something that's really only useful on Zulip
Cloud).
2022-07-28 09:30:56 -07:00
Zixuan James Li d1acd67897 test_realm: Fix realm confirmation object test case.
We are no longer creating confirmation objects associated with realms
directly. This should test for `RealmReactivationStatus` instead.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-27 17:07:11 -07:00
Mateusz Mandera 43213ea37b realm_reactivation: Migrate old Confirmations to the new data format.
cf74d7d140 changed what .content_object on
these Confirmations is, but old Confirmations still need to be migrated
to that to make sense.
2022-07-27 17:06:19 -07:00
rodwan-bakkar bfd9fc8622 integration: Fix integration with newrelic.
Newrelic updated the payload that's sent via the webhook incoming call
causing a bug in the newrelic webhook endpoint.
This fixes the bug by updating the endpoint to respect the new format
of the payload as well as the old format. This should be updated once
the old format is EOLed.

Fixes #22338.
2022-07-27 16:15:29 -07:00
Zixuan James Li 06d3f3cf64 2fa: Refactor is_2fa_verified to require type narrowing.
This makes it mandatory to narrow the type of the user to `UserProfile`
before calling this helper.

This effectively removes the `request.user` check. We do not call login_page
anywhere else without getting through the authentication middleware.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-27 14:28:31 -07:00
Adam Sah c6931434e9 email_error: Improve tests for inbound email edge case.
Tests to assist with clarifying #22585.
2022-07-27 13:53:16 -07:00
Zixuan James Li 01d3df0551 storage: Fix type annotation of content.
Currently django-stubs expects `File` to be generic, this is not yet
supported. We quote it for now before django-stubs gets integrated.

TODO: unquote this in the future.

See also: https://github.com/typeddjango/django-stubs/issues/1061#issuecomment-1185995937.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-27 13:46:13 -07:00
Zixuan James Li 07eccbde97 test_docs: Add assertion check.
In the test case `test_check_if_every_integration_has_logo_that_exists`,
`urlsplit(integration.logo_url).path` gets inferred as possibly bytes
because `integration.logo_url` might be `None`.

5598b49851/stdlib/urllib/parse.pyi (L166-L169)

TODO:
We might want to ensure that every integration has a `logo_url` with an
explicit assertion in `Integrations` (as noted in the comment).

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li 4e240b880e migrations: Define emoji_to_lowercase outside.
The `self` argument should otherwise present.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li 90a31d9e66 templates: Narrow the type of jinjia to Jinja2.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li 223a1ad0eb test_auth_backends: Add None checks.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li 51df4031ac test_auth_backends: Extract external_auth_backends.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li d3f976a0a3 message_flags: Add an assertion check for stream.recipient_id.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li 7233ad454f auth: Add an assertion check for context_data.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li d238a448e6 test_draft: Avoid inference with type annotation.
`expected_draft_contents` would be inferred as a list of mutable
mappings that only allow `int` as the value, and thus incompatible with
the `draft_dicts[i]` to be expanded. This is fixed by adding explicit
type annotation.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li 56e22e1a92 forms: Fix to_python method signature.
According to the documentation, `to_python` should account for `None`
being a possible value to be processed.

See also: https://docs.djangoproject.com/en/4.0/howto/custom-model-fields/#converting-values-to-python-objects.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li 7bf58dbede test_auth_backends: Remove unused self.user_profile.backend.
This was added in d43b031a32 and was
unused when it was added. This is an error that we want to remove.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li 4493f74507 models: Move the assertion checking bot_owner_id.
Both code blocks need to have an assertion ensuring that `bot_owner_id`
is not `None`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li 11a23aa5ef test_audit_log: Fix an assertion for a function that never returns.
We likely just wanted to check that `validate_password` succeeds without
any exception being raised. A simple call is sufficient to verify that,
since `validate_password` does not return anything and raises an
exception on failure.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li 4675cbfefd test_message_flags: Remove unused comma.
This was accidentally added in 20a97bdb05,
likely due to some typos.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li b0c5db1c22 test_realm_emoji: Remove redundant assertion.
This check was added in 495a8476be.
Now that django-stubs finds that the left operand of the `and` will
always evaluates to `True`, so it makes sense to remove it.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li 3ba51ef1e2 queue_processor: Fix type annotation for connection.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li aa733f0bbd webhook: Fix f-string str-bytes-safe error.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li da326aedff report: Correct type annotation allowing unauth access.
`report/error` is a path where we allow anonymous user access. This has
to be correctly denoted in the type annotation of the user argument of
the view function.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li 97fd662d48 test_import_export: Enhance typing of getters.
This fixes the type annotations of `Set` derived from `QuerySet` objects,
and add necessary assertions.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li 2925615ee7 test_decorators: Use the default value of user.
`HostRequestMock` has `user` default to `None`, which later gets
initialized as `AnonymousUser`. The separate initialization here is
unnecessary.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li 583116db14 email_notifications: Fix type annotation for tuples.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li f314171a7b streams: Add isinstance check for merge_streams.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li 6f42b1c75c uploads: Avoid redefinition of a variable.
Mypy disallows redefinition of the same variable with a different type.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Sahil Batra e83ea8b6a7 import: Call create_system_user_groups_for_realm outside the function.
Previously, we had a function named create_add_users_to_system_user_groups
for creating system user groups and adding users to them in case when
exports do not contain these groups when importing from other services.

This commit just separates out the call to create_system_user_groups_for_realm
outside the function and the function is thus renamed to
add_users_to_system_user_group. This change is done because in further
commits we would need to update the import order and user groups will
be created before creating user profile objects.
2022-07-26 17:36:53 -07:00
Sahil Batra 58fdcdb00c streams: Extract stream access checks in a function.
This commit extracts whether a stream is accessible or not
in a new function such that "Subscription" object is passed
by the caller and thus we can use these functions to check
access of multiple streams in a loop without querying the
database in a loop for subscription objects.
2022-07-26 17:25:52 -07:00
Sahil Batra 52c8f687cc streams: Rename admin_access_required to unsubscribing_others.
This commit renames admin_access_required parameter of
list_to_streams function to unsubscribing_others since that
parameter is used and passed as True only when calling
the function while unsubscribing others and in further
commits we would allow non-admins too to unsubscribe others
based on can_remove_subscribers_group setting.
2022-07-26 17:25:52 -07:00
Sahil Batra bcef35490d test_markdown: Use make_stream function for creating streams.
This commit removes the instances of using "Stream.objects.create"
in tests with make_stream function. This change will help us to
avoid adding code for things to be done after creating streams in
multiple places. We can instead just add it in make_stream function
only.
2022-07-26 17:25:52 -07:00
Mateusz Mandera cf74d7d140 realm_reactivation: Prevent realm reactivation link reuse.
This uses the approach analogical to EmailChangeStatus for email change
confirmation links.
2022-07-26 17:14:26 -07:00
Mateusz Mandera 46c6f33b10 reactivate_realm: Change error status code on invalid links to 404. 2022-07-26 17:14:26 -07:00
Mateusz Mandera 0e2691815e confirmation: Prevent re-use of email change links.
The .status value of EmailChangeStatus was not being looked
at anywhere to prevent re-use of email change confirmation links. This
is not a security issue, since the EmailChangeStatus object has a fixed
value for the new_email, while the confirmation link has expiry time of
1 day, which prevents any reasonable malicious scenarios.

We fix this by making get_object_from_key look at
confirmation.content_object.status - which applies
generally to all confirmations where the attached object has the .status
attribute. This is desired, because we never want to
successfully get_object_from_key an object that has already been used or
reused.
This makes the prereg_user.status check in check_prereg_key redundant so
it can be deleted.
2022-07-26 17:14:26 -07:00
Mateusz Mandera 9992c7b6cc test_email_change: Extract generate_email_change_link helper. 2022-07-26 17:14:26 -07:00
Zixuan James Li fe9ed2e69d settings: Make INVITATION_LINK_VALIDITY_MINUTES optional.
Type inference does not work when the default value of `REQ` is
non-optional while `ResultT` is optional. Mypy tries to unify
`json_validator` with `Validator[int]` in `invite_users_backend` instead
of the desired `Validator[Optional[int]]` because of the presence of the
default value `settings.INVITATION_LINK_VALIDITY_MINUTES`, which is
inferred to be an `int`. Mypy does not resort to a less specific type but
instead gives up early.

This issue applies to invite_users_backend and generate_multiuse_invite_backend
in zerver.views.invite.

There might be a way that we can add an overload to get around this, but
it's probably not worth the complexity until it comes up again more frequently.

We do in fact allow `invite_expires_in_minutes` to be `None` in places
like `do_invite_users`, `invite_users_backend`, etc, and we have
`settings.INVITATION_LINK_VALIDITY_MINUTES` as the default for them. So
it makes sense to allow having an optional value for this setting. And
since there isn't a way to independently set the value of this constant,
we move it to a different place.

TODO:

This is a temporary fix that should be refactored when the bug is fixed.

The encountered mypy issue: https://github.com/python/mypy/issues/13234

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 17:07:48 -07:00
Anders Kaseorg b17affc3da storage: Use Django 4.0 manifest_storage option.
https://code.djangoproject.com/ticket/27590
https://docs.djangoproject.com/en/4.0/ref/contrib/staticfiles/#manifeststaticfilesstorage

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-26 17:06:02 -07:00
Zixuan James Li 3104a7ea94 exceptions: Guard validation error conversion with message_dict.
Iterating over ValidatorError does not necessarily return a tuple. This
uses the `message_dict` property on `ValidationError` instead to make
sure that we always get a `dict` (it otherwise raises an `AttributeError`
when the `dict` is not available).

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 14:17:46 -07:00
Zixuan James Li 43106fb6ce avatar: Remove unnecessary try...except statement.
The `RateLimited` exception can be caught by `JsonErrorHandler`, so it
is not necessary to have the try...except statement here. It is also invalid
to pass a string to initialize `RateLimited`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 09:50:31 -07:00
Zixuan James Li ad17096c9c realm_audit_log: Explicitly stringify dict before insertion.
`extra_data` as a `TextField` expects a `str`, but we had been passing
`dict` instead. This is a temporary solution before #18391 to fix the
type annotation.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 09:48:33 -07:00
Sahil Batra 6287b87209 models: Remove unused "hidden_for_sponsorship" from ORG_TYPES.
We do not use "hidden_for_sponsorship" currently as it was
removed in d7ef0c7232.
2022-07-25 16:53:37 -07:00
Zixuan James Li 44f5c1cb33 muting: Add validation for update operations.
This adds a `check_string_in` validator to ensure that `op` is actually
valid before we finally return `json_success()`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-25 14:03:08 -07:00
Zixuan James Li cbaa4bd98c user_groups: Add MemberGroupUserDict.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-25 13:46:57 -07:00
Mateusz Mandera 39d8a81e51 registration: Tie PreregistrationUser to the original MultiUseInvite.
Fixes #21266.

We want to tie the prereg_user to the MultiUseInvite directly rather
than to the MultiUserInvite's confirmation object, because the latter is
not possible. This is because the flow is that after going through the
multiuse invite link, the PreregistrationUser is created together with a
Confirmation object, creating a confirmation link (via
create_confirmation_link) to which then the user is redirected to finish
account creation. This means that the PreregistrationUser is already
tied to a Confirmation, so that attribute is occupied.
2022-07-22 17:08:44 -07:00
Anders Kaseorg 2039aed821 openapi: Move endpoint URL to generator.
A standard OpenAPI document has no reason to redundantly include this
information in description fields, as standard generators already
display it.

This uniformly moves the URL above the description, which seems fine.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-22 16:41:55 -07:00