Commit Graph

7189 Commits

Author SHA1 Message Date
N-Shar-ma ef044b8697 markdown: Update characters allowed before @ and stream mentions.
Now the following characters are allowed before @-mentions and stream
references (starting with #) for proper rendering - {, [, /.

This commit makes the markdown rendering consistent with autocomplete
(anything that is autocompleted is also rendered properly).
2022-08-06 19:29:39 -07:00
Zixuan James Li 5ad515c560 onboarding: Use dictionary comprehension for dict initialization.
Initializing a dictionary from an iterable requires the each item to be
a tuple containg a key and a value. `mypy_django_plugin` cannot infer
the number of items in an queryset with annotated values, so we have to
explicitly unpack each row with a dictionary comprehension here.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-06 16:21:12 -07:00
Zixuan James Li adae8b6d42 request: Refactor has_request_variables with ParamSpec.
This makes `has_request_variables` more generic, in the sense of the return
value, and also makes it more accurate, in the sense of requiring the
first parameter of the decorated function to be `HttpRequest`, and
preserving the function signature without using `cast`.

This affects some callers of `has_request_variables` or the callers of its
decoratedfunctions in the following manners:

- Decorated non-view functions called directly in other functions cannot
use `request` as a keyword argument. Becasue `Concatenate` turns the
concatenated parameters (`request: HttpRequest` in this case) into
positional-only parameters. Callers of `get_chart_data` are thus
refactored.

- Functions to be decorated that accept variadic keyword arguments must
define `request: HttpRequest` as positional-only. Mypy in strict mode
rejects such functions otherwise because it is possible for the caller to
pass a keyword argument that has the same name as `request` for `**kwargs`.
No defining `request: HttpRequest` as positional-only breaks type safety
because function with positional-or-keyword parameters cannot be considered
a subtype of a function with the same parameters in which some of them are
positional-only.

Consider `f(x: int, /, **kwargs: object) -> int` and `g(x: int,
**kwargs: object) -> int`. `f(12, x="asd")` is valid but `g(12, x="asd")`
is not.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-06 16:19:48 -07:00
Zixuan James Li 9f99e6c43c cache: Fix type annotation for session_cache_items.
This adds an assertion ensuring the type of `store` before accessing the
`cache_key` attribute that does not exist in the base class. Also note
that `.decode` returns `Dict[str, Any]` instead of a `str`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-06 16:16:17 -07:00
Kartik Srivastava 1291e7000b user_topic: Add user_topic event.
We now send a new user_topic event while muting and unmuting topics.
fetch_initial_state_data now returns an additional user_topics array to
the client that will maintain the user-topic relationship data.
This will support any future addition of new features to modify the
relationship between a user-topic pair.

This commit adds the relevent backend code and schema for the new
event.
2022-08-04 17:44:00 -07:00
Kartik Srivastava 8b674ee3d7 user_topic: Use get_user_topics to build muted topics tuples.
This adds get_user_topics helper method and refactors get_muted_topics
to use it to build muted topics tuples.
2022-08-04 17:44:00 -07:00
Kartik Srivastava 6d514c7f78 markdown: Fix rendering of nested objects in API return values.
`render_table` calls itself recursively when it finds nested
`additionalProperties` (i.e. nested objects) in response schema,
to render their properties.

This fixes `render_table` to call `render_desc` along with
calling itself, to render the description of the nested
`additionalProperties` as well.
2022-08-04 16:34:13 -07:00
Zixuan James Li 77eef5b1ad types: Add OIDCIdPConfigDict.
The presence of `auto_signup` in idp_settings_dict in the test case
test_social_auth_registration_auto_signup is incompatible with the
previous type annotation of SOCIAL_AUTH_OIDC_ENABLED_IDPS, where `bool`
is not allowed.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-03 15:03:15 -07:00
Zixuan James Li fdd0d18f8b types: Add auto_signup to SAMLIdPConfigDict.
In 3b4f8cc85b,
we added support to `auto_signup`, but this field was not defined in
`SAMLIdPConfigDict`, causing mypy type error in
`SAMLAuthenBackendTest.test_social_auth_registration_auto_signup`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-03 15:03:15 -07:00
Mateusz Mandera e379d34043 create_system_user_groups_for_realm: Tweak docstring.
The migration number is 0382. 0375 must have been the number before
rebasing and didn't get updated in the docstring.
2022-08-01 13:58:33 -07:00
Sahil Batra 319ff6eba2 events: Update stream objects in "unsubscribed" and "never_subscribed" lists. 2022-07-31 17:33:53 -07:00
Zixuan James Li 9f2dfd848f message_send: Add service_queue_events to SendMessageRequest.
We should not monkey-patch message when unnecessary. Adding
`service_queue_events` to `SendMessageRequests` suits our need to type
safety here.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-30 18:03:56 -07:00
Anders Kaseorg b4b8691239 retention: Inline move_rows query arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-30 06:46:34 -07:00
Anders Kaseorg 2508b579a6 upload: Replace boto3.Session with boto3.session.Session.
boto3-stubs seems to have dropped the former for some reason.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-30 06:46:34 -07:00
Anders Kaseorg b945aa3443 python: Use a real parser for email addresses.
Now that we can assume Python 3.6+, we can use the
email.headerregistry module to replace hacky manual email address
parsing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-29 15:47:33 -07:00
Anders Kaseorg 8c2d478e6a Revert "test_runner: Monkey-patch to allow using multiprocessing in tests."
This reverts commit 73eb0aa891.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-29 15:27:09 -07:00
Anders Kaseorg fa740c0435 transfer: Migrate from multiprocessing to ProcessPoolExecutor.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-29 15:27:09 -07:00
Anders Kaseorg 25cf0319e8 import_realm: Migrate from multiprocessing to ProcessPoolExecutor.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-29 15:27:09 -07:00
Zixuan James Li e68fb802f4 upload: Replace File with UploadedFile.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-29 14:09:12 -07:00
Zixuan James Li f42465319b upload: Refactor file size out of get_file_info.
We have already checked the size of the file in `upload_file_backend`.
This is the only caller of `upload_message_image_from_request`, and
indirectly the only caller of `get_file_info`. There is no need to
retrieve this information again.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-29 14:09:12 -07:00
Zixuan James Li 62cbacbfe7 test_runner: Write failed tests with a side effect.
`DiscoverRunner.run_tests` has a return type of `int`. While
`Runner.run_tests` has a wildly different `Tuple[bool, List[str]]`.

This refactors it so that we have the correct return type, by passing
the additional information about failed tests through a side effect to directly
write the failed tests to a file.

Note that we have to make `failed_tests_path` optional as otherwise the method
signature will not be compatible with the supertype.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-29 13:59:30 -07:00
Zixuan James Li 8b29b37227 test_runner: Replace django.test.TestCase with unittest.TestCase.
The supertype uses unittest.TestCase. We conform to that for
type compatibility.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-29 13:59:29 -07:00
Anders Kaseorg c68c07619f test_runner: Remove _destroy_test_db monkey patch.
This reverts part of commit 1432067959
(#17047).  The spooky warnings foretold by the comment don’t seem to
show up.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-29 13:56:47 -07:00
Zixuan James Li c2342a5ece test_runner: Fix type annotation of get_tests_from_suite.
It is really a generator of test cases from the test suite. Which should
be typed as an `Iterable` instead.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-28 09:48:16 -07:00
Zixuan James Li b02779c005 request: Refactor remote_server into RequestNotes.
This eliminates the possibility of having `request.user` as
`RemoteZulipServer` by refactoring it as an attribute of `RequestNotes`.

So we can effectively narrow the type of `request.user` by testing
`user.is_authenticated` in most cases (except that of `SCIMClient`) in
code paths that require access to `.format_requestor_for_logs` where we
previously expect either `UserProfile` or `RemoteZulipServer` backed by
the implied polymorphism.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-28 09:38:40 -07:00
Zixuan James Li c7a0d5d273 test_helpers: Remove noop.
`BaseNotes(str, str).get_notes` does not do anything here.
It was introduced in
53888e5a26
by unintendedly.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-28 09:38:40 -07:00
Zixuan James Li 06d3f3cf64 2fa: Refactor is_2fa_verified to require type narrowing.
This makes it mandatory to narrow the type of the user to `UserProfile`
before calling this helper.

This effectively removes the `request.user` check. We do not call login_page
anywhere else without getting through the authentication middleware.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-27 14:28:31 -07:00
Zixuan James Li 01d3df0551 storage: Fix type annotation of content.
Currently django-stubs expects `File` to be generic, this is not yet
supported. We quote it for now before django-stubs gets integrated.

TODO: unquote this in the future.

See also: https://github.com/typeddjango/django-stubs/issues/1061#issuecomment-1185995937.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-27 13:46:13 -07:00
Zixuan James Li 07eccbde97 test_docs: Add assertion check.
In the test case `test_check_if_every_integration_has_logo_that_exists`,
`urlsplit(integration.logo_url).path` gets inferred as possibly bytes
because `integration.logo_url` might be `None`.

5598b49851/stdlib/urllib/parse.pyi (L166-L169)

TODO:
We might want to ensure that every integration has a `logo_url` with an
explicit assertion in `Integrations` (as noted in the comment).

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li 90a31d9e66 templates: Narrow the type of jinjia to Jinja2.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li 583116db14 email_notifications: Fix type annotation for tuples.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Sahil Batra e83ea8b6a7 import: Call create_system_user_groups_for_realm outside the function.
Previously, we had a function named create_add_users_to_system_user_groups
for creating system user groups and adding users to them in case when
exports do not contain these groups when importing from other services.

This commit just separates out the call to create_system_user_groups_for_realm
outside the function and the function is thus renamed to
add_users_to_system_user_group. This change is done because in further
commits we would need to update the import order and user groups will
be created before creating user profile objects.
2022-07-26 17:36:53 -07:00
Sahil Batra 58fdcdb00c streams: Extract stream access checks in a function.
This commit extracts whether a stream is accessible or not
in a new function such that "Subscription" object is passed
by the caller and thus we can use these functions to check
access of multiple streams in a loop without querying the
database in a loop for subscription objects.
2022-07-26 17:25:52 -07:00
Sahil Batra 52c8f687cc streams: Rename admin_access_required to unsubscribing_others.
This commit renames admin_access_required parameter of
list_to_streams function to unsubscribing_others since that
parameter is used and passed as True only when calling
the function while unsubscribing others and in further
commits we would allow non-admins too to unsubscribe others
based on can_remove_subscribers_group setting.
2022-07-26 17:25:52 -07:00
Mateusz Mandera cf74d7d140 realm_reactivation: Prevent realm reactivation link reuse.
This uses the approach analogical to EmailChangeStatus for email change
confirmation links.
2022-07-26 17:14:26 -07:00
Anders Kaseorg b17affc3da storage: Use Django 4.0 manifest_storage option.
https://code.djangoproject.com/ticket/27590
https://docs.djangoproject.com/en/4.0/ref/contrib/staticfiles/#manifeststaticfilesstorage

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-26 17:06:02 -07:00
Zixuan James Li 3104a7ea94 exceptions: Guard validation error conversion with message_dict.
Iterating over ValidatorError does not necessarily return a tuple. This
uses the `message_dict` property on `ValidationError` instead to make
sure that we always get a `dict` (it otherwise raises an `AttributeError`
when the `dict` is not available).

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 14:17:46 -07:00
Anders Kaseorg 946a0565c6 openapi: Fuse generate_api_title with generate_api_description.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-22 16:41:55 -07:00
Zixuan James Li 8ae838c5c8 users: Remove default values for add_service.
These default values are unused by the callers and incompatible with the
`Service` model.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-22 16:34:32 -07:00
Zixuan James Li ebfd2b25b1 user_status: Add UserInfoDict.
The shared fields of `RawUserInfoDict` and `UserInfoDict` could have
been reused if they both require all keys or none. This is unfortunately
not the case, because subclassing does not override `__total__`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-22 16:31:18 -07:00
Zixuan James Li 0dfec6b132 templates: Use Dict instead of Mapping for the context parameter.
According to the Django documentation, `Template.render` expects a
`dict`.

See also: https://docs.djangoproject.com/en/4.0/topics/templates/#django.template.backends.base.Template.render.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-22 16:29:14 -07:00
Zixuan James Li 22ebf701aa cache: Use getattr to access _cache.
`_cache` is not an attribute defined on `BaseCache`, but an
implementation detail of django_bmemcache.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-20 22:00:05 -07:00
Zixuan James Li 159449b448 response: Replace json_unauthorized with UnauthorizedError.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-18 18:01:42 -07:00
Zixuan James Li 52be020d0c custom_profile: Apply ProfileDataElementUpdateDict.
We explicitly annotate variables or parameters with
`ProfileDataElementUpdateDict` as necessary.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:55:03 -07:00
Zixuan James Li c7f75f071e types: Add ProfileDataElementUpdateDict.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:55:03 -07:00
Zixuan James Li 7c62af0d73 typing: Add assertions before accessing settings.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:00:56 -07:00
Zixuan James Li 0ef5eee3f0 error_notify: Fix type narrowing of settings.ERROR_BOT.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:00:56 -07:00
Zixuan James Li 0952b024c1 scim: Add SCIMConfigDict.
This adds a `TypedDict` to provide better type annotation for the
configuration object for SCIM.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:00:56 -07:00
Zixuan James Li 9d1ef72f08 test_helpers: Make use_db_models generic.
We use this decorator on subclasses of `MigrationsTestCase`, which does
not have `self`s being `MigrationsTestCase`, but the corresponding
subclass.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 13:49:45 -07:00
Zixuan James Li 0056becd04 test_helpers: Default user_profile to AnonymousUser.
A request that has went through the auth middleware shouldn't have
`.user` being `None`. We should use `AnonymousUser` by default to
represent unauthenticated users.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-14 17:24:24 -07:00