Commit Graph

15636 Commits

Author SHA1 Message Date
Tim Abbott 5bd1a85659 test_fixtures: Rebuild database when create_realm.py changes.
Now that we've split this out from the enormous actions.py, it makes
sense to include this in the set of inputs for generating the
database.
2022-08-12 13:16:35 -07:00
Sahil Batra 09b35b09c5 migrations: Add migration to set default value of enable_read_receipts.
This migration set default value of enable_read_receipts to True
for existing realms which require an invitation to join.
2022-08-12 22:26:14 +05:30
Sahil Batra 13e0311ef3 realm: Add support to change enable_read_receipts setting.
This commit adds support to change enable_read_receipts
setting through API and also adds the field to response
of "/register" endpoint so that the setting value
is available to clients.
2022-08-12 17:10:03 +05:30
Sahil Batra 05ce72d0bc models: Add enable_read_receipts organization setting field. 2022-08-12 17:01:52 +05:30
Mateusz Mandera 43928ca323 import: Remove dead do_import_system_bots code.
This code is actually a noop (and would be a bug if it wasn't a noop),
because when this runs the server is already initialized, meaning the
internal realm exists and the system bots have been created, so
UserProfile.objects.filter(email=email) is always truthy. Also, system
bots are supposed to live in the internal realm, not in the realm being
imported so this code doesn't make sense currently.
2022-08-11 13:59:50 -07:00
Zixuan James Li 4f4d61cb59 user_topics: Refactor the construction loop for UserTopicDict.
This ensures type safety by not mutating the original queryset values,
that django-stubs to type as a TypedDict without total=False.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-11 13:45:54 -07:00
Zixuan James Li 1d7abb3f5d settings: Extract unnecessary test specific settings.
APPLE_ID_TOKEN_GENERATOR_KEY and EXAMPLE_JWK are exclusively used in
test_auth_backends.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-11 13:45:28 -07:00
Zixuan James Li f994ba30b8 settings: Remove settings.BACKEND_DATABASE_TEMPLATE.
BACKEND_DATABASE_TEMPLATE was introduced in a507a47778.
This setting is only available for the test cases and it is not that
necessary to have it configurable.

We define it as a global variable in zerver.lib.test_fixtures.

This avoids requiring mypy_django_plugin to know the type of
settings.BACKEND_DATABASE_TEMPLATE for type checking purposes, given the fact
that settings.test_extra_settings is not available in production/development
setup.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-11 13:45:28 -07:00
Sahil Batra 544d58a693 user_groups: Add EVERYONE_GROUP_NAME constant.
We now use EVERYONE_GROUP_NAME instead of writing
the actual group name at multiple places, so that we
can have all the group names coded at one place only.
2022-08-11 04:38:36 -07:00
Sahil Batra 8eed801380 user_groups: Add MEMBERS_GROUP_NAME constant.
We now use MEMBERS_GROUP_NAME instead of writing
the actual group name at multiple places, so that we
can have all the group names coded at one place only.
2022-08-11 04:38:36 -07:00
Sahil Batra 9a94d2b762 user_groups: Add MODERATORS_GROUP_NAME constant.
We now use MODERATORS_GROUP_NAME instead of writing
the actual group name at multiple places, so that we
can have all the group names coded at one place only.
2022-08-11 04:38:36 -07:00
Sahil Batra 2f634e6640 user_groups: Add ADMINISTRATORS_GROUP_NAME constant.
We now use ADMINISTRATORS_GROUP_NAME instead of writing
the actual group name at multiple places, so that we can
have all the group names coded at one place only.
2022-08-11 04:38:36 -07:00
Sahil Batra 86b496a981 user_groups: Add OWNERS_GROUP_NAME constant.
We now use OWNERS_GROUP_NAME instead of writing
the actual group name at multiple places, so that
we can have all the group names coded at one place
only.
2022-08-11 04:38:36 -07:00
Sahil Batra 15ef0daaf8 user_groups: Add EVERYONE_ON_INTERNET_GROUP_NAME constant.
We now use EVERYONE_ON_INTERNET_GROUP_NAME instead of
writing the actual group name at multiple places, so
that we can have all the group names coded at one place
only.
2022-08-11 04:38:36 -07:00
Sahil Batra 31d639160f user_groups: Add FULL_MEMBERS_GROUP_NAME constant.
We now use FULL_MEMBERS_GROUP_NAME instead of
writing the actual full members system group
name at multiple places, so that we can have
all the group names coded at one place only.
2022-08-11 04:38:36 -07:00
Sahil Batra aef0b6fad4 migrations: Add migration to create system groups for internal realms.
There may be some internal realms which were created after applying
"0382_create_role_based_system_groups.py" migration and this migration
is used to create system groups for those realms.
2022-08-11 04:38:36 -07:00
Sahil Batra 150f77aea2 bulk_create: Add users to system user groups in bulk_create_users.
This commit modifies bulk_create_users to add the users to the
respective system groups. And due to this change, now bots in
development environment are also added to system groups.

Tests are changed accordingly as more UserGroupMembeship objects
are created.
2022-08-11 04:38:36 -07:00
Sahil Batra 393afc9781 realms: Create default system user groups for internal realm.
Since we include internal realms while creating system groups
in "0382_create_role_based_system_groups.py", we should do it
when creating new internal realms as well to be consistent.

Tests are changed accordingly as UserGroup objects are created.
We also change the user group ids used in api docs examples
such that user groups are of correct realm.
2022-08-11 04:38:36 -07:00
Mateusz Mandera 9b8d9d038a test_rate_limiter: Make test_add_remove_rule clean up its rules.
This test was polluting the rules for other tests, due to adding rate
limiting rules and not removing them.
2022-08-10 16:30:39 -07:00
Zixuan James Li abebab9f02 integrations: Fix misleading import.
When we wanted to import `gettext_lazy`, we actually imported to `gettext`.
This fixes the typo.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-10 16:28:38 -07:00
Tim Abbott 66b29fbb8c streams: Centralize logic for computing stream permissions.
I found the previous model for computing what settings to use for
streams increasingly difficult to understand, which is generally a
recipe for future bugs.

Refactor to have a clear computation of what complete permissions
state the client is requesting, validate that state, and then pass
that state to the do_change_stream_permission.
2022-08-09 17:05:38 -07:00
Sahil Batra 3e6463804e streams: Allow changing history access without is_private parameter.
We now allow changing access to history of the stream by only passing
"history_public_to_subscribers" parameter. Previously, "is_private"
parameter was also required to change history_public_to_subscribers
otherwise the request was silently ignored.

We also raise error when only history_public_to_subscribers parameter
is passed with value False without "is_private: True" for a public
or web-public stream since we do not allow public streams with
protected history.
2022-08-09 17:05:38 -07:00
Sahil Batra 6ccfebac56 streams: Raise error when making public stream with private history.
We raise error when we try to change a public stream (except for
zephyr mirror realms) to be public with protected history, as we do
not support such streams yet.

Previously, in such case we changed nothing and a notification was
sent to the "stream events" topic with message being "stream is
changed from public to public" and was weird.

Note that this commit only handles the case when both is_private and
history_public_to_subscribers parameters are passed to API and commit
not covers the case when only "history_public_to_subscribers" with
value False is passed to API, since we currently ignore requests
which has only history_public_to_subscribers parameter with not None
and not is_private and is_web_public.
We would do this in further commits when we add support for accepting
only history_public_to_subscribers parameter.
2022-08-09 17:05:38 -07:00
Sahil Batra 6a860c74d6 streams: Remove redundant asserts in do_change_stream_permission.
This commit removes the unnecessary assertion statements in
do_change_stream_permission for case when "is_web_public" is
True, since we already check those cases in the view function
update_stream_backend and this is the only place from where
do_change_stream_permission is called.

We aim to remove other assertions also from there as mentioned
in the comment and instead check the values in caller itself.
2022-08-09 17:05:38 -07:00
Sahil Batra 75768decf1 tests: Pass all arguments in do_change_stream_permission.
This is a prep commit for changing do_change_stream_permission
to require passing all (invite_only, history_public_to_subscribers
and is_web_public) arguments in further commits.
2022-08-09 17:05:38 -07:00
Anders Kaseorg 668a215ef9 decorator: Check Tornado secret with constant-time comparison.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-08-09 16:02:37 -07:00
Anders Kaseorg 2b1b070fda zilencer: Check remote server API keys with constant-time comparison.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-08-09 16:02:37 -07:00
Kartik Srivastava 887233a8eb api: Document /default_streams API endpoint. 2022-08-09 14:55:27 -07:00
Tim Abbott fe282b343a realm_emoji: Fix race while custom emoji are being uploaded.
During a brief period while a custom emoji is being uploaded, it could
be visible to clients even though it was still in the process of being
uploaded, an operation that can fail.
2022-08-09 14:44:28 -07:00
Zixuan James Li 172a166159 cache: Avoid flushing invalid cache for realm emoji.
In certain cases, we call `RealmEmoji.save()` before the filename
becomes available. This result in getting invalid urls generated and
flushed. Normally we call it again shortly after, making it harder to
trigger this bug.

Fixes #22552.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-09 14:44:28 -07:00
Sahil Batra 5a29f4133b realm: Don't allow changing enable_spectator_access to True on limited plan.
We do not allow changing enable_spectator_access to True using API on
limited plan realms. Frontend changes have been done previously.

This is a follow-up of #22179.
2022-08-09 11:47:33 -07:00
Alex Vandiver e653bb2733 rocketchat: Handle PMs with only one recipient.
These are either to a deleted user, or actually to the same user.  In
any case, treat them as self-messages.
2022-08-09 10:58:58 -07:00
Alex Vandiver 51421f378b rocketchat: Skip mentions of unknown users.
It is apparently possible to have a mention of a user who is not (or
no longer?) in the `users.bson` table.

Skip such mention for the purposes of Zulip import; there's nothing
better for us to do.
2022-08-09 10:58:58 -07:00
Alex Vandiver 28a29e64a0 rocketchat: File upload chunks may exist without their metadata.
This is likely an error somewhere in rocketchat's MongoDB "eventual
consistency," but there is no problem with skipping the chunks at this
step.

In the one case where this was observed so far, the upload-id was not
referenced in any message -- if it is referenced and has chunks, but
has no metadata, we will fail later, at that reference.
2022-08-09 10:58:58 -07:00
Lauryn Menard 41df246f5a api-docs: Remove incorrect schema reference in `unread_msgs` object.
Removes an extraneous `BasicStream` schema reference in the
`unread_msgs` object returned by the `/register-queue` endpoint.
2022-08-09 10:12:44 -07:00
Lauryn Menard aa796af0a8 upload: Remove `mimetype` url parameter in `get_file_info`.
This `mimetype` parameter was introduced in c4fa29a and its last
usage removed in 5bab2a3. This parameter was undocumented in the
OpenAPI endpoint documentation for `/user_uploads`, therefore
there shouldn't be client implementations that rely on it's
presence.

Removes the `request.GET` call for the `mimetype` parameter and
replaces it by getting the `content_type` value from the file,
which is an instance of Django's `UploadedFile` class and stores
that file metadata as a property.

If that returns `None` or an empty string, then we try to guess
the `content_type` from the filename, which is the same as the
previous behaviour when `mimetype` was `None` (which we assume
has been true since it's usage was removed; see above).

If unable to guess the `content_type` from the filename, we now
fallback to "application/octet-stream", instead of an empty string
or `None` value.

Also, removes the specific test written for having `mimetype` as
a url parameter in the request, and replaces it with a test that
covers when we try to guess `content_type` from the filename.
2022-08-08 16:06:09 -07:00
Lauryn Menard df3b8c590f user-settings: Make default `None` for name, email and password changes.
Updates `json_change_settings` so that the default value for the `email`,
`full_name`, `new_password` and `old_password` parameters is `None` instead
of an empty string, which also makes the type annotation `Optional[str]`.

Also, updates tests for email and full name changes to include an empty
string as one of the tested invalid values.
2022-08-08 15:37:41 -07:00
Tim Abbott 7661df20a9 migrations: Create realm reactivation ContentType if required.
Because Django's ContentType objects are, by default, created lazily
when an actual object is created that will use them, this migration
would fail on any server that actually had RealmReactivationStatus
objects already, and had not yet created the ContentType for them.

ContentType objects are very simple:

zulip=> select * from django_content_type where model = 'realmreactivationstatus';
 id | app_label |          model
----+-----------+-------------------------
 85 | zerver    | realmreactivationstatus

So we can simply patch this by using get_or_create.
2022-08-07 22:15:47 -07:00
Julia Bichler 0a278c39d2 settings: Send email after deactivating user.
This adds a feature where an admin can choose to send an email
with custom content to an user after they deactivated them.

Fixes #18943.
2022-08-06 21:41:53 -07:00
N-Shar-ma ef044b8697 markdown: Update characters allowed before @ and stream mentions.
Now the following characters are allowed before @-mentions and stream
references (starting with #) for proper rendering - {, [, /.

This commit makes the markdown rendering consistent with autocomplete
(anything that is autocompleted is also rendered properly).
2022-08-06 19:29:39 -07:00
Zixuan James Li 5ad515c560 onboarding: Use dictionary comprehension for dict initialization.
Initializing a dictionary from an iterable requires the each item to be
a tuple containg a key and a value. `mypy_django_plugin` cannot infer
the number of items in an queryset with annotated values, so we have to
explicitly unpack each row with a dictionary comprehension here.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-06 16:21:12 -07:00
Zixuan James Li b908f0d204 integrations: Broaden return type of check_send_webhook_fixture_message.
With the new signature of has_request_variables, we can now use
`HttpResponseBase` as the return type of the decorated function.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-06 16:19:48 -07:00
Zixuan James Li ca0d2f6854 decorator: Refactor decorators expecting UserProfile with ParamSpec.
Decorators like `require_server_admin_api` turns user_profile into a
positional-only parameter, requiring the callers to stop passing it as a
keyword argument.

Functions like `get_chart_data` that gets decorated by both
`require_non_guest_user` and `has_request_variables` now have accurate
type annotation during type checking, with the first two parameters
turned into positional-only, and thus the change in
`analytics.views.stats`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-06 16:19:48 -07:00
Zixuan James Li adae8b6d42 request: Refactor has_request_variables with ParamSpec.
This makes `has_request_variables` more generic, in the sense of the return
value, and also makes it more accurate, in the sense of requiring the
first parameter of the decorated function to be `HttpRequest`, and
preserving the function signature without using `cast`.

This affects some callers of `has_request_variables` or the callers of its
decoratedfunctions in the following manners:

- Decorated non-view functions called directly in other functions cannot
use `request` as a keyword argument. Becasue `Concatenate` turns the
concatenated parameters (`request: HttpRequest` in this case) into
positional-only parameters. Callers of `get_chart_data` are thus
refactored.

- Functions to be decorated that accept variadic keyword arguments must
define `request: HttpRequest` as positional-only. Mypy in strict mode
rejects such functions otherwise because it is possible for the caller to
pass a keyword argument that has the same name as `request` for `**kwargs`.
No defining `request: HttpRequest` as positional-only breaks type safety
because function with positional-or-keyword parameters cannot be considered
a subtype of a function with the same parameters in which some of them are
positional-only.

Consider `f(x: int, /, **kwargs: object) -> int` and `g(x: int,
**kwargs: object) -> int`. `f(12, x="asd")` is valid but `g(12, x="asd")`
is not.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-06 16:19:48 -07:00
Zixuan James Li 9f99e6c43c cache: Fix type annotation for session_cache_items.
This adds an assertion ensuring the type of `store` before accessing the
`cache_key` attribute that does not exist in the base class. Also note
that `.decode` returns `Dict[str, Any]` instead of a `str`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-06 16:16:17 -07:00
Tim Abbott bd04733c0f muted_topics: Clean up unmute topic event code duplication.
The previous construction was pretty fragile, and had a long comment
to reflect that fact; rework to achieve the same effect in a much
cleaner way.
2022-08-04 17:44:00 -07:00
Kartik Srivastava 1291e7000b user_topic: Add user_topic event.
We now send a new user_topic event while muting and unmuting topics.
fetch_initial_state_data now returns an additional user_topics array to
the client that will maintain the user-topic relationship data.
This will support any future addition of new features to modify the
relationship between a user-topic pair.

This commit adds the relevent backend code and schema for the new
event.
2022-08-04 17:44:00 -07:00
Kartik Srivastava 8b674ee3d7 user_topic: Use get_user_topics to build muted topics tuples.
This adds get_user_topics helper method and refactors get_muted_topics
to use it to build muted topics tuples.
2022-08-04 17:44:00 -07:00
Lauryn Menard f89c251b58 api-docs: Revise areas of documentation re: user presence objects.
Updates documentation to include information about user presence
objects with `aggregated` key (instead of the user's email) where
appropriate.

Also, cleans up spelling, grammar and formatting errors in the
descriptive text for these objects / endpoints.
2022-08-04 16:34:13 -07:00
Kartik Srivastava e2760a2bf2 api: Document /realm/presence API endpoint. 2022-08-04 16:34:13 -07:00