The previous error page was inadequate for serving the two different
scenarios where we show errors in realm_creations, in particular
containing a misleading sentence about realm creation being disabled
(even in the case where it was actually enabled and the user simply had
an expired link).
The query in display_in_profile_summary_limit_reached should check
realm also since there is per-realm limit of 2 fields, otherwise
this will cause issues where multiple realms are hosted on the
same server.
Fixes#23368.
This was broken by commit b945aa3443
(#22604), because email_to_domain implicitly lowercased the result.
No adjustment is needed for is_disposable_domain, which already
lowercases its argument.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Previously we did not send notification for topic-only edits.
Now, we add backend support for sending notification to topic-only
edits as well.
We would add support for this in webapp in further commits since
message edit UI will be updated as well. We just make sure that no
notifications are sent when editing topic using pencil icon in
message header.
We also change the API default for moving a topic to only notify the
new location, not the old one; this matches the current defaults in
the web UI.
Includes many tests.
We also update the puppeteer tests to test only content edit as
we are going to change the UI to not allow topic editing from
message edit UI. Also fixing the existing tests to pass while
doing topic edits is somewhat complex as notification message
is also sent to new topic by default.
Fixes#21712.
Co-authored-by: Aman Agrawal <amanagr@zulip.com>
Co-authored-by: Tim Abbott <tabbott@zulip.com>
This commit adds the OPTIONAL .realm attribute to Message
(and ArchivedMessage), with the server changes for making new Messages
have this set. Old Messages still have to be migrated to backfill this,
before it can be non-nullable.
Appropriate test changes to correctly set .realm for Messages the tests
manually create are included here as well.
Previously, we included all three message edit related settings
("allow_message_editing", "message_content_edit_limit_seconds" and
"edit_topic_policy") in the event data and api response irrespective
of which of these settings were changed. Now, we only include changed
settings and separate events are sent for each setting if more than
one of them is changed.
Note that the previous typed in event_schema.py for
`message_content_edit_limit_seconds` incorrectly did not allow `None`
as a value, which is used to encode no limit.
Unfortunately, doing so requires forking common API documentation
text, since we're not making any changes to other endpoints that don't
allow unauthenticated requests at all.
Follow-up on #21995.
We need to move this function to a separate actions file specifically
for `user_status` because otherwise we will have a circular import
between `actions/user_settings.py` and `actions/presence.py` in an
upcoming commit.
Prep commit for migrating "unavailable" user status feature to
"invisible" user presence feature.
Fixes#22821.
As explained in the comment in the code:
Topics can be large enough that this request will inevitably time out.
In such a case, it's good for some progress to be accomplished, so that
full deletion can be achieved by repeating the request. For that purpose,
we delete messages in atomic batches, committing after each batch.
The additional perk is that the ordering of messages should prevent some
hypothetical deadlocks - ref #19054
To allow `custom_profile_field` to display in user profile popover,
added new boolean field "display_in_profile_summary" in its model class.
In `custom_profile_fields.py`, functions are edited as per conditions,
like currently we can display max 2 `custom_profile_fields` except
`LONG_TEXT` and `USER` type fields.
Default external account custom profile fields made updatable for only
this new field, as previous they were not updatable.
Fixes part of: #21215
We allow only owners to change the waiting period setting to become
full member. This commit contains only backend changes, frontend
changes will be done separately.
We allow only owners to add, edit or delete the allowed domains.
This commit only contains backend changes, frontend changes will
be done in a separate commit.
We allow only owners to change disallow_disposable_email_addresses
and emails_restricted_to_domains settings. This commit only contains
change in backend part, frontend changes will be done separately.
We allow only owners to change the invite_required setting.
This commit only adds the restriction in backend, frontend
changes will be done separately.
We also add a helper function in test_realm.py to avoid
writing same code repeatedly and this helper will also
be used in tests for other settings to be added in
further commits.
As suggested by the new comments, the cost for a Zulip data export
scales with messages actually included in the export, so an
organizations with 1M private messages but only 50K public stream
messages should not be modeled the same as one with 1M public stream
messages for the purpose of the limits here.
Also improve the comments and variable names more generally.
Updates the base titles so that they begin with the page content,
and end with "| Zulip" + information about the type of doc: "help
center", "API documentation", "terms and policies", or "integrations".
Added a user_list_style personal user setting to the bottom of
Settings > Display settings > Theme section which controls the look
of the right sidebar user list.
The radio button UI includes a preview of what the styles look like.
The setting is intended to eventually have 3 possible values: COMPACT,
WITH_STATUS and WITH_AVATAR; the final value is not yet implemented.
Co-authored-by: Tim Abbott <tabbott@zulip.com>
We create "event-filter-instruction.md" and add it to
"create-bot-construct-url.md". This allows the user to keep track of the
supported event types for most of the integrations that implement this
feature. Note that not all integrations use "create-bot-construct-url.md".
We also need to rename "function" to "view_function" to make this change
type-check.
This is relevant to #18392.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This handler from commit a696141a25
(#7418) was almost certainly made unreachable when commit
c3e395b7d8 (#13092) removed anything
that might have thrown a UnicodeDecodeError from highlight_string.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Renames existing template variables starting with `OPEN_GRAPH` to be
either `PAGE` or `PAGE_METADATA` since these variables are used for
adding both open graph metdata and page titles/descriptions for SEO.
This is preparatory commit for #18941.
Importing `do_delete_message` from `message_edit.py` was causing a
circular import error. In order to avoid that, we create a separate
message_delete.py file which has all the functions related to deleting
messages.
The tests for deleting messages are present in
`zerver/tests/test_message_edit.py`.
Fixes a part of #18941
This uses a more specific type `_StrPromise` to replace `Promise`
providing typing information for lazy translation strings.
In places where the callee evaluates the `_StrPromise` object in all
cases we simply force the evaluation with `str()`. This includes
`JsonableError` that ends up handled by the error handler middleware,
and `internal_send_stream_message` that depends on `check_stream_topic`,
requiring the `topic` to be evaluated anyway. In other siuations, the
callee is expected to be able to handle `StrPromise` explicitly.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This removes ViewFuncT and all the associated type casts with ParamSpec
and Concatenate. This provides more accurate type annotation for
decorators at the cost of making the concatenated parameters
positional-only. This change does not intend to introduce any other
behavioral difference. Note that we retype args in process_view as
List[object] because the view functions can not only be called with
arguments of type str.
Note that the first argument of rest_dispatch needs to be made
positional-only because of the presence of **kwargs.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Our seat count calculation is different for guest user than normal users
(a number of initial guests are free, and additional marginal guests are
worth 1/5 of a seat) - so these checks we apply when a user is being
invited or signing up need to know whether it's a guest or non-guest
being added.
The only caller that passes the kwargs argument is the avatar rest_path.
The application of kwargs can be rewritten with a wrapper.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This commit changes the code to consider zero as an invalid value for
message_content_edit_time_limit_seconds. Now to represent the setting that
user can edit the message anytime, the setting value will be "None" in
database and "unlimited" will be passed to API from clients.
We sent the "message_content_delete_limit_seconds"
value to API when we change any setting in that
subsection and thus "do_set_realm_property" is
called even when we do not change that setting
since we handle it separately from most of the
other settings for which we use loop to call
"do_set_realm_property".
This commit changes the code to handle only
the case when the setting value is "None"
separately and rest all the values will be
changed from the "do_set_realm_property"
called in the loop as for other settings.
This modifies the QueryDict when it is mutable, and assign it to `.POST`
after it is turned immutable, as required by django-stubs for this
attribute.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This refactors rate limit related functions from `zerver.decorator` to
zerver.lib.rate_limiter.
We conditionally import `RemoteZulipServer`, `RequestNotes`, and
`RateLimitedRemoteZulipServer` to avoid circular dependency.
Most instances of importing these functions from `zerver.decorator` got
updated, with a few exceptions in `zerver.tests.test_decorators`, where
we do want to mock the rate limiting functions imported in
`zerver.decorator`. The same goes with the mocking example in the
"testing-with-django" documentation.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Adds an API endpoint for accessing read receipts for other users, as
well as a modal UI for displaying that information.
Enables the previously merged privacy settings UI for managing whether
a user makes read receipts data available to other users.
Documentation is pending, and we'll likely want to link to the
documentation with help_settings_link once it is complete.
Fixes#3618.
Co-authored-by: Tim Abbott <tabbott@zulip.com>
This commit adds support to change enable_read_receipts
setting through API and also adds the field to response
of "/register" endpoint so that the setting value
is available to clients.
I found the previous model for computing what settings to use for
streams increasingly difficult to understand, which is generally a
recipe for future bugs.
Refactor to have a clear computation of what complete permissions
state the client is requesting, validate that state, and then pass
that state to the do_change_stream_permission.
We now allow changing access to history of the stream by only passing
"history_public_to_subscribers" parameter. Previously, "is_private"
parameter was also required to change history_public_to_subscribers
otherwise the request was silently ignored.
We also raise error when only history_public_to_subscribers parameter
is passed with value False without "is_private: True" for a public
or web-public stream since we do not allow public streams with
protected history.
We raise error when we try to change a public stream (except for
zephyr mirror realms) to be public with protected history, as we do
not support such streams yet.
Previously, in such case we changed nothing and a notification was
sent to the "stream events" topic with message being "stream is
changed from public to public" and was weird.
Note that this commit only handles the case when both is_private and
history_public_to_subscribers parameters are passed to API and commit
not covers the case when only "history_public_to_subscribers" with
value False is passed to API, since we currently ignore requests
which has only history_public_to_subscribers parameter with not None
and not is_private and is_web_public.
We would do this in further commits when we add support for accepting
only history_public_to_subscribers parameter.
We do not allow changing enable_spectator_access to True using API on
limited plan realms. Frontend changes have been done previously.
This is a follow-up of #22179.
This `mimetype` parameter was introduced in c4fa29a and its last
usage removed in 5bab2a3. This parameter was undocumented in the
OpenAPI endpoint documentation for `/user_uploads`, therefore
there shouldn't be client implementations that rely on it's
presence.
Removes the `request.GET` call for the `mimetype` parameter and
replaces it by getting the `content_type` value from the file,
which is an instance of Django's `UploadedFile` class and stores
that file metadata as a property.
If that returns `None` or an empty string, then we try to guess
the `content_type` from the filename, which is the same as the
previous behaviour when `mimetype` was `None` (which we assume
has been true since it's usage was removed; see above).
If unable to guess the `content_type` from the filename, we now
fallback to "application/octet-stream", instead of an empty string
or `None` value.
Also, removes the specific test written for having `mimetype` as
a url parameter in the request, and replaces it with a test that
covers when we try to guess `content_type` from the filename.
Updates `json_change_settings` so that the default value for the `email`,
`full_name`, `new_password` and `old_password` parameters is `None` instead
of an empty string, which also makes the type annotation `Optional[str]`.
Also, updates tests for email and full name changes to include an empty
string as one of the tested invalid values.
With the new signature of has_request_variables, we can now use
`HttpResponseBase` as the return type of the decorated function.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This makes `has_request_variables` more generic, in the sense of the return
value, and also makes it more accurate, in the sense of requiring the
first parameter of the decorated function to be `HttpRequest`, and
preserving the function signature without using `cast`.
This affects some callers of `has_request_variables` or the callers of its
decoratedfunctions in the following manners:
- Decorated non-view functions called directly in other functions cannot
use `request` as a keyword argument. Becasue `Concatenate` turns the
concatenated parameters (`request: HttpRequest` in this case) into
positional-only parameters. Callers of `get_chart_data` are thus
refactored.
- Functions to be decorated that accept variadic keyword arguments must
define `request: HttpRequest` as positional-only. Mypy in strict mode
rejects such functions otherwise because it is possible for the caller to
pass a keyword argument that has the same name as `request` for `**kwargs`.
No defining `request: HttpRequest` as positional-only breaks type safety
because function with positional-or-keyword parameters cannot be considered
a subtype of a function with the same parameters in which some of them are
positional-only.
Consider `f(x: int, /, **kwargs: object) -> int` and `g(x: int,
**kwargs: object) -> int`. `f(12, x="asd")` is valid but `g(12, x="asd")`
is not.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Now that we can assume Python 3.6+, we can use the
email.headerregistry module to replace hacky manual email address
parsing.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
We have already checked the size of the file in `upload_file_backend`.
This is the only caller of `upload_message_image_from_request`, and
indirectly the only caller of `get_file_info`. There is no need to
retrieve this information again.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
We can express the same idea more simply by not passing `user` in
cases where it isn't valid for UserActivity.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This makes it mandatory to narrow the type of the user to `UserProfile`
before calling this helper.
This effectively removes the `request.user` check. We do not call login_page
anywhere else without getting through the authentication middleware.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
`report/error` is a path where we allow anonymous user access. This has
to be correctly denoted in the type annotation of the user argument of
the view function.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This commit renames admin_access_required parameter of
list_to_streams function to unsubscribing_others since that
parameter is used and passed as True only when calling
the function while unsubscribing others and in further
commits we would allow non-admins too to unsubscribe others
based on can_remove_subscribers_group setting.
The .status value of EmailChangeStatus was not being looked
at anywhere to prevent re-use of email change confirmation links. This
is not a security issue, since the EmailChangeStatus object has a fixed
value for the new_email, while the confirmation link has expiry time of
1 day, which prevents any reasonable malicious scenarios.
We fix this by making get_object_from_key look at
confirmation.content_object.status - which applies
generally to all confirmations where the attached object has the .status
attribute. This is desired, because we never want to
successfully get_object_from_key an object that has already been used or
reused.
This makes the prereg_user.status check in check_prereg_key redundant so
it can be deleted.
Type inference does not work when the default value of `REQ` is
non-optional while `ResultT` is optional. Mypy tries to unify
`json_validator` with `Validator[int]` in `invite_users_backend` instead
of the desired `Validator[Optional[int]]` because of the presence of the
default value `settings.INVITATION_LINK_VALIDITY_MINUTES`, which is
inferred to be an `int`. Mypy does not resort to a less specific type but
instead gives up early.
This issue applies to invite_users_backend and generate_multiuse_invite_backend
in zerver.views.invite.
There might be a way that we can add an overload to get around this, but
it's probably not worth the complexity until it comes up again more frequently.
We do in fact allow `invite_expires_in_minutes` to be `None` in places
like `do_invite_users`, `invite_users_backend`, etc, and we have
`settings.INVITATION_LINK_VALIDITY_MINUTES` as the default for them. So
it makes sense to allow having an optional value for this setting. And
since there isn't a way to independently set the value of this constant,
we move it to a different place.
TODO:
This is a temporary fix that should be refactored when the bug is fixed.
The encountered mypy issue: https://github.com/python/mypy/issues/13234
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
The `RateLimited` exception can be caught by `JsonErrorHandler`, so it
is not necessary to have the try...except statement here. It is also invalid
to pass a string to initialize `RateLimited`.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This adds a `check_string_in` validator to ensure that `op` is actually
valid before we finally return `json_success()`.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Fixes#21266.
We want to tie the prereg_user to the MultiUseInvite directly rather
than to the MultiUserInvite's confirmation object, because the latter is
not possible. This is because the flow is that after going through the
multiuse invite link, the PreregistrationUser is created together with a
Confirmation object, creating a confirmation link (via
create_confirmation_link) to which then the user is redirected to finish
account creation. This means that the PreregistrationUser is already
tied to a Confirmation, so that attribute is occupied.
Since we in fact are using the django test client to generate a response
here, the return type should be `TestHttpResponse` instead.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Due to an incorrect authorization check in Zulip Server 5.4 and
earlier, a member of an organization could craft an API call that
grants organization administrator privileges to one of their bots.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Before this, a link still couldn't be re-used because it would trip up
exception further down user creation codepaths, but that was still a
bug. check_prereg_key is supposed to correctly validate the key - and
trigger an error page being returned if a key (or for any other reason,
the attached PreregistrationUser object) is reused.
test_validate_email_not_already_in_realm needs to be adjusted, because
it was actually re-using a key.
This reverts commit 40fcf5a633.
This commit triggers bug that we haven't fully tracked down, where web
app clients will continually send `update_message_flags` requests,
that then send out via the events system "0 messages were marked as
read" notices, eventually leading to a load spike.
The Tornado part can likely be fixed by checking if
updated_message_ids is empty, but we need to track down the frontend
bug as well.
Instead of using `request.POST` to access `forward_address` for
the parameter used in `set_forward_address` in `email_page`, adds
`has_request_variable` decorator and an optional `forward_address`
parameter through the `REQ` framework.
Adds an assertion that `forward_address` is not `None` for `POST`
requests.
The supertype contains `*args` and `**kwargs`, this adapts the signature
of the `get` method to make MarkdownDirectoryView compatible with it.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
The calling arguments here are completely wrong. The first argument
should be `request`, and `self` should never get passed to `.get`.
Because `TemplateView` happened to not use `request`, and we happened
to pass `article` as a keyword argument, this error slipped through.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This change ensures that we can call the validate and update helper for
custom profile data later.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
We refactor the validator of `update_user_custom_profile_data` to ensure
that the validated variable is properly typed as
a `ProfileDataElementUpdateDict`, so we can call
`validate_user_custom_profile_data` and
`do_update_user_custom_profile_data_if_changed` directly later (unlike
`update_user_backend`, where `value` is allowed to be `None`, the
validator already ensures that no further check is required).
Instead of mutating the original `QueryDict`, we shall create a new
one when updating the `QueryDict`.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
The body of `json_success` should be a `Mapping[str, object]`, optional
value is not allowed here.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This replaces user.is_verified with is_2fa_verified.
The helper does extra checks such that the user being checked for 2fa
authentication status is valid.
`request.user.is_verified` is functionally the same as `is_verified`
from `django_otp.middleware`, except that the former is monkey-patched
onto the user object by the 2FA middleware. We use the latter wrapped
in `is_2fa_verified` instead to avoid accessing the patched attribute.
See also: 6b24d56e59/docs/source/overview.rst (authentication-and-verification)
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
To explain the rationale of this change, for example, there is
`get_user_activity_summary` which accepts either a `Collection[UserActivity]`,
where `QuerySet[T]` is not strictly `Sequence[T]` because its slicing behavior
is different from the `Protocol`, making `Collection` necessary.
Similarily, we should have `Iterable[T]` instead of `List[T]` so that
`QuerySet[T]` will also be an acceptable subtype, or `Sequence[T]` when we
also expect it to be indexed.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
We were blindly adding / removing flag from UserMessages without
check if they even need to be updated.
This caused server to repeatedly update flags for messages which
already had been updated, creating a confusion for other clients
like mobile.
Fixes#22164
This is a prep commit for tightening the types for our wrapped test
client.
The callers of the test client methods are refactored to either call
them without unpacking at all or create a TypedDict for the keyword
arguments to be unpacked. This allows the type checker to know exactly what
keys are present and their corresponding type.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
We no longer need to access the internal `LANGUAGE_CODE` attribute by
using `django.utils.translation.get_language`.
A test case overriding the translation is added to ensure the password
reset form sending to users requested from a wrong domain is properly
translated.
This is a part of django-stubs refactorings.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
We previously parsed any request with method other than {GET, POST} and
Content-Type other than multipart/form-data as if it were
application/x-www-form-urlencoded.
Check that Content-Type is application/x-www-form-urlencoded before
parsing the body that way. Restrict this logic to {DELETE, PATCH,
PUT} (having a body at all doesn’t make sense for {CONNECT, HEAD,
OPTIONS, TRACE}).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
`request.session` is `django.contrib.sessions.backends.base.SessionBase`
`instead of django.contrib.sessions.models.Session`.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This commit renames existing_subgroups variable to existing_direct_subgroup_ids
in add_subgroups_to_group_backend and remove_subgroups_from_group_backend functions
for better readability.
Conceptually, we're clearly intending to check whether the user we're
mutating is the last realm owner. The preexisting code was safe
because we've already checked that the target user is an owner, and
thus if we're the last owner, we're the target user.
This commit attempts to add the backend support by extending the
/json/bots/{bot_id}/ url support to accept the role field as a
parameter. This was previously already possible via
`/json/users/{user_id}`, so this change just simplifies client
implementation.
This is necessary for the mobile/terminal clients to build spectator
support down the line. We'll also be using it for the web application,
in an upcoming commit.
Adds `want_advertise_in_communities_directory` to the realm model
to track organizations that give permission to be listed on such
a site / directory on zulip.com.
Adds a checkbox to the organization profile admin for
organizations to give permission to be advertised in the
Zulip communities directory.
Adds a help center article about the Zulip communities directory
and uses a shared intro documentation file to create sections in
the articles on creating an organization profile and moderating
open organizations.
Co-authored-by: Alya Abbott <alya@zulip.com>
This commit reads the browser locale during user registration, and
sets it as default language of user if it is supported by Zulip.
Otherwise, it is set to realm's default language.
To provide a smoother experience of accessing a web public stream,
we don't ask user to login unless user directly requests a
`/login` URL.
Fixes#21690.
This commit renames get_user_group_direct_members function to
get_user_group_direct_member_ids as it returns a list of ids
and to avoid it being parallel to get_recursive_group_members,
which returns a QuerySet.
`org_type` already exists as a field in the Realm model and is
used when organizations are created / updated in Zulip Cloud,
via the `/analytics/support` view.
Extends the `PATCH /realm` view to be able update `org_type` as
other realm / organization settings are updated, but using the
special log / action that was created for the analytics view.
Adds a field to the `realm op: update` / `realm op: update_dict`
events, which also means an event is now sent when and if the
`org_type` is updated via the analytics view. This is similar
to how updates to an organization's `plan_type` trigger events.
Adds `realm_org_type` as a realm setting fetched from the
`POST /register` endpoint.
This commit adds 'GET /user_groups/{user_group_id}/members'
endpoint to get members of a user group. "direct_member_only"
parameter can be passed as True to the endpoint to get only
direct members of the user group and not the members of
subgroup.
This commit adds 'GET /user_groups/{id}/members/{id}' endpoint to check
whether a user is member of a group.
This commit also adds for_read parameter to access_user_group_by_id,
which if passed as True will provide access to read user group even
if it a system group or if non-admin acting user is not part of the
group.
This commit changes the invite API to accept invitation
expiration time in minutes since we are going to add a
custom option in further commits which would allow a user
to set expiration time in minutes, hours and weeks as well.
Added a setting to the bottom of Settings > Display settings > Theme section
to display the reacting users on a message when numnber of reactions are
small.
This is a preparatory commit for #20980.
We were not setting the `historical` flag correctly for
messages fetched via `json_fetch_raw_message` when used didn't
have any UserMessage.
Extended relevant tests to fetch check message flags too.
Instead of using request.POST to get any potential `sender`
parameters for `send_message_backend`, moves it to the REQ
framework parameters as `req_sender`.
Also, updates `create_mirrored_message_users` to take specific
parameters instead of an HTTP request parameter, which then
accessed parameters via request.POST. And updates existing tests
for those changes.
We now call this function inside do_create_user(...,
realm_creation=True), which generally improves readability and
robustness of the codebase.
This fixes a bug where this onboarding content was not correctly done
when creating a realm via LDAP, and also will be important as we add
new code paths that might let you create a realm.
This improves robustness of any code paths calling do_create_realm,
which previously needed to call this correctly to achieve the same
results as creating a user via the UI.
This also fixes a bug where this code was not called if a realm were
created using the LDAP code path.
This is an important design detail, so we document this aspect of
creating users via both the management command and API code paths with
an explicit parameter value and comment.
The BigBlueButton integration had a problem with generating
the random password with only 12 characters. This would
cause the attendeePW to be the same as the moderatorPW,
which might be fine but seems like something that could be an
error in a future version of BigBlueButton.
The name for a BigBlueButton meeting is now generated from the stream
name and topic name.
The createTime option is used to have the user redirected to a link
that is only valid for this meeting.
Even if the same link in Zulip is used again, a new createTime
parameter will be created, as the Meeting on the BigBlueButton server
has to be recreated.
Fixes#16498.
Fixes#20509.
Fixes#20804.
We pass list of user ids instead of user profile objects to
remove_members_from_user_group. We still need to call user_id_to_users
in the views function instead of directly passing the ids to
remove_members_from_user_group to make sure we check whether all
ids are valid or not.
We pass list of user ids instead of user profile objects to
bulk_add_members_to_user_group. We still need to call user_id_to_users
in the views function instead of directly passing the ids to
bulk_add_members_to_user_group to make sure we check whether all
ids are valid or not.
Previously, this URL just returned the `raw_content` field. It seems
cleanest to just make it a single-message variant of GET /messages,
deprecating the only format.
Translators found it confusing, since it's not at all obvious that the
word "quote" should not be translated.
I'm not altogether happy with the code formatting for this.
While we're changing this, also standardize on the "```` quote" style
of quote blocks to ensure code/quote blocks in stream descriptions are
unlikely to conflict with this syntax.
Provide stream privacy and description in stream notification events
when stream is created.
In function "send_messages_for_new_subscribers" for when stream is
created, put policy name and description of the stream.
Fixes#21004
Since we've changed the database to contain these new fields, we just
need to stop dropping them in the API code.
This also changes the public API to match the database format again
by removing `prev_subject` from edit history API.
Adds an API changelog feature update for the renamed `prev_subject`
field (to `prev_topic`) and new fields (`topic` and `stream`)
in the message `edit_history`.
Also, documents said `edit_history` in the `MessagesBase` schema
in the api documentation, which is used by the `/get-messages`,
`/get-events` and `/zulip-outgoing-webhooks` endpoints.
Fixes#21076.
Co-authored-by: Lauryn Menard <lauryn.menard@gmail.com>
We fix the mutation of caller and other bad patterns, as well as
adding explicit typing to make the code readable.
We also update the OpenAPI documentation for previously
undocumented `prev_strem` field in the `/get-message-history`
endpoint for API validation testing.
Co-authored-by: Lauryn Menard <lauryn.menard@gmail.com>
TestMaybeSendToRegistration needs tweaking here, because it wasn't
setting the subdomain for the dummy request, so
maybe_send_to_registration was actually running with realm=None, which
is not right for these tests.
Also, test_sso_only_when_preregistration_user_exists was creating
PreregistrationUser without setting the realm, which was also incorrect.
create_preregistration_user is a footgun, because it takes the realm
from the request. The calling code is supposed to validate that
registration for the realm is allowed
first, but can sometimes do that on "realm" taken from something else
than the request - and later on calls create_preregistration_user, thus
leading to prereg user creation on unvalidated request.realm.
It's safer, and makes more sense, for this function to take the intended
realm as argument, instead of taking the entire request. It follows that
the same should be done for prepare_activation_url.
The codepaths for joining an organization via a multi-use invitation
(accounts_home_from_multiuse_invite and maybe_send_to_registration)
weren't validating whether
the organization the invite was generated for matches the organization
the user attempts to join - potentially allowing an attacker with access
to organization A to generate a multi-use invite and use it to join
organization B within the same deployment, that they shouldn't have
access to.
The database value for expiry_date is None for the invite
that will never expire and the clients send -1 as value
in the API similar to the message retention setting.
Also, when passing invite_expire_in_days as an argument
in various functions, invite_expire_in_days is passed as
-1 for "Never expires" option since invite_expire_in_days
is an optional argument in some functions and thus we cannot
pass "None" value.
Adds `realm_web_public_access_enabled` as a realm-specific server
setting potentially returned by the `/get-server-settings` endpoint
so that clients that support browsing of web-public stream content
without an account can generate a login page that supports that
type of access.
This commit refactors get_user_by_email function
to use access_user_by_email which is similar to
already existing access_user_by_id and thus using
get_user_data function added recently.
We also remove the unnecessary check for email as
email will always be passed to this endpoint.
Preparatory commit for #10970.
This commit adds get_user_data which is called by
get_members_backend to compute the client_gravatar
value and then return the data of a single user or
all accessible users.
This function will also be used by get_user_by_email
in further commtis.
Putting all of the logic in a `finally` block is equivalent to a bare
`except` block, which silently consumes all exceptions.
Move only the most-necessary parts into the except; this lets
`BadImageError` exceptions from `zerver/lib/upload.py` to escape,
allowing better the generic "Image file upload failed" to be replaced
with a more specific message.
It also allows unexpected exceptions, as the previous commit resolved,
to escape and 500. This lets them be detected and resolved, rather
than give users a silently bad experience.
A recent Postgres upstream release appears to have broken PGroonga.
While we wait for https://github.com/pgroonga/pgroonga/issues/203 to
be resolved, disable PGroonga in our automated tests so that Zulip
CI passes.
Although our NonClosingPool prevents the SQLAlchemy connection from
closing the underlying Django connection, we still want to properly
dispose of the associated SQLAlchemy structures.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Fixes these warnings with SQLALCHEMY_WARN_20=1:
RemovedIn20Warning: Using non-integer/slice indices on Row is
deprecated and will be removed in version 2.0; please use
row._mapping[<key>], or the mappings() accessor on the Result
object. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9)
RemovedIn20Warning: Using the 'in' operator to test for string or
column keys, or integer indexes, in a :class:`.Row` object is
deprecated and will be removed in a future release. Use the
`Row._fields` or `Row._mapping` attribute, i.e. 'key in
row._fields' (Background on SQLAlchemy 2.0 at:
https://sqlalche.me/e/b8d9)
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Fixes this warning with SQLALCHEMY_WARN_20=1:
RemovedIn20Warning: The legacy calling style of select() is deprecated
and will be removed in SQLAlchemy 2.0. Please use the new calling
style described at select(). (Background on SQLAlchemy 2.0 at:
https://sqlalche.me/e/b8d9)
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Fixes “SADeprecationWarning: The Select.column() method is deprecated
and will be removed in a future release. Please use
Select.add_columns() (deprecated since: 1.4)”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Fixes “SADeprecationWarning: Implicit coercion of SELECT and textual
SELECT constructs into FROM clauses is deprecated; please call
.subquery() on any Core select or ORM Query object in order to produce
a subquery object.”
Signed-off-by: Anders Kaseorg <anders@zulip.com>
To avoid an uncaught IntegrityError causing a 500 HTTP response in a
race between two processes trying to mute a topic, we catch the
integrity error and raise the error exception with status 400 we'd
have gotten if the second request had been a bit later.
Fixes#21011.
Adds request as a parameter to json_success as a refactor towards
making `ignored_parameters_unsupported` functionality available
for all API endpoints.
Also, removes any data parameters that are an empty dict or
a dict with the generic success response values.
As a preparatory step to refactoring json_success to accept
request as a parameter, update helper function `compose_views`
in `views.streams.py` to return the response data and call
json_success from view functions that utilize `compose_views`.
Also, updates related test in `zerver.tests.test_subs.py`.
As a preparatory step to refactoring json_success to accept
request as a parameter, change interface of helper functions:
`handle_deferred_message` in `views.message_send.py` and
`mute_topic` and `unmute_topic` in `views.muting.py`, so
that they return None or data for json_success.
Instead call json_sucess in the caller function, which already
has the HttpRequest as a parameter.
In English, compound adjectives should essentially always be
hyphenated. This makes them easier to parse, especially for users who
might not recognize that the words “web public” go together as a
phrase.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
The change to curl_param_value_generators.py warrants a brief
explanation. Stream permission changes now generate a notification
message. Our curl example test for removing a reaction comes after
the two tests for updating the stream permission changes, thus the
hardcoded message ID in that test needs to be incremented by 2 to
account for the two notification messages that now come before it.
This is a part of #20289.
do_make_stream_web_public and do_change_stream_invite_only seem
to contain very similar logic that could just live inside the
do_change_stream_permission function that handles all permission
changes in one place.
This was deprecated in Django 3.1 for being jQuery-specific, and
removed in Django 4.0. Replicate the jQuery-specific check.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Fixes “DeprecationWarning: 'jinja2.Markup' is deprecated and will be
removed in Jinja 3.1. Import 'markupsafe.Markup' instead.”
Signed-off-by: Anders Kaseorg <anders@zulip.com>
An explanatory note on the changes in zulip.yaml and
curl_param_value_generators is warranted here. In our automated
tests for our curl examples, the test for the API endpoint that
changes the posting permissions of a stream comes before our
existing curl test for adding message reactions.
Since there is an extra notification message due to the change in
posting permissions, the message IDs used in tests that come after
need to be incremented by 1.
This is a part of #20289.
It's slightly annoying to plumb Optional[MentionBackend]
down the stack, but it's a one-time change.
I tried to make the cache code relatively unobtrusive
for the single-message use case.
We should be able to eliminate redundant stream queries
using similar techniques.
I considered caching at the level of rendering the message
itself, but this involves nearly as much plumbing, and
you have to account for the fact that several users on
your realm may have distinct default languages (French,
Spanish, Russian, etc.), so you would not eliminate as
many query hops. Also, if multiple streams were involved,
users would get slightly different messages based on
their prior subscriptions.
This replaces the TERMS_OF_SERVICE and PRIVACY_POLICY settings with
just a POLICIES_DIRECTORY setting, in order to support settings (like
Zulip Cloud) where there's more policies than just those two.
With minor changes by Eeshan Garg.
We do s/TOS/TERMS_OF_SERVICE/ on the name, and while we're at it,
remove the assumed zerver/ namespace for the template, which isn't
correct -- Zulip Cloud related content should be in the corporate/
directory.
These variables can be unset if the `os.path.exists` check fails.
That should be rare, since we've previously checked the files do
exist before getting here.
In many of our stream notification messages, we make use of the
same silent user mention syntax, the template for which was always
hardcoded. This commit adds a helper function that all relevant
callers can call to get the right syntax when mentioning users.
Thanks to Tim Abbott for this suggestion!
A confirmation link takes a user to the check_prereg_key_and_redirect
endpoint, before getting redirected to POST to /accounts/register/. The
problem was that validation was happening in the check_prereg_key_and_redirect
part and not in /accounts/register/ - meaning that one could submit an
expired confirmation key and be able to register.
We fix this by moving validation into /accouts/register/.
Migrates the `/update-subscription-settings` api endpoint to the
`ignored_parameters_unsupported` model, which is also currently used
by `/update-settings` and `update-realm-user-settings-defaults`.
This change is a step towards preparing for an eventual migration to
have all endpoints return an `ignored_parameters_unsupported` block.
Previously the `/update-subscription-settings` endpoint returned a
copy of the data object sent in the request.
Fixes#15307.
This commit switches the BigBlueButton integration
to use SHA256 instead of SHA1 as BigBlueButton supports
it and scalelite does now, too.
Fixes#19966.
We restrict access of messages from web public streams if
anonymous login is disabled via `enable_spectator_access`.
Display of `Anonymous login` button is now controlled by
the value of `enable_spectator_access`.
Admins can toggle `enable_spectator_access` via org settings in UI.
The user id is a very useful piece of information that the mobile
client should have access to - instead of only getting the email. This
makes it much simpler to impleent clients that might be robust to
changes in email address.
Previously, our codebase contained links to various versions of the
Django docs, eg https://docs.djangoproject.com/en/1.8/ref/
request-response/#django.http.HttpRequest and https://
docs.djangoproject.com/en/2.2/ref/settings/#std:setting-SERVER_EMAIL
opening a link to a doc with an outdated Django version would show a
warning "This document is for an insecure version of Django that is no
longer supported. Please upgrade to a newer release!".
Most of these links are inside comments.
Following the replacement of these links in our docs, this commit uses
a search with the regex "docs.djangoproject.com/en/([0-9].[0-9]*)/"
and replaces all matches with "docs.djangoproject.com/en/3.2/".
All the new links in this commit have been generated by the above
replace and each link has then been manually checked to ensure that
(1) the page still exists and has not been moved to a new location
(and it has been found that no page has been moved like this), (2)
that the anchor that we're linking to has not been changed (and it has
been found that no anchor has been changed like this).
One comment where we mentioned a Django version in text before linking
to a page for that version has also been changed, the comment
mentioned the specific version when a change happened, and the history
is no longer relevant to us.
Both `create_realm_by_ip` and `find_account_by_ip` send emails to
arbitrary email addresses, and as such can be used to spam users.
Lump their IP rate limits into the same bucket; most legitimate users
will likely not be using both of these endpoints at similar times.
The rate is set at 5 in 30 minutes, the more quickly-restrictive of
the two previous rates.
Since `prefers_web_public_view` key in session is only
relevant to users without an account, this key should no longer
be present in the user's session object.
Fixes#19907
If realm is web_public, spectators can now view avatar of other
users.
There is a special exception we had to introduce in rest model to
allow `/avatar` type of urls for `anonymous` access, because they
don't have the /api/v1 prefix.
Fixes#19838.
Add `escape_navigates_to_default_view` as a bool setting in
UserBaseSettings model and implement it as a checkbox that toggles
the hotkey implementation of escape to the default view in the
advanced user display settings.
With /help/ documentation edits from Alya Abbott.
Fixes#20043.
It is confusing to have the plan type constants not be namespaced
by the thing they represent. We already have a namespacing
convention in place for constants, so we should use it for
Realm.plan_type as well.
* Remove unnecessary json_validator for full_name parameter.
* Update frontend to pass the right parameter.
* Update documentation and note the change.
Fixes#18409.
We pass allow_realm_admin as True to access_stream_by_id for
`GET users/{user_id}/subscriptions{stream_id}` endpoint
because we want to allow non-subscribed admins to get
subscription status in private streams.
Fixes#19077.
This is a prep commit for new permissions model in
which a user group would be able to have a subgroup.
This commit renames get_memberships_of_users to
get_direct_memberships_of_users to specify that
the function is used only to fetch the direct
memberships and not memberships of subgroups of
the direct group.
Extracted this commit from #19866.
Co-authored-by: Anders Kaseorg <anders@zulip.com>
This is a prep commit for new permissions model in which a user group would
be able to have a subgroup.
This commit renames get_user_group_members to get_user_group_direct_members
to specify that the function is used only to fetch direct members of group
and excludes the subgroup's members.
Extracted this commit from #19866.
Co-authored-by: Anders Kaseorg <anders@zulip.com>
This will be useful to let users enable/disable
sharing read receipts once we add that feature.
Note: Added "I've" to IGNORED_PHRASES in
tools/lib/capitalization.py to avoid capitalization
errors for the label text of this setting.
Note: These are not functional in enabling/disabling sending of
typing notifications with this commit.
Refactored the privacy settings update to keep the code less
duplicated along with making the addition of new settings easier.
For users who are not logged in and for those who don't have
'prefers_web_public_view' set in session, we redirect them
to the default login page where they can choose to login
as spectator or authenticated user.
This commit adds create_web_public_stream_policy
field to Realm table which controls the roles that
can create web-public streams and by default its
value is set to POLICY_OWNERS_ONLY.
Users wanted a feature where they could specify
which users can create public streams and which users can
create private streams.
This splits stream creation code into two parts,
public and private stream creation.
Fixes#17009.
This commit replaces 'allow_message_deleting' boolean setting
with an integer setting 'delete_own_message_policy'. We have a
separate dropdown now for deciding which user-roles can delete
messages sent by themselves and the time-limit setting droddown
is different.
This new setting has two options - everyone and admins only. Other
options including moderators will be added further.
We also remove the "Never" option from the original time-limit
dropdown, as admins are always allowed to delete message. This
never option resembled the case of only admins being allowed to
delete but this state is now resembled by setting the dropdown
to "admins only" and we also disable the time-limit dropdown in
this case as admins are allowed to delete irrespective of limit.
Note, this setting is only for deleting messages sent by the
deleting user themselves, and only admins are allowed to delete
messages sent by others as before.
We make zero invalid value for message_content_delete_limit_seconds and
for handling the case of "Allow to delete message any time", the API-level
value of message_content_delete_limit_seconds is "anytime" and "None"
as the DB-level value. We also use these values for message retention
setting, so it helps maintain consistency.
This commit does not remove the 'enable_login_emails' field from
RealmUserDefault table but it is just not used and cannot be
changed from UI or API similar to 'enable_marketing_emails' setting.
This commit removes the existing default_twenty_four_hour_time field in
Realm table which was used to set the twenty_four_hour_time setting of
new user on joining and instead we now use the twenty_four_hour_time
field of RealmUserDefault table for the same.
With some tweaks by tabbott to clarify the documentation.
None of the existing custom profile field types have the value as an
integer like declared in many places - nor is it a string like currently
decalred in types.py. The correct type is Union[str, List[int]]. Rather
than tracking this in so many places throughout the codebase, we add a
new ProfileDataElementValue type and insert it where appropriate.
We allow clients to make existing streams web public via the API.
This feature is still disabled via settings in production
environments, because we may have additional policy rules or UI
warnings we wish to add to this sort of conversion.
User can now create web public stream via the /subscribe API.
So, when a web public stream present in the API request does not
exist, it will be created now by specifying the is_web_public
parameter. The parameter would have been ignored without this
commit.
This extends the invite api endpoints to handle an extra
argument, expiration duration, which states the number of
days before the invitation link expires.
For prereg users, expiration info is attached to event
object to pass it to invite queue processor in order to
create and send confirmation link.
In case of multiuse invites, confirmation links are
created directly inside do_create_multiuse_invite_link(),
For filtering valid user invites, expiration info stored in
Confirmation object is used, which is accessed by a prereg
user using reverse generic relations.
Fixes#16359.
SOCIAL_AUTH_SUBDOMAIN was potentially very confusing when opened by a
user, as it had various Login/Signup buttons as if there was a realm on
it. Instead, we want to display a more informative page to the user
telling them they shouldn't even be there. If possible, we just redirect
them to the realm they most likely came from.
To make this possible, we have to exclude the subdomain from
ROOT_SUBDOMAIN_ALIASES - so that we can give it special behavior.
This commit moves check_settings_values to user_settings.py
from validator.py such that we can import the functions at
the top without any issue of cyclic imports.
We extract the checks for default_language, notification_sound,
and email_notifications_batching_period_seconds setting values
in json_change_settings to a new function check_settings_values.
Since 84742a0, all settings are sent in the `user_settings` dictionary
which were previously sent inline with other fields in /register
response.
In order to simplify the process of adding new personal settings, we
want to transition to a world where new settings only need to consider
the `property_types` object, and code that needs to reference the
legacy behavior interacts with an object with `legacy` in its name.
This way, contributors working on new settings don't need to think
about the legacy code paths at all.
See https://chat.zulip.org/#narrow/stream/378-api-design/topic/user.20settings.20response.20in.20.2Fregister
to understand this better.
This utilizes the generic `BaseNotes` we added for multipurpose
patching. With this migration as an example, we can further support
more types of notes to replace the monkey-patching approach we have used
throughout the codebase for type safety.
This adds the X-Smokescreen-Role header to proxy connections, to track
usage from various codepaths, and enforces a timeout. Timeouts were
kept consistent with their previous values, or set to 5s if they had
none previously.
No codepath except tests passes in more than one user_profile -- and
doing so is what makes the deduplication necessary.
Simplify the API by making it only take one user_profile id.
Closes#19287
This endpoint allows submitting multiple addresses so we need to "weigh"
the rate limit more heavily the more emails are submitted. Clearly e.g.
a request triggering emails to 2 addresses should weigh twice as much as
a request doing that for just 1 address.
This commit adds "user_settings_object" field to
client_capabilities which will be used to determine
if the client needs 'update_display_settings' and
'update_global_notifications' event.
Return zulip_merge_base alongside zulip_version
in `/register`, `/event` and `/server_settings`
endpoint so that the value can be used by other
clients.
The main reason why this is needed is because this seems to be
convention and because we can't easily test event creation without
doing this.
Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
In this commit:
* We update the `UserStatus` model to accept
`AbstractReaction` as a base class so, we can get all the
fields related to store status emoji.
* We update the user status endpoint
(`users/me/status`) to accept status emoji fields.
* We update the user status event to add status emoji
fields.
Co-authored-by: Yash Rathore <33805964+YashRE42@users.noreply.github.com>
This commit replaces boolean field add_emoji_by_admins_only with an
integer field add_custom_emoji_policy as we would also add full members
and moderators option for this setting in further commits.
This removes a bunch of non-functional duplicate JavaScript, HTML, and
CSS that was interfering with maintenance on the functional originals,
because it was never clear how to update the duplicates or how to
check that you’d updated the duplicates correctly.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
This commit moves "enter_sends" setting to property_types dict.
With this change, changing enter_sends setting also sends an
event of type "update_display_settings" and thus enables us
to live-update the UI.
Mateusz Mandera
Sahil Batra
Anders Kaseorg
Julia Bichler
Aman Agrawal
Lauryn Menard
yogesh sirsat
Tim Abbott
Raghav Luthra
Zixuan James Li
madrix01
Dinesh
Zixuan James Li
Somesh Ranjan
Siddharth Asthana
Alex Vandiver
Felix
Kartik Srivastava
Steve Howell
NerdyLucifer
Lauryn Menard
Alya Abbott
jai2201
Eeshan Garg
YashRE42
Gaurav Pandey
Ganesh Pawar
sahil839
shanukun
Mateusz Mandera
Vishnu KS
Hemanth V. Alluri
Riken Shah
Suyash Vardhan Mathur