Adds warning banner to the organization settings overlay/tabs
for demo organizations. For owners, clicking on the link in the
banner opens a modal to convert the demo organization into a
permanent organization.
For admins that are not owners, clicking on the link will go
to the help center article on demo organizations.
Non-admin users will not see the warning banner.
The modal for converting a demo organization requires the owner
to have set their email address. Once the owner's email address
is set, then the organization can be made permanent by changing
the subdomain of the organization. The deletion date for the
demo organization will be removed as part of updating the
subdomain.
The organization owner must also have updated the organization
type to be any value other than unsepecified to convert the
demo organization to a permanent organization. The modal's
submit button will be disabled if that is the value for the
organization type in the form. The demo organizations created
in the dev environment now have their organization type set to
unspecified on creation.
This is a part of #19523.
Co-authored by: Lauryn Menard <lauryn@zulip.com>
In commit 268f858f3, we removed the "realm_filters" event from the
schemas that we test in `zerver/lib/event_schemas.py`, but the event
is still documented (as deprecated) in the api/get-events doc.
Updates `tools/check_schemas` to not print a warning for an event
schema in the OpenAPI documentation if it's include in the list of
deprecated events list.
Letting the clock tick without a reason introduces the
possibility of nondeterministic test failures depending on the execution
time. The default value when not specified is tick=True, which makes it
easy to miss.
The rule doesn't prohibit setting tick=True, as perhaps there will be
tests wanting to use that feature on purpose, but such a test should
explicitly set it to make the intent clear.
Refactor tools/lib/provision_inner to conditionally write activation commands to user's bash profile based on the host machine type. Automatic activation now skipped for native linux containers.
Fixes#15029
Semgrep 0.118.0 changed the default of --scan-unknown-extensions to
false. It also seems that it no longer respects --lang (or never
did), so rename the config file to reflect that it only includes
Python rules, to make it clear that additional languages will require
separate config files.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
This commit creates a new file integration_url_modal.js which
now contains the code for showing integration url modal.
Since show_generate_integration_url_modal is used in multiple
places, this change helps us in avoiding import cycles.
This commit creates a new file user_deactivation_ui.js which
now contains the code for showing deactivation modal for user
and bots.
Since confirm_deactivation and confirm_bot_deactivation functions
are used in multiple places, this change helps us in avoiding
import cycles.
This commit moves initialize_custom_pronouns_type_fields,
initialize_custom_user_type_fields, initialize_custom_date_type_fields
and append_custom_profile_fields functions to the new file
custom_profile_fields_ui.js from settings_account.js since
they are used for both showing custom profile fields in
"Profile" settings panel and "Edit user" form shown in
user profile modal. This change also helps us in avoiding
import cycles.
We already allowed reruns for failing tests, and this adds
the ability to rerun tests that succeeded as well, which is
helpful for debugging flaky tests.
The intent behind this commit is to tidy up how we handle user info
popovers. The first step is to move everything related to them into
its own module. This commit should not have any functional changes.
The type annotation for functools.partial uses unchecked Any for all
the function parameters (both early and late). returns.curry.partial
uses a mypy plugin to check the parameters safely.
https://returns.readthedocs.io/en/latest/pages/curry.html
Signed-off-by: Anders Kaseorg <anders@zulip.com>
This is a preparatory commit before we migrate `user_group_popover`
from Bootstrap to Tippy library.
The previous implementation was weirdly sharing the logic around
`current_message_info_popover_elem` with the user info popovers based
on a message; very likely an unfortunate latent bug caused by
copy/paste.
To address that, we need to add dedicated functions like
get_user_group_popover_items to avoid breaking keyboard navigation
with this extraction.
We want to reject ambiguous type annotations that set ApiParamConfig
inside a Union. If a parameter is Optional and has a default of None, we
prefer Annotated[Optional[T], ...] over Optional[Annotated[T, ...]].
This implements a check that detects Optional[Annotated[T, ...]] and
raise an assertion error if ApiParamConfig is in the annotation. It also
checks if the type annotation contains any ApiParamConfig objects that
are ignored, which can happen if the Annotated type is nested inside
another type like List, Union, etc.
Note that because
param: Annotated[Optional[T], ...] = None
and
param: Optional[Annotated[Optional[T], ...]] = None
are equivalent in runtime prior to Python 3.11, there is no way for us
to distinguish the two. So we cannot detect that in runtime.
See also: https://github.com/python/cpython/issues/90353
The goal of typed_endpoint is to replicate most features supported by
has_request_variables, and to improve on top of it. There are some
unresolved issues that we don't plan to work on currently. For example,
typed_endpoint does not support ignored_parameters_supported for 400
responses, and it does not run validators on path-only arguments.
Unlike has_request_variables, typed_endpoint supports error handling by
processing validation errors from Pydantic.
Most features supported by has_request_variables are supported by
typed_endpoint in various ways.
To define a function, use a syntax like this with Annotated if there is
any metadata you want to associate with a parameter, do note that
parameters that are not keyword-only are ignored from the request:
```
@typed_endpoint
def view(
request: HttpRequest,
user_profile: UserProfile,
*,
foo: Annotated[int, ApiParamConfig(path_only=True)],
bar: Json[int],
other: Annotated[
Json[int],
ApiParamConfig(
whence="lorem",
documentation_status=NTENTIONALLY_UNDOCUMENTED
)
] = 10,
) -> HttpResponse:
....
```
There are also some shorthands for the commonly used annotated types,
which are encouraged when applicable for better readability and less
typing:
```
WebhookPayload = Annotated[Json[T], ApiParamConfig(argument_type_is_body=True)]
PathOnly = Annotated[T, ApiParamConfig(path_only=True)]
```
Then the view function above can be rewritten as:
```
@typed_endpoint
def view(
request: HttpRequest,
user_profile: UserProfile,
*,
foo: PathOnly[int],
bar: Json[int],
other: Annotated[
Json[int],
ApiParamConfig(
whence="lorem",
documentation_status=INTENTIONALLY_UNDOCUMENTED
)
] = 10,
) -> HttpResponse:
....
```
There are some intentional restrictions:
- A single parameter cannot have more than one ApiParamConfig
- Path-only parameters cannot have default values
- argument_type_is_body is incompatible with whence
- Arguments of name "request", "user_profile", "args", and "kwargs" and
etc. are ignored by typed_endpoint.
- positional-only arguments are not supported by typed_endpoint. Only
keyword-only parameters are expected to be parsed from the request.
- Pydantic's strict mode is always enabled, because we don't want to
coerce input parsed from JSON into other types unnecessarily.
- Using strict mode all the time also means that we should always use
Json[int] instead of int, because it is only possible for the request
to have data of type str, and a type annotation of int will always
reject such data.
typed_endpoint's handling of ignored_parameters_unsupported is mostly
identical to that of has_request_variables.
**Background**
User groups are expected to comply with the DAG constraint for the
many-to-many inter-group membership. The check for this constraint has
to be performed recursively so that we can find all direct and indirect
subgroups of the user group to be added.
This kind of check is vulnerable to phantom reads which is possible at
the default read committed isolation level because we cannot guarantee
that the check is still valid when we are adding the subgroups to the
user group.
**Solution**
To avoid having another transaction concurrently update one of the
to-be-subgroup after the recursive check is done, and before the subgroup
is added, we use SELECT FOR UPDATE to lock the user group rows.
The lock needs to be acquired before a group membership change is about
to occur before any check has been conducted.
Suppose that we are adding subgroup B to supergroup A, the locking protocol
is specified as follows:
1. Acquire a lock for B and all its direct and indirect subgroups.
2. Acquire a lock for A.
For the removal of user groups, we acquire a lock for the user group to
be removed with all its direct and indirect subgroups. This is the special
case A=B, which is still complaint with the protocol.
**Error handling**
We currently rely on Postgres' deadlock detection to abort transactions
and show an error for the users. In the future, we might need some
recovery mechanism or at least better error handling.
**Notes**
An important note is that we need to reuse the recursive CTE query that
finds the direct and indirect subgroups when applying the lock on the
rows. And the lock needs to be acquired the same way for the addition and
removal of direct subgroups.
User membership change (as opposed to user group membership) is not
affected. Read-only queries aren't either. The locks only protect
critical regions where the user group dependency graph might violate
the DAG constraint, where users are not participating.
**Testing**
We implement a transaction test case targeting some typical scenarios
when an internal server error is expected to happen (this means that the
user group view makes the correct decision to abort the transaction when
something goes wrong with locks).
To achieve this, we add a development view intended only for unit tests.
It has a global BARRIER that can be shared across threads, so that we
can synchronize them to consistently reproduce certain potential race
conditions prevented by the database locks.
The transaction test case lanuches pairs of threads initiating possibly
conflicting requests at the same time. The tests are set up such that exactly N
of them are expected to succeed with a certain error message (while we don't
know each one).
**Security notes**
get_recursive_subgroups_for_groups will no longer fetch user groups from
other realms. As a result, trying to add/remove a subgroup from another
realm results in a UserGroup not found error response.
We also implement subgroup-specific checks in has_user_group_access to
keep permission managing in a single place. Do note that the API
currently don't have a way to violate that check because we are only
checking the realm ID now.
Fixes#11767.
Previously multi-character emoji sequences weren't matched in the
emoji regex, so we'd convert the characters to separate images,
breaking the intended display.
This change allows us to match the full emoji sequence, and
therefore show the correct image.
This saves the blue box position as state on the location
in brower history, with `history.replaceState`.
The position is restored when a narrow is activated.
Fixes#20066.
Tweaked provision script to run successfully in Fedora 38 and
included a script to build the groonga libs from source because
the packages in Fedora repos are outdated.
There is a major version jump from the last supported version (F34)
which is EOL so references and support for older versions were
removed.
Fixes: #20635
It's relatively rare that you want to read the coverage report
if the coverage tests pass. This allows devs to get quicker
feedback. Particularly on tricky rebases this can be a real
time saver. It takes about a minute to write the coverage
report on my box.
Dropping support for url_prefix for RealmPlayground, the server now uses
url_template instead only for playground creation, retrieval and audit
logging upon removal.
This does the necessary handling so that url_template is expanded with
the extracted code.
Fixes#25723.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Restore the default django.utils.log.AdminEmailHandler when
ERROR_REPORTING is enabled. Those with more sophisticated needs can
turn it off and use Sentry or a Sentry-compatible system.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Anders Kaseorg
Eeshan Garg
Lauryn Menard
Aman Agrawal
Tim Abbott
Mateusz Mandera
Sudip Bose
Hardik Dharmani
Sahil Batra
YashRE42
Daniil Fadeev
evykassirer
Alex Vandiver
Lalit Kumar Singh
Zixuan James Li
shu.chen
Junyao Chen
Steve Howell