Commit Graph

48177 Commits

Author SHA1 Message Date
Alex Vandiver 8c1c02d164 js: Prevent scheduling a non-immediate reload while one is happening.
Prevent a non-immediate reload from being scheduled while an immediate
reload is already in progress.  This is highly unlikely in practice,
but is a reasonable safeguard.
2022-07-19 17:29:42 -07:00
Alex Vandiver 95ad507658 js: Ensure that reload.initiate cannot fall through.
A `reload.initiate({immediate: true, ...})` *should* not return, as it
should trigger a `window.location.reload` and stop execution.

In the event that it continues execution and returns (for instance,
due to being in the background and reloads being suppressed for
power-saving -- see #6821), there is no need to fall through and
potentially schedule a 90-second-later retry.
2022-07-19 17:29:42 -07:00
Alex Vandiver 58962f0b7b release: Use the right variable to build predictable tarball path.
`tools/build-release-tarball` respects `OUTPUT_DIR`, not `OUTDIR`.
2022-07-19 17:29:03 -07:00
Julia Bichler ccbdbe4e3b message-editing: Reword move message menu. 2022-07-19 11:50:28 -07:00
Alex Vandiver 102481bc47 migrations: Adjust stats size for tsvector to 10k, from 100.
PostgreSQL's `default_statistics_target` is used to track how many
"most common values" ("MCVs") for a column when performing an
`ANALYZE`.  For `tsvector` columns, the number of values is actually
10x this number, because each row contains multiple values for the
column[1].  The `default_statistics_target` defaults to 100[2], and
Zulip does not adjust this at the server level.

This translates to 1000 entries in the MCV for tsvectors. For
large tables like `zerver_messages`, a too-small value can cause
mis-planned query plans.  The query planner assumes that any
entry *not* found in the MCV list is *half* as likely as the
least-likely value in it.  If the table is large, and the MCV list is
too short (as 1000 values is for large deployments), arbitrary
no-in-the-MCV words will often be estimated by the query planner to
occur comparatively quite frequently in the index.  Based on this, the
planner will instead choose to scan all messages accessible by the
user, filtering by word in tsvector, instead of using the tsvector
index and filtering by being accessible to the user.  This results in
degraded performance for word searching.

However, PostgreSQL allows adjustment of this value on a per-column
basis.  Add a migration to adjust the value up to 10k for
`search_tsvector` on `zerver_message`, which results in 100k entries
in that MCV list.

PostgreSQL's documentation says[3]:

> Raising the limit might allow more accurate planner estimates to be
> made, particularly for columns with irregular data distributions, at
> the price of consuming more space in `pg_statistic` and slightly
> more time to compute the estimates.

These costs seem adequate for the utility of having better search.
In the event that the pgroonga backend is in use, these larger index
statistics are simply wasted space and `VACUUM` computational time,
but the costs are likely still reasonable -- even 100k values are
dwarfed by the size of the database needed to generate 100k unique
entries in tsvectors.

[1]: https://github.com/postgres/postgres/blob/REL_14_4/src/backend/utils/adt/array_typanalyze.c#L261-L267
[2]: https://www.postgresql.org/docs/14/runtime-config-query.html#GUC-DEFAULT-STATISTICS-TARGET
[3]: https://www.postgresql.org/docs/14/planner-stats.html#id-1.5.13.5.3
2022-07-19 09:24:06 -07:00
Zixuan James Li 159449b448 response: Replace json_unauthorized with UnauthorizedError.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-18 18:01:42 -07:00
Zixuan James Li 31c7344979 test_message_fetch: Verify the value of WWW-Authenticate.
In `JsonableErrorHandler`, we convert `MissingAuthenticationError` into
a response that has `WWW-Authenticated` set for `/api` or `/json` views.

This covers and verify the value of the header for unauthenticated
access.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-18 18:01:42 -07:00
Mateusz Mandera ff01777139 create_user: Improve comment about prereg_user handling.
The dangling comment was not very helpful and unclear about the pieces
of code it was referring to.

We expand the part about linking the prereg_user to the created user,
while the part about "revoking other preregistration users" is
redundant, because the relevant code block lower down already has
comments on it with better explanations.
2022-07-18 12:16:20 -07:00
Mateusz Mandera 6b451ce6f0 create_user: Remove assert about needing prereg_user if realm_creation.
Closes #22274.

This assertion was added in 4b903c5dcd
where it may have made sense, because indeed when doing realm creation
there was always a PreregistrationUser (created because realms were
created via going to a generated realm creation link). With the addition
of the create_realm command that's no longer the case.

It would be unnatural to create a PreregistrationUser in the
realm_creation command, because there is no confirmation link for it to
be tied to - and it just doesn't make sense conceptually.

The intended, correct behavior added in
4b903c5dcd is still maintained - the code
lower down correctly handles the
(prereg_user is None and realm_creation) case.
2022-07-18 12:16:20 -07:00
Alex Vandiver 7ae3708c02 teleport: Add explicit WebAuthn config, not just U2F.
WebAuthn is the default, replacing U2F, in Teleport 10 and above[1].
While Teleport can derive a WebAuthn configuration from a U2F
configuration[2], it's useful to be explicit.

[1]: https://goteleport.com/docs/access-controls/guides/webauthn/
[2]: https://goteleport.com/docs/access-controls/guides/webauthn/#u2f
2022-07-18 11:41:00 -07:00
Anders Kaseorg 7d8be670a5 message_send: Remove TypeGuard.
The type safety of a TypeGuard is unchecked by mypy.  While this
particular TypeGuard is safe given the current context, one could
imagine future changes that make it unsafe, so it’s preferable to
avoid unchecked constructs whenever possible.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-16 10:44:16 -07:00
Alex Vandiver 39efe6f312 ci: Remove references to CircleCI, which we no longer use. 2022-07-16 10:43:40 -07:00
Alex Vandiver f8e2d652e1 ci: Test upgrades from the minimum of each major version, not the max. 2022-07-16 10:43:40 -07:00
Alex Vandiver 61fce82b0b docs: Canonicalize on "Major releases only".
Some locations were inconsistent about ""major releases" vs "except
minor releases".
2022-07-16 10:43:40 -07:00
Sahil Batra 690420ffa0 populate_db: Fix data for "Favorite editor" custom field.
The field_data sent from client while creating a select
type field is a dict with a number as key.

In development database the field data for "Favorite editor"
field was of different form where the option label was used
as key in the dict.

This commit fixes it to be of the same as it is when creating
a field from web-app. As a result, we also need to update
the tests and this commit also update field_data for other
select-type fields.
2022-07-15 16:51:24 -07:00
Sahil Batra 2b30c3c37c settings_profile_fields: Show confirmation modal when deleting option.
We show a confirmation modal when deleting option of a select-type
field mentioning number of users who are using that option. We also
show the modal if no user has selected that option. We show the modal
only in edit-form and not while creating a new profile field. We do
not show the modal when removing a newly added option during editing
the field.

Fixes #22145.
2022-07-15 16:51:24 -07:00
Sahil Batra 57e0261ff6 settings_profile_fields: Keep option value constant.
Previously, the value for an option of the select type custom
profile field was set as "order - 1". This commit changes it
to remain same even when we reorder the options or delete an
option. When we add a new option, its value is set as 1 more
than largest value already used.

This helps is eliminating various bugs in this subsystem, where
user's choice is changed unexpectedly when reordering or deleting
options.

Discussion -
https://chat.zulip.org/#narrow/stream/378-api-design/topic/custom.20profile.20fields.20option.20deletion.
2022-07-15 16:51:24 -07:00
Sahil Batra 0afda70c3f migration: Add migration to remove user values for deleted options.
This commit adds migration to delete CustomProfileFieldValue
objects for deleted options of SELECT type custom profile
fields.
2022-07-15 16:51:22 -07:00
Sahil Batra fac5e0605b custom_profile_fields: Delete invalid values for select-type fields.
We delete the user values for options which were deleted for the
field.
2022-07-15 16:33:15 -07:00
Julia Bichler 2fcd8d5e21 message_edit: Immediately show propagation menu for topic edits.
This immediately shows the menu for which messages to move when the
user can edit the stream/topic of a message, but not the content of
the message. This balances the concern that this menu might feel like
clutter when doing a content edit with the fact that most of the time
when one visits this menu without content edit permissions, one will
want to be reassured that one can choose which messages to move.

Fixes #19196.
2022-07-15 15:31:33 -07:00
jai2201 511c589e2d message-edit: Add maxlength attribute for editing topic names. 2022-07-15 15:16:55 -07:00
jai2201 97ffeb0b7e message-edit: Avoid hardcoding maxlength for message_edit_form. 2022-07-15 15:16:55 -07:00
jai2201 3853a65a4c compose: Avoid hardcoding maxlength for stream/topic names.
Note that previously, the limit for stream names was 30, even though
stream names have been up to 60 characters in length for years.
2022-07-15 15:15:48 -07:00
jai2201 96be6678e4 stream-edit: Restrict users to maxlength in input for `Stream name`. 2022-07-15 15:10:40 -07:00
jai2201 475b442963 stream-edit: Avoid hardcoding maxlength for stream description. 2022-07-15 15:10:40 -07:00
jai2201 51f2a7f5f4 stream-settings: Rename variables for stream name/description maxlength. 2022-07-15 15:10:40 -07:00
Lauryn Menard bc862d9730 docs: Add 'Writing style' section to writing help doc tutorial.
Adds a section for writing and style guidlines to the tutorial
for writing help center articles. Moves the sections on 'Voice'
and 'User interface' to be in the new section, and adds a section
on 'Keyboard shortcuts'.
2022-07-15 15:01:00 -07:00
Alya Abbott c074006b78 help center: Update /help/change-your-language. 2022-07-15 14:59:35 -07:00
Lauryn Menard cdb4c8e3d4 help-docs: Update translation general information text.
Adds a shared file with general information about Zulip's
translation project based on the text in web-app's language
picker, and uses that text at the beginning of the help articles
for setting the organization notifications/announcement language
and for the user setting their personal language setting.

Also makes some small edits/updates to the help center article
about the user's personal language setting to align with current
UI and current documentation styles.
2022-07-15 14:59:35 -07:00
Zixuan James Li 6277f09296 corporate: Fix type annotations for payment failed handler.
When being called, the wrapped function is passed `PaymentIntent`
(the `content_object` of `Event`). With that, since `customer` can be
`None`, an assertion is also required.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:56:22 -07:00
Zixuan James Li 4099e56a1f custom_profile: Refactor test_update_profile_data_successfully.
This refactors the test case with more explicit type annotations, fixing
type errors discovered provided type annotations for
`CustomProfileField`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:55:03 -07:00
Zixuan James Li 52be020d0c custom_profile: Apply ProfileDataElementUpdateDict.
We explicitly annotate variables or parameters with
`ProfileDataElementUpdateDict` as necessary.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:55:03 -07:00
Zixuan James Li 5116efa3de users: Tighten the type annotation of clean_profile_data.
This change ensures that we can call the validate and update helper for
custom profile data later.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:55:03 -07:00
Zixuan James Li 00f2040ffa custom_profile: Use cast to ensure ProfieDataElementUpdateDict.
We refactor the validator of `update_user_custom_profile_data` to ensure
that the validated variable is properly typed as
a `ProfileDataElementUpdateDict`, so we can call
`validate_user_custom_profile_data` and
`do_update_user_custom_profile_data_if_changed` directly later (unlike
`update_user_backend`, where `value` is allowed to be `None`, the
validator already ensures that no further check is required).
2022-07-15 14:55:03 -07:00
Zixuan James Li c7f75f071e types: Add ProfileDataElementUpdateDict.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:55:03 -07:00
Zixuan James Li b210c22329 message_send: Extract ActiveUserDict for accurate typing.
django-stubs types the return type of query.values(...) as a TypedDict.
This makes Dict[str, Any] that we have been using incompatible with it.

We use TypeGuard to ensure that `service_bot_tuples` is correctly
inferred to be `Tuple[int, int]` instead of `Tuple[int, Optional[int]]`.
Given that `bot_type` is optional for `ActiveUserDict`, we need to
narrow `row` to `ActiveBotUserDict` to make sure that `bot_type` is
non-optional. An advantage of this approach is that no assertions or
type casts are needed.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:02:40 -07:00
Zixuan James Li 0bedf0cf22 integrations: Fix wrong type annotation.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:00:56 -07:00
Zixuan James Li db7a6d15dc test_auth_backends: Replace handcrafted requests with HostRequestMock.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:00:56 -07:00
Zixuan James Li 56855e0187 typing: Add assertions for Optional values.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:00:56 -07:00
Zixuan James Li 7c62af0d73 typing: Add assertions before accessing settings.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:00:56 -07:00
Zixuan James Li a09c9a4063 test_auth_backends: Remove unused return.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:00:56 -07:00
Zixuan James Li a81fab46e3 webhook: Do not allow None default for headers.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:00:56 -07:00
Zixuan James Li 0ef5eee3f0 error_notify: Fix type narrowing of settings.ERROR_BOT.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:00:56 -07:00
Zixuan James Li c5075be9aa test_retention: Add type annotation for expected_result
Otherwise mypy infers the type of `expected_result` to be incompatible
with the first argument of `fix_ordering_of_result`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:00:56 -07:00
Zixuan James Li e665ec8ae2 middleware: Add isinstance check before retrieving content.
StreamingHttpResponse is inferred without the isinstance check in the
else branch. We refactor this is shorten the code and also type narrow
it appropriately.
2022-07-15 14:00:56 -07:00
Zixuan James Li 2095258aa5 middleware: Assert request.method is not None.
`request.method` is not `None` in normal use cases, unless an
`HttpRequest` is directly instantiated without the method being set.
This situation does not apply to `WSGIRequest` at all.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:00:56 -07:00
Zixuan James Li ece3fbdf95 migrations: Use the correct type annotation for the Model type.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:00:56 -07:00
Zixuan James Li 7d86d291d4 middleware: Remove inappropriate StreamingHttpResponse annotation.
Asserting response.stream is False is just suggesting the response being
an `HttpResponse`. This removes `StreamingHttpResponse` with the more
generic `HttpResponseBase` with an isinstance-check.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:00:56 -07:00
Zixuan James Li da4654d838 test_send_email: Fix the to_email argument to pass str.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:00:56 -07:00
Zixuan James Li 45db04a39e test_auth_backends: Fix invalid assertions.
`m.output` is a `list` of `str`s. It does not make sense comparing it to
a `str`. Guessed the intention here is to use `self.assert_length`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:00:56 -07:00