Commit Graph

47798 Commits

Author SHA1 Message Date
Lauryn Menard 0301941eef email-log: Move `forward_address` to `REQ` framework.
Instead of using `request.POST` to access `forward_address` for
the parameter used in `set_forward_address` in `email_page`, adds
`has_request_variable` decorator and an optional `forward_address`
parameter through the `REQ` framework.

Adds an assertion that `forward_address` is not `None` for `POST`
requests.
2022-07-20 14:22:25 -07:00
Lauryn Menard 8ff948ba47 translation: Translate new user messages to realm's default language.
Previously, automated stream messages for new user signups were not
being translated into the realm's default language for said messages.

Moves `override_language` context manager so that it wraps the
new user message content in `notify_new_user` and topic string in
`send_message_to_signup_notification_stream`.

Fixes #22510.
2022-07-20 12:38:05 -07:00
Alex Vandiver 526a04b4e6 restore-backup: Provide flags to leave settings.py and zulip.conf as-is. 2022-07-20 12:35:51 -07:00
Lauryn Menard 35e9e4cd3b keyboard-shortcuts: Add class to arrow keys for Mac shortcuts.
The <kbd> elements in `static.templates.keyboard_shortcuts.hbs`
that are arrow keys have a class of "arrow-key". This adds that
class to arrow keys that are updated via `adjust_mac_shortcuts`.

Follow-up task due to changes introduced in #22330.
2022-07-20 11:05:15 -07:00
Alex Vandiver 7553c7ca4e docs: Document that you need to copy slack-data.zip onto the server. 2022-07-20 09:38:28 -07:00
Anders Kaseorg b0160074f4 docs: Correct path to zulip.yaml.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-20 09:37:57 -07:00
Alex Vandiver d8ae270899 restore-backup: Only extract /etc/zulip once.
This is already handled in the earlier block; there is no need to
extract it twice.
2022-07-19 17:56:40 -07:00
Alex Vandiver 1b57669771 restore-backup: Switch to run() to check exit codes. 2022-07-19 17:56:40 -07:00
Alex Vandiver c71c6187ea restore-backup: Ensure it is run as root. 2022-07-19 17:56:40 -07:00
Alex Vandiver a9183d2208 grafana: Enable auto-sign-up.
This avoids the need to explicitly create new users in Grafana, by
simply trusting Teleport.
2022-07-19 17:52:17 -07:00
Alex Vandiver 9bd88a93e2 puppet: Tell needrestart to not default to restarting core services.
The `needrestart` tool added in 22.04 is useful in terms of listing
which services may need to be restarted to pick up updated libraries.
However, it prompts about the current state of services needing
restart for *every* subsequent `apt-get upgrade`, and defaulting core
services to restarting requires carefully manually excluding them
every time, at risk of causing an unscheduled outage.

Build a list of default-off services based on the list in
unattended-upgrades.
2022-07-19 17:51:18 -07:00
Zixuan James Li 32af240af3 documentation: Make get compatible with the supertype.
The supertype contains `*args` and `**kwargs`, this adapts the signature
of the `get` method to make MarkdownDirectoryView compatible with it.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-19 17:48:27 -07:00
Zixuan James Li 5a15fd1dd2 documentation: Call TemplateView.get correctly.
The calling arguments here are completely wrong. The first argument
should be `request`, and `self` should never get passed to `.get`.

Because `TemplateView` happened to not use `request`, and we happened
to pass `article` as a keyword argument, this error slipped through.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-19 17:48:27 -07:00
Lauryn Menard eb2ee5605c api-docs: Update changelog and notes about GitLab authentication.
Updates changelog entry for feature level 1 about GitLab to include
the endpoint with the changes. Also noted that the change updated
a deprecated return value.

Added changes note to the `gitlab` boolean in the
`authentication_methods` return value for the
`/get-server-settings` endpoint.

Part of work on #22102.
2022-07-19 17:45:34 -07:00
Lauryn Menard edeacb63ff api-docs: Update changelog and notes about video call provider None.
Updates the changelog note in feature level 1 about adding None as
a video call provider to include the endpoints where this realm
setting is used.

Updates the OpenAPI doc for the realm setting `video_chat_provider`
to include information about the enum values and meanings.

Part of work on #22102.
2022-07-19 17:45:34 -07:00
Lauryn Menard ddd9cc3ec9 api_docs: Correct Zulip 3.0 feature level 1 changelog and notes.
Corrects omissions or inconsistencies between the api changelog
and the api documentation for Zulip 3.0, feature level 1,
except for the final two bullet points about GitLab authentication
and adding None as a video call provider option.

The final two bullet points will be addressed in separate commits.

Part of work on #22102.
2022-07-19 17:45:34 -07:00
Lauryn Menard c5ebb74280 api-docs: Fix errors found in audit of 3.0 changelog entries.
Initial round of fixes and clean-ups found during audit of
changelog entries for feature levels 1-27, which correspond
to the 3.0 release.

There are a few changes that are not related to those feature
levels, but fit within the context of clean-ups (spelling mistakes
or errors in api documentation formatting/structure/style).

One notable non-3.0 release fix is making all changes notes in
the OpenAPI documentation for 2.x releases use the correct
version numbering-scheme for those releases (e.g. 2.0.0).

Follow-up commits / PRs will address inconsitencies and omissions
for these feature levels found during the audit.
2022-07-19 17:45:34 -07:00
ritikBhandari 861ddea1cd help-docs: Remove no longer accurate "notifications" terminology.
Updates references / language about organization settings that
were previously labeled as "Notifications", but are now labeled
as "Automated messages and emails".

Fixes #22136.

Co-authored by: Lauryn Menard <lauryn@zulip.com>
2022-07-19 17:32:47 -07:00
ritikBhandari 8b880e8942 org-settings: Update "Notifications" header and subheaders.
Updates header "Notifications" to "Automated messages and emails".

Updates these subheaders in that section:
 - "New stream notifications" to "New stream announcements"
 - "New user notifications" to "New user announcements"
 - "Notifications language" to "Language for automated messages
   and invitation emails".

Co-authored by: Lauryn Menard <lauryn@zulip.com>
2022-07-19 17:32:47 -07:00
Zixuan James Li cd8510607a queue_processor: Remove unreachable code.
This change was added in
c93f1d4eda (diff-d88010b113b79080cab5885fdfbbb56ae2d380cb601d8f520621b3361ad8cebc).
`message.content` cannot be `None` by the model definition.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-19 17:30:15 -07:00
Alex Vandiver 2d218366cc js: Only trigger reload-on-focus once.
We have observed infrequent storms of accesses (tens of thousands of
requests to minute) to `/` after an event queue expires.  The current
best theory is that the act of reloading the page itself triggers a
focus event, which itself triggers a reload before the prior one had
had time to do anything but send the network request.

Since the `focus` event here is merely as a backstop in case the
synchronous reloading and deferred reloading fail, we need only run it
once.
2022-07-19 17:29:42 -07:00
Alex Vandiver 8c1c02d164 js: Prevent scheduling a non-immediate reload while one is happening.
Prevent a non-immediate reload from being scheduled while an immediate
reload is already in progress.  This is highly unlikely in practice,
but is a reasonable safeguard.
2022-07-19 17:29:42 -07:00
Alex Vandiver 95ad507658 js: Ensure that reload.initiate cannot fall through.
A `reload.initiate({immediate: true, ...})` *should* not return, as it
should trigger a `window.location.reload` and stop execution.

In the event that it continues execution and returns (for instance,
due to being in the background and reloads being suppressed for
power-saving -- see #6821), there is no need to fall through and
potentially schedule a 90-second-later retry.
2022-07-19 17:29:42 -07:00
Alex Vandiver 58962f0b7b release: Use the right variable to build predictable tarball path.
`tools/build-release-tarball` respects `OUTPUT_DIR`, not `OUTDIR`.
2022-07-19 17:29:03 -07:00
Julia Bichler ccbdbe4e3b message-editing: Reword move message menu. 2022-07-19 11:50:28 -07:00
Alex Vandiver 102481bc47 migrations: Adjust stats size for tsvector to 10k, from 100.
PostgreSQL's `default_statistics_target` is used to track how many
"most common values" ("MCVs") for a column when performing an
`ANALYZE`.  For `tsvector` columns, the number of values is actually
10x this number, because each row contains multiple values for the
column[1].  The `default_statistics_target` defaults to 100[2], and
Zulip does not adjust this at the server level.

This translates to 1000 entries in the MCV for tsvectors. For
large tables like `zerver_messages`, a too-small value can cause
mis-planned query plans.  The query planner assumes that any
entry *not* found in the MCV list is *half* as likely as the
least-likely value in it.  If the table is large, and the MCV list is
too short (as 1000 values is for large deployments), arbitrary
no-in-the-MCV words will often be estimated by the query planner to
occur comparatively quite frequently in the index.  Based on this, the
planner will instead choose to scan all messages accessible by the
user, filtering by word in tsvector, instead of using the tsvector
index and filtering by being accessible to the user.  This results in
degraded performance for word searching.

However, PostgreSQL allows adjustment of this value on a per-column
basis.  Add a migration to adjust the value up to 10k for
`search_tsvector` on `zerver_message`, which results in 100k entries
in that MCV list.

PostgreSQL's documentation says[3]:

> Raising the limit might allow more accurate planner estimates to be
> made, particularly for columns with irregular data distributions, at
> the price of consuming more space in `pg_statistic` and slightly
> more time to compute the estimates.

These costs seem adequate for the utility of having better search.
In the event that the pgroonga backend is in use, these larger index
statistics are simply wasted space and `VACUUM` computational time,
but the costs are likely still reasonable -- even 100k values are
dwarfed by the size of the database needed to generate 100k unique
entries in tsvectors.

[1]: https://github.com/postgres/postgres/blob/REL_14_4/src/backend/utils/adt/array_typanalyze.c#L261-L267
[2]: https://www.postgresql.org/docs/14/runtime-config-query.html#GUC-DEFAULT-STATISTICS-TARGET
[3]: https://www.postgresql.org/docs/14/planner-stats.html#id-1.5.13.5.3
2022-07-19 09:24:06 -07:00
Zixuan James Li 159449b448 response: Replace json_unauthorized with UnauthorizedError.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-18 18:01:42 -07:00
Zixuan James Li 31c7344979 test_message_fetch: Verify the value of WWW-Authenticate.
In `JsonableErrorHandler`, we convert `MissingAuthenticationError` into
a response that has `WWW-Authenticated` set for `/api` or `/json` views.

This covers and verify the value of the header for unauthenticated
access.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-18 18:01:42 -07:00
Mateusz Mandera ff01777139 create_user: Improve comment about prereg_user handling.
The dangling comment was not very helpful and unclear about the pieces
of code it was referring to.

We expand the part about linking the prereg_user to the created user,
while the part about "revoking other preregistration users" is
redundant, because the relevant code block lower down already has
comments on it with better explanations.
2022-07-18 12:16:20 -07:00
Mateusz Mandera 6b451ce6f0 create_user: Remove assert about needing prereg_user if realm_creation.
Closes #22274.

This assertion was added in 4b903c5dcd
where it may have made sense, because indeed when doing realm creation
there was always a PreregistrationUser (created because realms were
created via going to a generated realm creation link). With the addition
of the create_realm command that's no longer the case.

It would be unnatural to create a PreregistrationUser in the
realm_creation command, because there is no confirmation link for it to
be tied to - and it just doesn't make sense conceptually.

The intended, correct behavior added in
4b903c5dcd is still maintained - the code
lower down correctly handles the
(prereg_user is None and realm_creation) case.
2022-07-18 12:16:20 -07:00
Alex Vandiver 7ae3708c02 teleport: Add explicit WebAuthn config, not just U2F.
WebAuthn is the default, replacing U2F, in Teleport 10 and above[1].
While Teleport can derive a WebAuthn configuration from a U2F
configuration[2], it's useful to be explicit.

[1]: https://goteleport.com/docs/access-controls/guides/webauthn/
[2]: https://goteleport.com/docs/access-controls/guides/webauthn/#u2f
2022-07-18 11:41:00 -07:00
Anders Kaseorg 7d8be670a5 message_send: Remove TypeGuard.
The type safety of a TypeGuard is unchecked by mypy.  While this
particular TypeGuard is safe given the current context, one could
imagine future changes that make it unsafe, so it’s preferable to
avoid unchecked constructs whenever possible.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-16 10:44:16 -07:00
Alex Vandiver 39efe6f312 ci: Remove references to CircleCI, which we no longer use. 2022-07-16 10:43:40 -07:00
Alex Vandiver f8e2d652e1 ci: Test upgrades from the minimum of each major version, not the max. 2022-07-16 10:43:40 -07:00
Alex Vandiver 61fce82b0b docs: Canonicalize on "Major releases only".
Some locations were inconsistent about ""major releases" vs "except
minor releases".
2022-07-16 10:43:40 -07:00
Sahil Batra 690420ffa0 populate_db: Fix data for "Favorite editor" custom field.
The field_data sent from client while creating a select
type field is a dict with a number as key.

In development database the field data for "Favorite editor"
field was of different form where the option label was used
as key in the dict.

This commit fixes it to be of the same as it is when creating
a field from web-app. As a result, we also need to update
the tests and this commit also update field_data for other
select-type fields.
2022-07-15 16:51:24 -07:00
Sahil Batra 2b30c3c37c settings_profile_fields: Show confirmation modal when deleting option.
We show a confirmation modal when deleting option of a select-type
field mentioning number of users who are using that option. We also
show the modal if no user has selected that option. We show the modal
only in edit-form and not while creating a new profile field. We do
not show the modal when removing a newly added option during editing
the field.

Fixes #22145.
2022-07-15 16:51:24 -07:00
Sahil Batra 57e0261ff6 settings_profile_fields: Keep option value constant.
Previously, the value for an option of the select type custom
profile field was set as "order - 1". This commit changes it
to remain same even when we reorder the options or delete an
option. When we add a new option, its value is set as 1 more
than largest value already used.

This helps is eliminating various bugs in this subsystem, where
user's choice is changed unexpectedly when reordering or deleting
options.

Discussion -
https://chat.zulip.org/#narrow/stream/378-api-design/topic/custom.20profile.20fields.20option.20deletion.
2022-07-15 16:51:24 -07:00
Sahil Batra 0afda70c3f migration: Add migration to remove user values for deleted options.
This commit adds migration to delete CustomProfileFieldValue
objects for deleted options of SELECT type custom profile
fields.
2022-07-15 16:51:22 -07:00
Sahil Batra fac5e0605b custom_profile_fields: Delete invalid values for select-type fields.
We delete the user values for options which were deleted for the
field.
2022-07-15 16:33:15 -07:00
Julia Bichler 2fcd8d5e21 message_edit: Immediately show propagation menu for topic edits.
This immediately shows the menu for which messages to move when the
user can edit the stream/topic of a message, but not the content of
the message. This balances the concern that this menu might feel like
clutter when doing a content edit with the fact that most of the time
when one visits this menu without content edit permissions, one will
want to be reassured that one can choose which messages to move.

Fixes #19196.
2022-07-15 15:31:33 -07:00
jai2201 511c589e2d message-edit: Add maxlength attribute for editing topic names. 2022-07-15 15:16:55 -07:00
jai2201 97ffeb0b7e message-edit: Avoid hardcoding maxlength for message_edit_form. 2022-07-15 15:16:55 -07:00
jai2201 3853a65a4c compose: Avoid hardcoding maxlength for stream/topic names.
Note that previously, the limit for stream names was 30, even though
stream names have been up to 60 characters in length for years.
2022-07-15 15:15:48 -07:00
jai2201 96be6678e4 stream-edit: Restrict users to maxlength in input for `Stream name`. 2022-07-15 15:10:40 -07:00
jai2201 475b442963 stream-edit: Avoid hardcoding maxlength for stream description. 2022-07-15 15:10:40 -07:00
jai2201 51f2a7f5f4 stream-settings: Rename variables for stream name/description maxlength. 2022-07-15 15:10:40 -07:00
Lauryn Menard bc862d9730 docs: Add 'Writing style' section to writing help doc tutorial.
Adds a section for writing and style guidlines to the tutorial
for writing help center articles. Moves the sections on 'Voice'
and 'User interface' to be in the new section, and adds a section
on 'Keyboard shortcuts'.
2022-07-15 15:01:00 -07:00
Alya Abbott c074006b78 help center: Update /help/change-your-language. 2022-07-15 14:59:35 -07:00
Lauryn Menard cdb4c8e3d4 help-docs: Update translation general information text.
Adds a shared file with general information about Zulip's
translation project based on the text in web-app's language
picker, and uses that text at the beginning of the help articles
for setting the organization notifications/announcement language
and for the user setting their personal language setting.

Also makes some small edits/updates to the help center article
about the user's personal language setting to align with current
UI and current documentation styles.
2022-07-15 14:59:35 -07:00