Commit Graph

51607 Commits

Author SHA1 Message Date
Mateusz Mandera b55adbef3d export: Handle RealmAuditLog with .acting_user in different realm. 2023-05-19 11:12:19 -07:00
Alex Vandiver c978bfaa32 models: Add a unique index on UserProfile.api_key.
This prevents `get_user_profile_by_api_key` from doing a sequential
scan.

Doing this requires moving the generation of initial api_key values
into the column definition, so that even bare calls to
`UserProfile.objects.create` (e.g. from tests) call appropriately
generate a random initial value.
2023-05-19 11:11:04 -07:00
evykassirer 27bc36b7d6 settings: Revert dark mode background color changes. 2023-05-19 10:57:50 -07:00
Lauryn Menard 145d6915c1 api-changelog: Add 2.1.0 entry for realm_default_external_accounts.
Adds an API changelog note to 2.1 for the addition of
realm_default_external_accounts to the `/register-queue` response.

Also adds a Changes note to the field in the endpoint's response
API documentation.

The original commit that added it to that endpoint's response was
commit d7ee2aced1.
2023-05-19 10:50:17 -07:00
Sahil Batra a4db19f9aa signup: Fix selector used for validating the forms.
This commit fixes a typo in the selector used to
validate registration, support, realm creation,
password reset and terms of service forms. This
typo also resulted in a bug where "This field is
required" error message was shown at incorrect
position and this change fixes it.

This commit also fixes the client side error
handling of these forms which did not work
perfectly due to the selector being wrong.
2023-05-19 10:48:52 -07:00
Alex Vandiver 18578cc5da ui_init: Properly call reject with an error, not an xhr.
If the spectator registration call fails, properly log the error and
call `reject` with an error object, not the xhr that `channel.post`
calls its error callback with.

This does nothing to address the UI question of what to do should this
request fail.
2023-05-19 10:48:21 -07:00
Alex Vandiver b312a86ada sentry: Add the observed user's IP address before forwarding.
The default for Javascript reporting is that Sentry sets the IP
address of the user to the IP address that the report was observed to
come from[^1].  Since all reports come through the Zulip server, this
results in all reports being "from" one IP address, thus undercounting
the number of affected unauthenticated users, and making it difficult
to correlate Sentry reports with server logs.

Consume the Sentry Envelope format[^2] to inject the submitting
client's observed IP address, when possible.  This ensures that Sentry
reports contain the same IP address that Zulip's server logs do.

[^1]: https://docs.sentry.io/platforms/python/guides/logging/enriching-events/identify-user/
[^2]: https://develop.sentry.dev/sdk/envelopes/
2023-05-18 16:25:54 -07:00
David Rosa 19eb98a74a help: Document "User list style" display setting.
- Documents the "User list style" (under Settings > Display settings)
- Adds new section to /help/status-and-availability#statuses

Fixes #23492.
2023-05-18 16:17:56 -07:00
Toyam Cox 650cdc474d docs: Also set X-Forwarded-Proto in proxies.
Django 4.0 and higher began checking the `Origin` header, which made
it important that Zulip know accurately if the request came over HTTPS
or HTTP; failure to do so would result in "CSRF verification failed"
errors.

For Zulip servers which are accessed via proxies, this means that
`X-Fowarded-Proto` must be set accurately.  Adjust the documentation
for the suggested configurations to add the header.

Fixes: #24599.

Co-authored-by: Alex Vandiver <alexmv@zulip.com>
2023-05-18 17:17:35 -04:00
Karl Stolley 0407abc788 help_pages: Improve anchor highlights.
These changes ensure that only headings targeted by URL fragments are
highlighted in full. Div elements will have their immediate first
child element highlighted instead (e.g., the first element of an API
parameter box).
2023-05-18 13:57:42 -07:00
Aman Agrawal 44ae67d426 css: Fix gap between header and sticky header at some zoom level. 2023-05-18 13:39:46 -07:00
Aman Agrawal 3acd7b8f47 css: Move message header padding to the element with border.
Apply border and top margin on the same element so that browser
has no way to introduce a gap between them.
2023-05-18 13:39:46 -07:00
Alex Vandiver a95b796a91 supervisor: Drop minfds back down from 1000000 to 40000.
1c76036c61 raised the number of `minfds` in Supervisor from 40k to
1M.  If Supervisor cannot guarantee that number of available file
descriptors, it will fail to start; `/etc/security/limits.conf` was
hence adjusted upwards as well.  However, on some virtualized
environments, including Proxmox LXC, setting
`/etc/security/limits.conf` may not be enough to raise the
system-level limits.  This causes `supervisord` with the larger
`minfds` to fail to start.

The limit of 1000000 was chosen to be arbitrarily high, assuming it
came without cost; it is not expected to ever be reached on any
deployment.  262b19346e already lowered one aspect of that
changeset, upon determining it did come with a cost.  Potentially
breaking virtualized deployments during upgrade is another cost of
that change.

Lower the `minfds` it back down to 40k, partially reverting
1c76036c61, but allow adjusting it upwards for extremely large
deployments.  We do not expect any except the largest deployments to
ever hit the 40k limit, and a frictionless deployment for the
vanishingly small number of huge deployments is not worth the
potential upgrade hiccups for the much more frequent smaller
deployments.
2023-05-18 13:04:33 -07:00
David Rosa 7fc474c7a6 help: Update "Restrict stream invitation" page.
- Renames page title to "Restrict stream membership management".
- Renames section about "Who can add users to streams" setting.
- Adds new section "Configure who can remove users".
- Updates help/stream-permissions

Fixes #25264.
2023-05-18 12:50:54 -07:00
David Rosa 55127b6178 help: Add intro section to help/add-or-remove-users-from-a-stream.
- Adds an intro section about what's possible by consolidating
  references to permissions.
2023-05-18 12:50:54 -07:00
Lauryn Menard 5802f7775f api-changelog: Update and clarify docs for feature level 178 entry.
Updates the descriptions and examples for there only being two key
values: "website" and "aggregated".

Also, clarifies that email keys are the Zulip display email.

And removes any descriptive text that says presence objects have
information about the clients the user is logged into.
2023-05-18 11:57:02 -07:00
Alex Vandiver 1184bdc934 push_notifications: Lock message while we mark it pending for push.
Deleting a message can race with sending a push notification for it.
b47535d8bb handled the case where the Message row has gone away --
but in such cases, it is also possible for `access_message` to
succeed, but for the save of `user_message.flags` to fail, because the
UserMessage row has been deleted by then.

Take a lock on the Message row over the accesses of, and updates to,
the relevant UserMessage row.  This guarantees that the
message's (non-)existence is consistent across that transaction.

Partial fix for #16502.
2023-05-18 11:53:21 -07:00
Anders Kaseorg 12310189ed install: Support Debian 12.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-18 11:52:22 -07:00
Anders Kaseorg 16dedb08fd ci: Fix matrix definition for tests job.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-18 11:52:22 -07:00
Anders Kaseorg 4b19863065 test_timeout: Skip test_timeout_warn on Python 3.11 for coverage issue.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-18 11:52:22 -07:00
Anders Kaseorg 480a2e2d64 postgresql-init-dev-db: Work around PGroonga WAL issue.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-18 11:52:22 -07:00
Anders Kaseorg 83c0ed2a9d build-pgroonga: Upgrade PGroonga from 2.3.6 to 3.0.3.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-18 11:52:22 -07:00
Lalit a2db6df826 ts: Convert `deprecated_feature_notice` module to TypeScript. 2023-05-18 08:52:26 -07:00
Lalit 0216eb8915 deprecated_feature_notice: Use `zod` for validating data when fetching it from `localstorage`.
Refactored `maybe_show_deprecation_notice` to use zod to parse the `shown_deprecation_notices`
array stored in localstorage for better type-safety.
2023-05-18 08:52:26 -07:00
Lauryn Menard 2f96b1756e api-changelog: Add notes for feature level 3 `zulip_version` change.
Original API changelog note was made in commit 2c63130195.
2023-05-18 08:51:46 -07:00
Lalit 656a31d38d refactor: Move `maybe_get_stream_name` from `stream_data` to `sub_store`.
This commit moves `maybe_get_stream_name` function from `stream_data` to `sub_store`
as it didn't had any dependency on `stream_data` and it also helps us to cut off
dependency on `stream_data` for some of the modules including `user_topics`.
2023-05-18 08:46:16 -07:00
Lauryn Menard 3023091caa scheduled-messages: Add help center link to fetch / delete descriptions. 2023-05-18 08:45:51 -07:00
Lauryn Menard 67e1889817 scheduled-messages: Add explicit test with emails for direct message. 2023-05-18 08:45:51 -07:00
Lauryn Menard a2a50e1408 scheduled-messages: Simplify message_to type in check_schedule_message.
As of commit 38f6807af1, we accept only stream and user IDs for
the recipient information for scheduled messages, which means we
can simplify the type for `message_to` in `check_schedule_message`.
2023-05-18 08:45:51 -07:00
Lauryn Menard 1ad0ad8ece tests-scheduled-messages: Use "direct" instead of "private". 2023-05-18 08:45:51 -07:00
Lalit a3f46957bc ts: Convert `confirm_dialog` module to TypeScript. 2023-05-18 08:44:30 -07:00
Anders Kaseorg 271df2c98e requirements: Upgrade coverage.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-16 21:57:01 -07:00
Tim Abbott ac3acc941f css: Increase opacity for grey elements.
The message timestamp and a few settings elements looked off with the
new 25% off white dark theme font color.
2023-05-16 16:52:22 -07:00
Trident Pancake 6fb16f3598 message_edit: Disable save btn after edit time limit
Fixes #25413.

The old code was disabling the save button wrong by using
`.addClass("disabled")` instead of `prop()`.

Added tooltip for the disabled save button as per issue #25413 and changed
"Times up!" color to red.

The textbox readonly logic was changed to no longer becoming readonly. Reason
being there are edge cases involving the compose buttons such that simply
marking the textbox as readonly is not sufficient.

E.g. using the compose buttons after readonly still modifies the content.

One solution might be to just hide the compose buttons visually. However, there
are edge cases for that too. If preview mode was previously active, then
perhaps that state needs to be reverted. If any modal is open, such as the emoji
picker, then that needs to be closed. Solving these edge cases doesn't
improve the user experience. Keeping the textbox editable allows an easier way
for user to copy the text and don't have weird cases.
2023-05-16 15:59:29 -07:00
Trident Pancake b1c801847b message_edit: Remove redundant is_editable check 2023-05-16 15:59:29 -07:00
Lakshay Mittal a208da9c4d style: Fix dropdown text alignment by reducing vertical padding.
Zulip's select widgets have a 30px height; this comes from Bootstrap
but is also generally nice for visual consistency.

In modals, we use a 15px font-size, instead of the 14px used in the
rest of the app, and in that context, the 4px vertical padding plus
30px fixed height resulted in the text not being vertically aligned.

Fix this by removing that vertical padding; all of our select elements
with these classes appear to position the text in the center of the
dropdown through other CSS mechanisms.
2023-05-16 15:43:06 -07:00
Lauryn Menard 8f4de3afc0 api-docs: Update create/edit scheduled message errors for IDs.
In commit 38f6807af1, we updated the `POST /scheduled_messages`
endpoint to only accept user IDs for direct messages. The endpoint
alread only accepted a stream ID for stream messages.

But the API documentation was not updated for the errors returned
when either a stream or user with the specified ID does not exist.

Updates the API documentation for the correct error responses.
2023-05-16 15:14:54 -07:00
Alex Vandiver 24c3e25f86 middleware: Redirect non-canonical realm domain names.
If a host is in REALM_HOSTS, it has its own domain name.  Redirect
access from other domain names to that name.
2023-05-16 15:13:51 -07:00
Alex Vandiver 724de9cd49 rocketchat: Treat users with "bot" roles as bots when importing.
We previously relied on `type`, but we have observed bots typed with a
`bot` role as well.
2023-05-16 15:10:58 -07:00
Alex Vandiver 34394cec9a rocketchat: Handle users with no email address set.
Fixes: #25596.
2023-05-16 15:10:58 -07:00
Alex Vandiver 3160c3cce0 realm_export: Return export id from POST which create it. 2023-05-16 14:05:01 -07:00
Alex Vandiver 7811e99548 realm_export: Handle hard head-of-queue failures.
Realm exports may OOM on deployments with low memory; to ensure
forward progress, log the start time in the RealmAuditLog entry, and
key off of the existence of that to prevent re-attempting an export
which was already tried once.
2023-05-16 14:05:01 -07:00
Alex Vandiver 4a43856ba7 realm_export: Do not assume null extra_data is special.
Fixes: #20197.
2023-05-16 14:05:01 -07:00
Alex Vandiver 5eeb616666 realm_export: Test failures during export. 2023-05-16 14:05:01 -07:00
Alex Vandiver 362177b788 workers: Run realm export with one thread if in low-memory environment.
We previously hard-coded 6 threads for the realm export; in low-memory
environments, spawning 6 threads for an export can lean to an OOM,
which kills the process and leaves a partial export on disk -- which
is then tried again, since the export was never completed.  This leads
to excessive disk consumption and brief repeated outages of all other
workers, until the failing export job is manually de-queued somehow.

Lower the export to only use on thread if it is already running in a
multi-threaded environment.  Note that this does not guarantee forward
progress, it merely makes it more likely that exports will succeed in
low-memory deployments.
2023-05-16 14:05:01 -07:00
Alex Vandiver 9f231322c9 workers: Pass down if they are running multi-threaded.
This allows them to decide for themselves if they should enable
timeouts.
2023-05-16 14:05:01 -07:00
Alex Vandiver 5329fed387 test_queue_worker: Do a full recursion on subclasses.
This makes it less likely we will accidentally fail to include a class
if the subclassing of QueueProcessingWorker changes, and lets mypy
more accurately understand the typing.
2023-05-16 14:05:01 -07:00
Alex Vandiver 8d8b5935ac puppet: Prevent unattended upgrades of erlang-base.
When upgraded, the `erlang-base` package automatically stops all
services which depend on the Erlang runtime; for Zulip, this is the
`rabbitmq-server` service.  This results in an unexpected outage of
Zulip.

Block unattended upgrades of the `erlang-base` package.
2023-05-16 14:02:06 -07:00
Sahil Batra 007a51f277 accounts: Allow user to change email visibility during first login.
We now allow users to change email address visibility setting
on the "Terms of service" page during first login. This page is
not shown for users creating account using normal registration
process, but is useful for imported users and users created
through API, LDAP, SCIM and management commands.
2023-05-16 13:52:56 -07:00
Sahil Batra 7f01b3fb63 users: Set tos_version to -1 for users who have not logged-in yet.
We now set tos_version to "-1" for imported users and the ones
created using API or using other methods like LDAP, SCIM and
management commands. This value will help us to allow users to
change email address visibility setting during first login.
2023-05-16 13:52:56 -07:00