Commit Graph

17718 Commits

Author SHA1 Message Date
Anders Kaseorg 0c8d6dab65 models: Move some functions to zerver.lib.display_recipient.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg f77648f279 models: Extract zerver.models.realm_playgrounds.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg 968b7e6c1d models: Extract zerver.models.linkifiers.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg 8855d8b3bc models: Extract zerver.models.realm_emoji.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg 086df4a81e models: Extract zerver.models.realms.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg a8b5a99281 models: Extract zerver.models.groups.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg ee85ac5433 models: Extract zerver.models.users.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg c1c9db59cd models: Extract zerver.models.constants.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg f78d8e13c1 models: Rename zerver/models.py to zerver/models/__init__.py.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Karl Stolley e44ba3b1f1 portico: Add new self-hosted plans.
Co-authored-by: Alya Abbott <alya@zulip.com>
2023-12-15 11:03:42 -08:00
Sahil Batra 03323b0124 push_notifications: Enforce max user count on self managed plan.
We do not support sending push notifications for realms having
more than 10 users on self managed plan.
2023-12-15 11:03:42 -08:00
Tim Abbott 10862451ef billing: Refactor code to compute push status.
This moves the function which computes can_push and
expected_end_timestamp outside RemoteRealmBillingSession
because we might use this function for RemoteZulipServer
as well and also renames it.
2023-12-15 11:03:42 -08:00
Aman Agrawal 3aed22dcd0 gear_menu_popover: Enable `Plan management` in production. 2023-12-15 11:03:42 -08:00
Mateusz Mandera fb5137f8b5 zilencer: Handle deleted realms nicely at server/analytics. 2023-12-15 09:18:26 -08:00
Alex Vandiver 8102519242 send_email: Remove `List-Unsubscribe-Post` from remote-server emails.
For remote servers, we cannot advertise `List-Unsubscribe=One-Click`,
which is specified in RFC 8058[^1] to mean that the `List-Unsubscribe`
URL supports a POST request with no arguments to unsubscribe.  Because
we show an interstitial and confirmation page, as this is not just a
mailing list which is disabled if you click the link, it does not
support the mail system performing the unsubscribe for the user.

Remove the inaccurate header for remote servers.

[^1]: https://datatracker.ietf.org/doc/html/rfc8058
2023-12-15 09:12:25 -08:00
Alex Vandiver 0f2cc383f6 send_email: Support add_context for remote servers.
612f2c73d6 started passing add_context to
`send_custom_server_email`, but did not make it make use of it.

Also add the `hostname` as a built-in value, since that is most likely
the most useful property.
2023-12-15 08:37:02 -08:00
Tim Abbott 63e5712837 push_notifications: Improve logging when not contacting bouncer. 2023-12-15 08:15:36 -08:00
Mateusz Mandera d351353126 zilencer: Fix server/register failing to check if server is deactivated.
A deactivated server shouldn't be updated like that - we want to return
an error.
2023-12-15 08:06:58 -08:00
Tim Abbott 53f40133eb remote_server: Handle JsonableErrors and connection failures better.
This fixes the exception case on the initial
`/api/v1/remotes/server/analytics/status` case.  Other exceptions from
`send_to_push_bouncer` are allowed to escape.

Co-authored-by: Alex Vandiver <alexmv@zulip.com>
2023-12-14 15:08:52 -08:00
Anders Kaseorg b472d769d1 change_user_role: List role choices in --help output.
Fixes #28227.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-14 15:06:18 -08:00
Sahil Batra 633ec698f5 realm: Enfore length restriction on jitsi_server_url at API level.
Previously, passing a url longer than 200 characters for
jitsi_server_url caused a low-level failure at DB level. This
commit adds this restriction at API level.

Fixes part of #27355.
2023-12-14 12:11:59 -08:00
Alex Vandiver bedb68b2fe registration: Prevent injecting arbitrary strings via query param.
While the query parameter is properly excaped when inlined into the
template (and thus is not an XSS), it can still produce content which
misleads the user via carefully-crafted query parameter.

Validate that the parameter looks like an email address.

Thanks to jinjo2 for reporting this, via HackerOne.
2023-12-14 12:00:16 -08:00
Alex Vandiver 9067220af6 signup: Send status code 400 on invalid emails. 2023-12-14 12:00:16 -08:00
Anders Kaseorg c7245d5f3b models: Make ScheduledMessage.read_by_sender non-nullable.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-14 08:16:31 -08:00
Anders Kaseorg 77a6f44455 message_send: Add read_by_sender API parameter.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-14 08:16:31 -08:00
Anders Kaseorg d7d5b6c73e models: Move Message.sent_by_human to Client.default_read_by_sender.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-14 08:16:31 -08:00
Anders Kaseorg d893ff5ba8 digest: Exclude bots with sender.is_bot, not sent_by_human.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-14 08:16:31 -08:00
Sahil Batra 92c1dfc248 message_send: Optimize checking access to DM recipients.
We previously used get_accessible_user_ids to check whether the
sender can access all DM recipients, which was not efficient as
it queries the Message table. This commit updates the code to
make sure we use get_inaccessible_user_ids which is much more
efficient as it limits the queries to only DM recipients and
also queries the Message table only if needed.

This can still be optimized further as mentioned in #27835 but
this commit is a nice first step.
2023-12-14 08:14:09 -08:00
Alex Vandiver 612f2c73d6 send_custom_email: Support JSON extra context with remote servers. 2023-12-13 22:28:20 -08:00
Alex Vandiver 2863b5954e send_custom_email: Make --json an additional filter, not a target. 2023-12-13 22:28:20 -08:00
Alex Vandiver 2de1a23721 send_custom_email: Provide an unsubscribe link for remote servers. 2023-12-13 22:28:20 -08:00
Alex Vandiver 791d66fe28 send_custom_email: Split out the sending to remote servers. 2023-12-13 22:28:20 -08:00
Alex Vandiver 62e6b10ecd send_email: Pull admins_only back into send_custom_email filtering.
There is no reason for this to be an explicit argument to
send_custom_email, when the rest of the filtering already happens
upstream.
2023-12-13 22:28:20 -08:00
Alex Vandiver 1229e38a0d send_custom_email: Stop lying about the type of options.
Saying `**options: str` is a lie, since it contains bools.  We pluck
out the two bools that we need properly typed because we will be
pushing them into function calls, and type them explicitly as bools.
2023-12-13 22:28:20 -08:00
Tim Abbott 6308e07e53 billing: Standardize remote server plan type IDs.
This will likely save us at least one headache.
2023-12-13 16:40:44 -08:00
Sahil Batra f0d429d0f9 remote_server: Handle data of non existent realms.
This commit adds code to make sure that the push
notification does not crash on receiving data for
a non-existent realm.
2023-12-13 13:13:12 -08:00
Sayam Samal c61b353b4b gear_menu_popover: Update the rocket icon used for plan and pricing.
Fixes #28163.
2023-12-13 07:59:13 -08:00
Alex Vandiver 44e9171a46 register_server: POST requests take data, not params.
This currently _works_, but only because we are very forgiving about
how we parse requests.
2023-12-12 19:58:19 -08:00
Alex Vandiver 7e1f212366 tornado: Handle the handler having been cleared by connection close.
As premonitioned in c741c527d7, it is
indeed possible for `get_handler_by_id` to error out by cause the
handler has been unset elsewhere.

Protect the callsites of `get_handler_by_id` to be able to gracefully
handle when the handler has already done away.
2023-12-12 10:29:37 -08:00
Tim Abbott f60312b2da realm_settings: Clear sessions outside transaction.
This fixes a bug introduced in
6f93ab72c0 where deactivating a realm
would fail with an exception that sessions cannot be cleared inside
database transactions.
2023-12-12 09:20:47 -08:00
Alex Vandiver 2b37a35f71 queue: Only NAK the events if the channel is still open.
If the exception was because the channel closed, attempting to NAK the
events will just raise another error, and is pointless, as the server
already marked the pending events as NAK'd.
2023-12-12 09:20:29 -08:00
Anders Kaseorg 55b26da82b run-dev: Rewrite development proxy with aiohttp.
This allows request cancellation to be propagated to the server.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-12 08:07:41 -08:00
Mateusz Mandera c1988a14a7 zilencer: Return can_push info at the push/notify endpoint.
This provides the remote server this information to refresh it on its
Realm attributes whenever it sends a push notification.

Fixes #27483.
2023-12-12 08:06:12 -08:00
Mateusz Mandera b09f3a2da1 do_set_realm_property: Noop if value isn't really changing.
It makes no sense to do operations if the value isn't changing. In
particular, this creates RealmAuditLog entries and sends useless events.
2023-12-12 08:06:12 -08:00
Vector73 2e71ec78e3 settings: Add "Automatically follow topics where I'm mentioned" setting.
Fixes: #26795
2023-12-11 23:26:11 -08:00
Alex Vandiver c741c527d7 tornado: Support clearing a handler more than once.
4af00f61a8 claimed that `on_finish` and
`on_connection_close` were mutually exclusive.  In cases where a
`DELETE` is called on the queue while a longpoll is in progress, this
can cause _both_ to happen:

- The `DELETE` pushes a `cleanup_queue` event, which triggers
`finish_handler` to begin pushing out an empty event response to the
longpoll connection.

- In the midst of that, in an `await`, the longpoll connection drops,
and `on_connection_close` clears the handler.

- The `await` resumes, calls `finish`, and attempts to clear the
handler.

The easiest solution is to make `clear_handler_by_id` tolerant to
multiple attempts to clear it.  Since these processes run in parallel,
it means that parts may have a `handler_id` but `get_handler_by_id`
may error in attempting to look it up.  We have not observed this in
testing, and I cannot currently prove it is impossible.
2023-12-11 21:05:50 -08:00
Mateusz Mandera 3bcfb9c005 push_notifs: Order device args to send_notifications_to_bouncer by id.
This ensures determinism in these tests doing mock_send.assert_called
with - avoids producing test flakes due to a different order of
retrieval of these objects from the database.
2023-12-11 18:36:23 -08:00
Mateusz Mandera 2916a601b3 push_notifs: Don't send request to bouncer if no devices found. 2023-12-11 18:36:23 -08:00
Mateusz Mandera b82ea179ac zilencer: Have push/notify endpoint return registrations to delete.
- The server sends the list of registrations it believes to have with
  the bouncer.
- The bouncer includes in the response the registrations that it doesn't
  actually have and therefore the server should delete.
2023-12-11 18:36:23 -08:00
Prakhar Pratyush dd8a33f03e import_realm: Create audit log with user count data.
This commit creates a RealmAuditlog entry with a new event_type
'RealmAuditLog.REALM_IMPORTED' after the realm is reactivated.

It contains user count data (using realm_user_count_by_role)
stored in extra_data.

This helps to have an accurate user count data for the billing
system if someone tries to signup just after doing an import.
2023-12-11 15:03:24 -08:00