Commit Graph

455 Commits

Author SHA1 Message Date
sanchi-t af7ebde9e4 subscription: Include archived channels in streams list.
`is_archived` field is added to the stream and types.

Include a new `archived_channeels` client capability, to allow clients
to access data on archived channels, without breaking
backwards-compatibility for existing clients that don't know how to
handle these.

Also, included `exclude_archived` parameter to `/get-streams`,
which defaults to `true` as basic clients may not be interested
in archived streams.
2024-10-25 16:06:40 -07:00
Prakhar Pratyush e763d065a3 django_api: Rename 'send_event' to 'send_event_rollback_unsafe'.
This commit renames the 'send_event' function to
'send_event_rollback_unsafe' to reflect the fact that it doesn't
wait for the db transaction (within which it gets called, if any)
to commit and sends event irrespective of commit or rollback.

In most of the cases we don't want to send event in the case of
rollbacks, so the caller should be aware that calling the function
directly is rollback unsafe.
2024-09-20 15:20:18 -07:00
Sahil Batra fb63c47ea6 user_groups: Add client capability to handle deactivated groups.
This commit adds a client capability to not receive data about
deactivated groups.
2024-09-18 13:41:13 -07:00
Mateusz Mandera ce5439d87f retention: Remove outdated compat block for delete_message events.
As noted in the comment in that block, this is no longer useful.
2024-09-10 17:07:47 -07:00
Kenneth Rodrigues 97f15d8811 events: Migrate to typed_endpoint.
Migrate `event_register.py` and `tornado` to typed_endpoint.
Modified the tests to work with the migrated endpoints.
2024-07-31 17:10:06 -07:00
Anders Kaseorg 0fa5e7f629 ruff: Fix UP035 Import from `collections.abc`, `typing` instead.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 531b34cb4c ruff: Fix UP007 Use `X | Y` for type annotations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg e08a24e47f ruff: Fix UP006 Use `list` instead of `List` for type annotation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 5af8dfda3e ruff: Fix FURB167 Use of regular expression alias.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-06-30 22:37:15 -07:00
Anders Kaseorg f13e94d9ae requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-05-06 12:56:30 -07:00
Tim Abbott 0a756c652c push_notifications: Shard mobile push notifications. 2024-05-02 14:25:10 -07:00
Anders Kaseorg d32d4434dd partial: Replace returns plugin with an annotation.
The returns plugin hasn’t been updated for mypy ≥ 1.6.  This
annotation is more limited in that it only supports a fixed number of
positional arguments and no keyword arguments, but is good enough for
our purposes.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-29 17:14:41 -07:00
Alex Vandiver 823f732132 narrow: Move python-only implemenation into its own file.
Tornado uses this, and none of the heavy-weight SQLAlchemy code; so it
just adds to the startup time of the Tornado server.
2024-04-16 09:48:11 -07:00
Anders Kaseorg a82a3eb4d7 ruff: Fix UP033 Use `@functools.cache`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-01 18:32:52 -07:00
Anders Kaseorg d748ec8d52 ruff: Fix PLW0108 Lambda may be unnecessary.
This is a preview rule, not yet enabled by default.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-01 09:30:04 -08:00
Alex Vandiver 7e9d8cd192 tornado: Fix GC of old queues. 2024-02-20 22:07:25 -08:00
Alex Vandiver da6b0b1cc6 tornado: Add a web_reload_clients endpoint to notify web clients. 2024-02-15 15:42:50 -08:00
Alex Vandiver b25d966352 tornado: Add a --no-immediate-reloads flag to leave web clients alone. 2024-02-15 15:42:50 -08:00
Alex Vandiver 1d3813ec4f tornado: Track which queues were inherited from old Tornado instances. 2024-02-15 15:42:50 -08:00
Alex Vandiver fc41d6085b tornado: Split server restart events from web client reload events. 2024-02-15 15:42:50 -08:00
Alex Vandiver a6287faea4 tornado: Stop collapsing "restart" events via virtual events.
Collapsing was done incorrectly, as 65c400e06d added `zulip_version`
and `zulip_feature_level`, but did not update the virtual event logic
to copy those new values into the virtual event.

However, it is unlikely that a server will be upgraded multiple times
in quick enough succession for this to ever be relevant.  Remove the
logic, which is additional complication for little or no gain.
2024-02-15 15:42:50 -08:00
Alex Vandiver 52a80e1d78 tornado: Switch to time.perf_counter for timing data. 2024-02-15 15:42:50 -08:00
Alex Vandiver ab683fac29 decorator: Rename internal_notify_view.
As noted in the previous commit, this decorator is not just used for
"notify" endpoints anymore.
2024-02-15 15:42:50 -08:00
Alex Vandiver 84fa9be73a tornado: Move the /notify-tornado endpoint, and document it better. 2024-02-15 15:42:50 -08:00
Alex Vandiver 737a751f5c message: Split MessageDict and friends into its own file. 2024-02-14 12:27:03 -08:00
Anders Kaseorg f165ba0fb3 ruff: Fix SIM910 Use `d.get(…)` instead of `d.get(…, None)`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-02-02 10:30:45 -08:00
Anders Kaseorg bac027962f models: Extract zerver.models.clients.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Anders Kaseorg 45bb8d2580 models: Extract zerver.models.users.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Alex Vandiver 7e1f212366 tornado: Handle the handler having been cleared by connection close.
As premonitioned in c741c527d7, it is
indeed possible for `get_handler_by_id` to error out by cause the
handler has been unset elsewhere.

Protect the callsites of `get_handler_by_id` to be able to gracefully
handle when the handler has already done away.
2023-12-12 10:29:37 -08:00
Anders Kaseorg 55b26da82b run-dev: Rewrite development proxy with aiohttp.
This allows request cancellation to be propagated to the server.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-12 08:07:41 -08:00
Alex Vandiver c741c527d7 tornado: Support clearing a handler more than once.
4af00f61a8 claimed that `on_finish` and
`on_connection_close` were mutually exclusive.  In cases where a
`DELETE` is called on the queue while a longpoll is in progress, this
can cause _both_ to happen:

- The `DELETE` pushes a `cleanup_queue` event, which triggers
`finish_handler` to begin pushing out an empty event response to the
longpoll connection.

- In the midst of that, in an `await`, the longpoll connection drops,
and `on_connection_close` clears the handler.

- The `await` resumes, calls `finish`, and attempts to clear the
handler.

The easiest solution is to make `clear_handler_by_id` tolerant to
multiple attempts to clear it.  Since these processes run in parallel,
it means that parts may have a `handler_id` but `get_handler_by_id`
may error in attempting to look it up.  We have not observed this in
testing, and I cannot currently prove it is impossible.
2023-12-11 21:05:50 -08:00
Alex Vandiver 4af00f61a8 tornado: Explicitly remove handler when clients disconnect.
This partially reverts 579bdc18f85ea8599c8cf1f53ddb02fd41d97993; it
assumed (based on its documentation) that `on_finish` was called for
all requests, even client-terminated ones.  This is not accurate; it
is only called when the request calls `finish`, which only happens for
successful requests.  This caused every client-closed connection to
leak a handler (ironically, exactly re-introducing the bug previously
fixed in 12a5a3a6e1).

This behaviour was obscured by the development environment's proxy;
see comment added in the previous commit.

Instead of replacing the `clear_handler_by_id` call into
`ClientDescriptor.disconnect_handler`, we instead place it on
`AsyncDjangoHandler.on_connection_close`.  This is more correct for
a few reasons:

- `on_connection_close` will be called if the client goes away during
a request without a client descriptor.  If the handler garbage
collection of handlers runs inside the ClientDescriptor, we leak
handlers.

- `disconnect_handler` also runs when successfully sending an event,
which already calls `on_finish`.  We avoid double-calling
`clear_handler_by_id` by doing it in two clearly exclusive cases,
`on_finish` and `on_connection_close`.

- It combines the creation and garbage collection logic into one
file, decreasing action at a distance which causes memory leaks.
2023-12-11 14:10:39 -08:00
Alex Vandiver b032b2a4da tornado: Replace a TODO comment with an explanation. 2023-12-11 14:10:39 -08:00
Sahil Batra 198568522a message: Do not include details of inaccessible users in message data.
This commit adds code to not include original details of senders like
name, email and avatar url in the message objects sent through events
and in the response of endpoint used to fetch messages.

This is the last major commit for the project to add support for
limiting guest access to an entire organization.

Fixes #10970.
2023-12-09 17:23:16 -08:00
Anders Kaseorg cf16ddd4d6 tornado: Fix autoreload for Tornado 6.4 changes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-09 14:24:45 -08:00
Alex Vandiver 579bdc18f8 tornado: Consistently clear handler, via on_finish.
initialize() is called on every request, and stored the
`RequestHandler` (and thus `HTTPServerRequest`) in a global shared
dict.  However, the object is only removed from that structure if the
request was successful.  This means that failed requests (such as 405
Method Not Allowed) leaked `RequestHandler`s and
`HTTPServerRequest`s.

Move the cleanup to `on_finish`, which is called at the close of all
requests, async and not, successful or not.
2023-12-08 09:23:30 -08:00
Alex Vandiver 8eccb3af20 tornado: Stop supporting HEAD requests.
These are not part of the API, and lead to moderately confusing
behaviour -- they block (or not) requested, but of course send no
actual data in the body.
2023-12-08 09:23:30 -08:00
Alex Vandiver 875622fe57 tornado: Set an explicit SUPPORTED_METHODS. 2023-12-08 09:23:30 -08:00
Alex Vandiver 0109658238 event_queue: Switch finish_current_handler to early-abort via return. 2023-12-08 09:23:30 -08:00
Sahil Batra 965869d3f8 register: Add client capability to not receive unknown users data.
This commit adds a new client capability to decide whether the
client needs unknown users data or not.
2023-12-06 00:09:53 -08:00
Anders Kaseorg 223b626256 python: Use urlsplit instead of urlparse.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-05 13:03:07 -08:00
Anders Kaseorg 3853fa875a python: Consistently use from…import for urllib.parse.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-05 13:03:07 -08:00
Anders Kaseorg a50eb2e809 mypy: Enable new error explicit-override.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-12 12:28:41 -07:00
Anders Kaseorg 28597365da python: Delete superfluous parens.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-13 13:40:19 -07:00
Anders Kaseorg 6988622fe8 ruff: Enable B023 Function definition does not bind loop variable.
Python’s loop scoping is misdesigned, resulting in a very common
gotcha for functions that close over loop variables [1].  The general
problem is so bad that even the Go developers plan to break
compatibility in order to fix the same design mistake in their
language [2].

Enable the Ruff rule function-uses-loop-variable (B023) [3], which
conservatively prohibits functions from binding loop variables at all.

[1] https://docs.python-guide.org/writing/gotchas/#late-binding-closures
[2] https://go.dev/s/loopvar-design
[3] https://beta.ruff.rs/docs/rules/function-uses-loop-variable/

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-11 18:03:45 -07:00
Prakhar Pratyush 860eee94fd notifications: Rename 'pm' to 'dm' in 'RecipientInfoResult' dataclass.
This commit renames the keyword 'pm' to 'dm' in the
'pm_mention_email_disabled_user_ids' and
'pm_mention_push_disabled_user_ids' attributes of the
'RecipientInfoResult' dataclass.

'pm' and 'dm' are the acronyms for 'private message' and
'direct message' respectively.

It includes 'TODO/compatibility' code to support the old format
fields in the tornado queues during the Zulip server upgrades.
2023-08-10 17:41:49 -07:00
Prakhar Pratyush 4c9d26ce17 mention: Send notifications for @topic wildcard mentions.
This commit completes the notifications part of the @topic
wildcard mention feature.

Notifications are sent to the topic participants for the
@topic wildcard mention.
2023-07-17 09:39:24 -07:00
Prakhar Pratyush 179d5cb37d mention: Replace 'wildcards' with 'stream_wildcards'.
This prep commit replaces the 'wildcard' keyword in the codebase
with 'stream_wildcard' at some places for better readability, as
we plan to introduce 'topic_wildcards' as a part of the
'@topic mention' project.

Currently, 'wildcards = ["all", "everyone", "stream"]' which is an
alias to mention everyone in the stream, hence better renamed as
'stream_wildcards'.

Eventually, we will have:
'stream_wildcard' as an alias to mention everyone in the stream.
'topic_wildcard' as an alias to mention everyone in the topic.
'wildcard' refers to 'stream_wildcard' and 'topic_wildcard' as a whole.
2023-07-03 22:03:17 -07:00
Steve Howell c4d8f501d6 narrow: Split out narrow_helpers.
This will make more sense as I get deeper into modernizing
how we accept narrows from users via the API and represent
the narrows in event queues.
2023-06-30 11:26:23 -07:00
Steve Howell c38b72c014 narrow: Rename *narrow_filter to *narrow_predicate.
This is consistent with how we name similar functions on
the frontend, and "filter" is misleading when you are
not dealing with lists.
2023-06-30 11:26:23 -07:00