Commit Graph

17041 Commits

Author SHA1 Message Date
Prakhar Pratyush 3675a44471 push_notifications: Add the missing compatibility code.
This commit adds 'TODO/compatibility' code to support the
old notification trigger values in the push notification queue
during the Zulip server upgrades.

In f4fa82e, we renamed the following notification triggers:
* 'wildcard_mentioned' to 'stream_wildcard_mentioned'
* 'followed_topic_wildcard_mentioned' to
'stream_wildcard_mentioned_in_followed_topic'.

This should have been added in f4fa82e.
2023-08-10 17:41:49 -07:00
Sahil Batra 0e23280b4f tests: Pass required args to select_related call for Message objects.
This commit adds code to pass all the required arguments to
select_related call for Message objects such that only the
required related fields are fetched from the database.

Previously, we did not pass any arguments to select_related,
so all the directly and indirectly related fields were fetched
when many of them were actually not being used and made the
query unnecessarily complex.
2023-08-10 17:35:43 -07:00
Sahil Batra a6df377688 openapi: Pass required arguments to select_related call.
This commit adds code to pass all the required arguments to
select_related call for Message objects such that only the
required related fields are fetched from the database.

Previously, we did not pass any arguments to select_related,
so all the directly and indirectly related fields were fetched
when many of them were actually not being used and made the
query unnecessarily complex.
2023-08-10 17:35:43 -07:00
Sahil Batra 7ff5423e21 topic: Pass args to select_related call for Message objects.
This commit adds code to pass all the required arguments to
select_related call for Message objects such that only the
required related fields are fetched from the database.

Previously, we did not pass any arguments to select_related,
so all the directly and indirectly related fields were fetched
when many of them were actually not being used and made the
query unnecessarily complex.
2023-08-10 17:35:43 -07:00
Sahil Batra 36f8aba7db message: Pass args to select_related call for Message objects.
This commit adds code to pass all the required arguments to
select_related call for Message objects such that only the
required related fields are fetched from the database.

Previously, we did not pass any arguments to select_related,
so all the directly and indirectly related fields were fetched
when many of them were actually not being used and made the
query unnecessarily complex.
2023-08-10 17:35:43 -07:00
Sahil Batra 35559ae324 export: Pass "user_profile" as arg to select_related call for Subscription.
This commit adds code to pass all the required arguments to
select_related call for Subscription objects such that only
the required related fields are fetched from the database.

Previously, we did not pass any arguments to select_related,
so all the directly and indirectly related fields were fetched
when many of them were actually not being used and made the
query unnecessarily complex.
2023-08-10 17:35:43 -07:00
Sahil Batra ebd91b152c email_mirror: Pass required args to select_related.
This commit adds code to pass all the required arguments to
select_related call for MissedMessageEmailAddress such that
only the required related fields are fetched from the database.

Previously, we did not pass any arguments to select_related,
so all the directly and indirectly related fields were fetched
when many of them were actually not being used and made the
query unnecessarily complex.
2023-08-10 17:35:43 -07:00
Sahil Batra ab488010b3 models: Pass args to select_related in get_stream_by_id_in_realm.
This commit updates the code to pass "realm" and "recipient" as
arguments to select_related call in get_stream_by_id_in_realm.

Previously, since there was no arguments, it fetched
can_remove_subscribers_group and the related fields of
"Realm" model as well which were not being used, but
did not fetch "recipient" as it is a nullable field.
2023-08-10 17:35:43 -07:00
Sahil Batra ddee3007cb drafts: Access recipient_id when creating or editing Draft objects.
We only need ID of the recipient and not the full object, so we
directly access ID using "stream.recipient_id" instead of using
the complete recipient object.
2023-08-10 17:35:43 -07:00
Sahil Batra 537ecbab45 narrow: Directly access recipient_id from Stream object.
We previously used to access the whole recipient object
unnecessarily in NarrowBuilder.by_stream code when only
ID was needed.
2023-08-10 17:35:43 -07:00
Sahil Batra 91a58d026b models: Remove get_huddle_recipient and use get_or_create_huddle.
This commit removes get_huddle_recipient function and we now use
get_or_create_huddle in get_recipient_from_user_profiles.

As a result of this change, we do not fetch the recipient from
Huddle object but instead get it using the "id" and "recipient_id"
fields available from Huddle object like we do for a personal
message. This change allows us to not fetch recipient object
using select_related when querying the Huddle object.
2023-08-10 17:35:43 -07:00
Sahil Batra 2c28b49680 models: Fetch "recipient" object when along with "Huddle" object.
We now fetch recipient object when querying "Huddle" object in
get_or_create_huddle_backend as this query is eventually used
to get the recipient object only in get_huddle_recipient.

This commit also updates the select_related call in the code to
populate Huddle objects in cache to pass "Recipient" as argument.
Previously no argument was passed to select_related and thus no
related objects were being fetched, with no non-null related fields
being present.
2023-08-10 17:35:43 -07:00
Sahil Batra a3bb5207d2 tests: Check query count for process_missed_message. 2023-08-10 17:35:43 -07:00
David Rosa 347c3dce5e help: Update "View your starred messages" section.
- Adds instructions block and relative link to Starred messages.
- Adds "Toggle starred messages counter" subheading.
- Adds "Searching for messages" as a related article.
2023-08-10 17:31:10 -07:00
Lauryn Menard 2db715030d api-docs: Add Changes notes for stream object fields added in 2.1.0. 2023-08-09 22:07:47 -07:00
Lauryn Menard 67ccd66ed5 api-docs: Add required fields to stream objects.
Adds documentation for which fields are required/expected to be
in stream objects that are checked via examples in the API docs.
2023-08-09 22:07:47 -07:00
Lauryn Menard 4d44a1b277 api-docs: Update "Get all streams" endpoint documentation.
Makes a few updates to the text to match current API documentation
styles.

Updates the endpoint example to have accurate stream objects that
are returned in the response.
2023-08-09 22:07:47 -07:00
Lauryn Menard ec8531e55d api-docs: Fix feature level 199 changelog and add new field to examples.
Fixes formatting of link in feature level 199 changeog entry.

Updates stream object examples for the `stream_weekly_traffic`
field added in feature level 199.
2023-08-09 22:07:47 -07:00
Lauryn Menard b7c9f77860 api-docs: Add missing "date_created" field to stream object examples. 2023-08-09 22:07:47 -07:00
Alex Vandiver 292595afc4 send_custom_email: Add a mode which reads data from JSON.
This allows us to not have to keep extending the tool for every
one-off use case and set of users; we build a pipeline to generate the
appropriate JSON file, write a template which uses the data it
provides, and run the tool with them together.
2023-08-09 15:49:49 -07:00
Alex Vandiver 95b0ab31be send_custom_email: Only put the unsubscribe footer on marketing emails. 2023-08-09 15:49:49 -07:00
Alex Vandiver cb42a1b88d test_email_notifications: These fixtures are markdown files, not HTML. 2023-08-09 15:49:49 -07:00
Alex Vandiver fa41942f4a send_custom_email: Add a way to change the Jinja context for each user. 2023-08-09 15:49:49 -07:00
Alex Vandiver 5a32ea52ae send_custom_email: Stop turning every user query into an id-based set.
The set of objects in the `users` object can be very large (in some
cases, literally every object in the database) and making them into a
giant `id in (...)` to handle the one tiny corner case which we never
use is silly.

Switch the `--users` codepath to returning a QuerySet as well, so it
can be composed.  We pass a QuerySet into send_custom_email as well,
so it can ensure that the realm is `select_related` in as well, no
matter how the QuerySet was generated.
2023-08-09 15:49:49 -07:00
Alex Vandiver 6b25fab38c send_custom_email: Use a mutually-exclusive group for recipients. 2023-08-09 15:49:49 -07:00
Alex Vandiver ae51fbdda2 send_email: Substitute body directly, not via Jinja2.
Substituting the rendered body via Jinja2 means that it cannot
perform any interpolation itself.  While the string replacement is
hacky, it is the only solution which avoids running Jinja2 more than
once, and also allows the user-supplied content to have Jinja2
substitutions in it.
2023-08-09 15:49:49 -07:00
Alex Vandiver def6bf17eb tests: Split out missed_message email tests. 2023-08-09 15:49:49 -07:00
Hemant Umre 63173ce1bc stream_settings: Add 'Default stream' option in create stream UI.
In this commit, we introduce a new option in the stream creation
UI - a 'Default stream for new users' checkbox. By default, the
checkbox is set to 'off' and is only visible to admins. This
allow admins to easily designate a stream as the default stream
for new users during stream creation.

Fixes #24048.
2023-08-09 15:20:09 -07:00
Hemant Umre a81715786c stream_settings: Add 'Default stream' option in edit stream UI.
This commit adds a 'Default stream for new users' checkbox in
the stream editing UI to allow admins to easily add or remove
a stream as the default stream for new users. Previously, this
functionality required navigating to separate menu.

Fixes a part of #24048.
2023-08-09 14:38:52 -07:00
Anders Kaseorg 562a79ab76 ruff: Fix PERF401 Use a list comprehension to create a transformed list.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-07 17:23:55 -07:00
Anders Kaseorg c4748298bb ruff: Fix PERF102 Using only the keys/values of a dict.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-07 17:23:55 -07:00
Anders Kaseorg 55aa29bef4 ruff: Fix FLY002 Consider f"…" instead of string join.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-07 17:12:41 -07:00
David Rosa fa59d4f345 help: Add gear menu icons to relative links. 2023-08-07 11:40:44 -07:00
Lauryn Menard 195e5b8dc1 events: Add test to remove existing value for custom profile field.
Adds a test for when a value for a user's custom profile field is
removed and not set to a new value. The omission of this event in
the tests was noted as a possibility in #22103, which updated the
API documentation for these events having `null` for the field
value.

When adding the test discovered that the events logic was not
deleting the field from the user object and instead setting it to
`None`, so fixes that logic as well. There was a similar bug fixed
in commit 96c61a1a41 for when custom profile fields are removed
from a realm.
2023-08-07 11:39:27 -07:00
Lauryn Menard c1cd5c527c events: Update raw user data fields once.
When applying realm_user update events, some of the event fields
for the person object were being updated to the same value in a
loop. Unnests those calls from the loop over the existing fields
so that they are only updated once.

The original nesting was introduced in commit 649fccde6b and
was expanded in other additions to the logic for these events.
2023-08-07 11:39:27 -07:00
Prakhar Pratyush bdbd405328 test_event_queue: Set the notification settings explicitly.
This commit explicitly sets the following user settings:
* 'enable_followed_topic_email_notifications'
* 'enable_followed_topic_push_notifications'
to True.

Collectively, this improves the readability of the test and
the following two tests.
2023-08-07 10:08:52 -07:00
Anders Kaseorg c2c96eb0cf python: Annotate type aliases with TypeAlias.
This is not strictly necessary but it’s clearer and improves mypy’s
error messages.

https://docs.python.org/3/library/typing.html#typing.TypeAlias
https://mypy.readthedocs.io/en/stable/kinds_of_types.html#type-aliases

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-07 10:02:49 -07:00
Prakhar Pratyush f55711dae3 test_users: Use 'do_change_user_setting' instead of '.save()'.
Use the 'do_change_user_setting' function instead of directly
using '.save()' to change the user settings values.
2023-08-07 10:01:52 -07:00
Prakhar Pratyush ef64d8df97 test_users: Update a test to use the default values for 'hamlet'.
In 'test_copy_default_settings_from_another_user', we verify that
'cordelia' and 'iago' have the same values for their user settings,
but 'hamlet' has the defaults.

Earlier, we explicitly set the 'color_scheme' setting for 'hamlet' as
'UserProfile.COLOR_SCHEME_NIGHT', which is not needed.

As we verify, 'hamlet' should have the defaults.
So just verifying if the 'color_scheme' setting for 'hamlet' is
'UserProfile.COLOR_SCHEME_AUTOMATIC' (default) fulfils our purpose.

The extra line of code was introduced in b10f156.
2023-08-07 10:01:52 -07:00
Sahil Batra ae72151ec1 streams: Pass stream_weekly_traffic field in stream objects.
This commit adds code to pass stream traffic data using
the "stream_weekly_traffic" field in stream objects.

We already include the traffic data in Subscription objects,
but the traffic data does not depend on the user to stream
relationship and is stream-only information, so it's better
to include it in Stream objects. We may remove the traffic
data and other stream information fields for Subscription
objects in future.

This will help clients to correctly display the stream
traffic data in case where client receives a stream
creation event and no subscription event, for an already
existing stream which the user did not have access to before.
2023-08-06 18:06:42 -07:00
Sahil Batra 261fca11ec openapi: Improve description for stream_weekly_traffic.
This commit improves the description for stream_weekly_traffic
field in API documentation to make it clear to the readers about
how to interpret the value.
2023-08-06 18:02:47 -07:00
Sahil Batra 2533e64be6 streams: Remove get_client_data function.
This commit changes the code to not use get_client_data
function and instead use `stream_to_dict` function to
get the stream data in a dictionary form. This is a
prep commit add stream traffic data to Stream objects.
2023-08-06 18:02:47 -07:00
Sahil Batra 8c9ad30411 streams: Add stream_to_dict method.
This commit adds stream_to_dict method which is same as
Stream.to_dict method as of now. This is a prep commit
to include stream traffic data in stream objects.
2023-08-06 18:02:47 -07:00
Tim Abbott d15c4b787f lint: Fix lint issues caught after rebasing.
733083c65d introduced a new lint rule
that the new tests in b67108c8c6 needed
adjustment for.
2023-08-06 13:47:38 -07:00
Ujjawal Modi fc6ce40523 user_groups: Update code for changing group level settings.
Earlier while changing group level group based settings
there was no check if the new value for setting is same as
the current value.

This commit adds this check now a setting value will be only
changed when it is not equal to present value.
2023-08-06 13:42:52 -07:00
Alex Vandiver b67108c8c6 retention: Prevent deletion of partially-archived messages.
Previously, this code:
```python3
old_archived_attachments = ArchivedAttachment.objects.annotate(
    has_other_messages=Exists(
        Attachment.objects.filter(id=OuterRef("id"))
        .exclude(messages=None)
        .exclude(scheduled_messages=None)
    )
).filter(messages=None, create_time__lt=delta_weeks_ago, has_other_messages=False)
```

...protected from removal any ArchivedAttachment objects where there
was an Attachment which had _both_ a message _and_ a scheduled
message, instead of _either_ a message _or_ a scheduled message.
Since files are removed from disk when the ArchivedAttachment rows are
deleted, this meant that if an upload was referenced in two messages,
and one was deleted, the file was permanently deleted when the
ArchivedMessage and ArchivedAttachment were cleaned up, despite being
still referenced in live Messages and Attachments.

Switch from `.exclude(messages=None).exclude(scheduled_messages=None)`
to `.exclude(messages=None, scheduled_messages=None)` which "OR"s
those conditions appropriately.

Pull the relevant test into its own file, and expand it significantly
to cover this, and other, corner cases.
2023-08-06 13:40:02 -07:00
Alex Vandiver 0f918d9071 retention: Do not archive attachments with scheduled messages. 2023-08-06 13:40:02 -07:00
Anders Kaseorg 8ab855a3df event_schema: Remove useless “force vertical” comments.
A trailing comma suffices to tell Black to force vertical
formatting.  (This Black feature is newer than these comments.)

https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#the-magic-trailing-comma

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-02 18:09:22 -07:00
Anders Kaseorg 733083c65d ruff: Collapse short multi-line import statements.
isort did this by default, though it’s unclear whether that was
intended; see https://github.com/astral-sh/ruff/issues/4153.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-02 17:41:41 -07:00
Anders Kaseorg c419c6369e users: Fix TypedDict name for get_accounts_for_email return.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-02 17:37:30 -07:00