Commit Graph

52676 Commits

Author SHA1 Message Date
Zixuan James Li 011b4c1f7a populate_db: Populate linkifiers.
The curl examples of reordering linkifiers require there to be some
linkifiers in the database to be reordered. This adjusts some test cases
so they do not assume that there is no linkifier in the test db.
2023-08-14 15:21:48 -07:00
Zixuan James Li 112a6e445e linkifiers: Support multiple linkifiers in puppeteer tests.
Select the linkifier with `:nth-child` assuming the possibility of other
linkifiers in the realm. This prepares for a later change that requires
us to populate the test database with linkifiers.
2023-08-14 15:21:48 -07:00
Zixuan James Li aa5765ca64 test_openapi: Use subtests for arguments test.
Each unittest subTest can fail without interrupting the other subTests.
By wrapping the test for each view function, we can get all validation
errors at once, which can be useful if multiple endpoints are updated.

More importantly, if the test fails anywhere inside test_openapi but
before the formatted output is printed, we will not lose the information
of which view function fails the validation. Because we attach the name
of the function to the subTest:

```
FAIL: test_openapi_arguments (zerver.tests.test_openapi.OpenAPIArgumentsTest) [zerver.views.alert_words.add_alert_words]
```
2023-08-14 14:24:16 -07:00
Zixuan James Li 6c9e89e6c5 test_openapi: Extract test_openapi_arguments_for_endpoint. 2023-08-14 14:24:16 -07:00
Alex Vandiver 570ff08fde topic: Set a max batch_size on bulk_upate call.
The number of affected objects may be quite high, and they are
selected by `id IN (...)` query, and updated with a giant `CASE`.
This turns out to be quadratic, and can cause large queries to take
hours, in a state where they cannot be terminated, when PostgreSQL >11
tries to JIT the query.

Set a batch_size as a stopgap performance fix before moving to
`.update()` as a real fix.
2023-08-14 13:33:20 -07:00
Satyam Bansal 25cc0e4030 integrations-docs: Make use of consistent spacing. 2023-08-14 13:29:59 -07:00
Satyam Bansal 2ba2646730 integrations-docs: Make use of consistent numbering for list. 2023-08-14 13:29:59 -07:00
David Rosa d205850d54 docs: Update .html links pointing to "Upgrade Zulip" or "Modify Zulip".
Follow-up to #24089.
2023-08-11 16:49:32 -07:00
evykassirer 4c2995c613 stream settings: Warn users before locking them out of a stream.
Organization owners can make streams private even if they're not
subscribed to them, but cannot access private streams they're not
subscribed to. This means they're able to lock themself out of streams.

This change warns users of this and give them a chance to subscribe.

Fixes #26437.
2023-08-11 16:46:41 -07:00
Zixuan James Li e8b7aad462 requests: Split out test_has_request_variables. 2023-08-11 16:43:12 -07:00
Zixuan James Li c9a299a8f8 validators: Split out test_validators. 2023-08-11 16:43:12 -07:00
Satyam Bansal fe39f28497 drafts: Rename class name to provide more context. 2023-08-11 15:54:17 -07:00
Satyam Bansal ee0d45d13c drafts: Increase tippy delay for "Delete Drafts" icon.
Previously, the tippy delay was too short, resulting in
the tippy being flashed for an instant, which could lead
to confusion for the users.
2023-08-11 15:54:17 -07:00
Julia Bichler fed866449d drafts: Add functionality to bulk delete drafts.
Fixes #19360.

Co-authored-by: Satyam Bansal <sbansal1999@gmail.com>
2023-08-11 15:54:17 -07:00
Satyam Bansal e4048de0a9 app_components: Convert "overlay_message_controls" div to flexbox.
Previously, the icons were spaced by hardcoded values of margins
and would require changes everytime a new icon is added.
2023-08-11 15:54:17 -07:00
Satyam Bansal b4be3f6266 drafts: Remove unused CSS styles. 2023-08-11 15:54:17 -07:00
Satyam Bansal 0f0c52e096 drafts: Remove classes that had the same effect.
In commit 5edc8fc, abstract classes were added to the
drafts overlay, but the classes that were already there
were not removed. This resulted in the same styles being
applied twice.
2023-08-11 15:54:17 -07:00
Daniil Fadeev 28ba839857 condense: Focus on the message after condensing/expanding it. 2023-08-11 12:39:43 -07:00
Lauryn Menard 225be0bb97 dropdown-list-widget: Remove dropdown-list-body class and reference.
Removes the "dropdown-list-body" class and references as it is no
longer used in the new DropdownWidget. The previous uses of the
class were removed in commit 875d564f2d.
2023-08-11 12:26:47 -07:00
David Rosa d24a689e9e help: Add heading on linking to messages in "Writing messages" section.
We should have a clear reference in the "Writing messages" sidebar
section on linking to other conversations below "Mention a user
or group" to clarify mentioning users vs linking to conversations.

This adds a link to the "Link to a message or conversation" article
with a different heading.
2023-08-11 11:13:22 -07:00
Lauryn Menard 3d997c4af2 docs: Add multiple-organizations.md to production section of TOC.
Adds the existing article about hosting multiple Zulip organizations
to the production section of the table of contents.
2023-08-11 11:11:44 -07:00
Steve Howell 51db22c86c per-request caches: Add per_request_cache library.
We have historically cached two types of values
on a per-request basis inside of memory:

    * linkifiers
    * display recipients

Both of these caches were hand-written, and they
both actually cache values that are also in memcached,
so the per-request cache essentially only saves us
from a few memcached hits.

I think the linkifier per-request cache is a necessary
evil. It's an important part of message rendering, and
it's not super easy to structure the code to just get
a single value up front and pass it down the stack.

I'm not so sure we even need the display recipient
per-request cache any more, as we are generally pretty
smart now about hydrating recipient data in terms of
how the code is organized. But I haven't done thorough
research on that hypotheseis.

Fortunately, it's not rocket science to just write
a glorified memoize decorator and tie it into key
places in the code:

    * middleware
    * tests (e.g. asserting db counts)
    * queue processors

That's what I did in this commit.

This commit definitely reduces the amount of code
to maintain. I think it also gets us closer to
possibly phasing out this whole technique, but that
effort is beyond the scope of this PR. We could
add some instrumentation to the decorator to see
how often we get a non-trivial number of saved
round trips to memcached.

Note that when we flush linkifiers, we just use
a big hammer and flush the entire per-request
cache for linkifiers, since there is only ever
one realm in the cache.
2023-08-11 11:09:34 -07:00
Steve Howell 751b8b5bb5 tests: Flush per-request caches automatically for query counts. 2023-08-11 11:09:34 -07:00
Steve Howell 730ae61ce5 tests: Improve linkifiers test.
We test at a higher level now.
2023-08-11 11:09:34 -07:00
Steve Howell 549891266d tests: Add assert_memcached_count.
We use a specific name to distinguish from other caches
like per-request caches.
2023-08-11 11:09:34 -07:00
Steve Howell 0eea42b48c tests: Remove spurious nocoverage directive. 2023-08-11 11:00:57 -07:00
Steve Howell 031e3ae3f0 caching docs: Tweak a few things.
Some claims seemed a little exaggerated or overly
precise.
2023-08-11 10:57:42 -07:00
Tim Abbott 0a181bca86 tests: Fix a query count incorrect due to rebase. 2023-08-10 18:32:10 -07:00
Steve Howell f8ec00b895 mypy: Improve type checks for user display recipients. 2023-08-10 18:13:43 -07:00
Steve Howell 1b7880fc21 push notifications: Go to the DB for streams.
We want to phase out the use of get_display_recipient
for streams, and this is the last place that I
eliminate it. The next commit will eliminate the
dead code and make mypy types tighter.

This change will make push notifications slightly
slower in some situations, but we avoid all the
complexity of a cache, and this code tends to run
offline.

We could always make this code a bit more efficient
by being a little smarter about what data we fetch
up front. For example, get_apns_alert_title gets
called by a function that already has the stream
name. It's just a bit of a pain to refactor when
you have all the DM codepath mucked up with the
stream codepath.
2023-08-10 18:13:43 -07:00
Steve Howell 63c0ed303d tests: Mock Recipient.label.
We may eventually want to decouple how we send recipients
over the wire from how we represent them in debugging.
2023-08-10 18:13:43 -07:00
Steve Howell 5b569ab865 cache: Stringify stream recipients without the cache.
We generally want to avoid extra moving parts when we
stringify objects. We also want to phase out the use
of get_display_recipient for streams.

Note that we still hit get_display_recipient to
stringify DM and huddle objects, and it's kind of ugly
how we do it, but that's outside the scope of my
current PR.
2023-08-10 18:13:43 -07:00
Steve Howell 8295b0d46e tests: Simplify how we get active streams.
There's no need for the complexity and extra round
trips to call get_display_recipient in a testing
context.

We also eliminate the unnecessary call to check_string.

This function is poorly named, but that's a sweep
for another day.
2023-08-10 18:13:43 -07:00
Steve Howell a54760da0e tests: Add assert_message_stream_name
The get_display_recipient helper is a clumsy way to get
stream names, and it's not even representative of how
most of our code retrieves stream names.

The new helper also double-checks that the Stream
object has the correct recipient id.
2023-08-10 18:13:43 -07:00
Steve Howell df068ae7a5 tests: Test directly for stream name. 2023-08-10 18:13:43 -07:00
Steve Howell 7c864db8f2 email mirror: Avoid silly email lookup.
We can search by id, which is more resilient and still
hits a cache.
2023-08-10 18:13:43 -07:00
Steve Howell 257b32a4a4 narrow urls: Avoid complicated optional types.
We no longer have to reason about the 12 possible
ways of invoking get_narrow_url. We also avoid
double computation in a couple places.

Finally, we get stricter type checks by just inlining
the calls.
2023-08-10 18:13:43 -07:00
Steve Howell 6ff7c17f82 tests: Avoid Union type to verify stream names.
There's also no need to fetch a full Stream object when
the thing being verified is just that the display_recipient
field matches the stream name.
2023-08-10 18:13:43 -07:00
Steve Howell 233486f7b3 push notifications: Rename variable. 2023-08-10 18:13:43 -07:00
Steve Howell 6be2a08ed8 cache: Avoid cache spam for push notifications.
We don't need to call get_display_recipient for
non-stream messages.

I will rename display_recipient in the next commit;
if I were to combine the steps the diff would be too
hard to read.
2023-08-10 18:13:43 -07:00
Steve Howell 538f498447 tests: Fix clumsy narrow test.
We now explicitly write messages to three different streams,
as well a DM, to make sure each narrow result filters out
all the noise.
2023-08-10 18:13:43 -07:00
Prakhar Pratyush 860eee94fd notifications: Rename 'pm' to 'dm' in 'RecipientInfoResult' dataclass.
This commit renames the keyword 'pm' to 'dm' in the
'pm_mention_email_disabled_user_ids' and
'pm_mention_push_disabled_user_ids' attributes of the
'RecipientInfoResult' dataclass.

'pm' and 'dm' are the acronyms for 'private message' and
'direct message' respectively.

It includes 'TODO/compatibility' code to support the old format
fields in the tornado queues during the Zulip server upgrades.
2023-08-10 17:41:49 -07:00
Prakhar Pratyush c4e4737cc6 notification_trigger: Rename `private_message` to `direct_message`.
This commit renames the 'PRIVATE_MESSAGE' attribute of the
'NotificationTriggers' class to 'DIRECT_MESSAGE'.

Custom migration to update the existing value in the database.

It includes 'TODO/compatibility' code to support the old
notification trigger value 'private_message' in the
push notification queue during the Zulip server upgrades.

Earlier 'private_message' was one of the possible values for the
'trigger' property of the '[`POST /zulip-outgoing-webhook`]' response;
Update the docs to reflect the change in the above-mentioned trigger
value.
2023-08-10 17:41:49 -07:00
Prakhar Pratyush 3675a44471 push_notifications: Add the missing compatibility code.
This commit adds 'TODO/compatibility' code to support the
old notification trigger values in the push notification queue
during the Zulip server upgrades.

In f4fa82e, we renamed the following notification triggers:
* 'wildcard_mentioned' to 'stream_wildcard_mentioned'
* 'followed_topic_wildcard_mentioned' to
'stream_wildcard_mentioned_in_followed_topic'.

This should have been added in f4fa82e.
2023-08-10 17:41:49 -07:00
Sahil Batra 0e23280b4f tests: Pass required args to select_related call for Message objects.
This commit adds code to pass all the required arguments to
select_related call for Message objects such that only the
required related fields are fetched from the database.

Previously, we did not pass any arguments to select_related,
so all the directly and indirectly related fields were fetched
when many of them were actually not being used and made the
query unnecessarily complex.
2023-08-10 17:35:43 -07:00
Sahil Batra a6df377688 openapi: Pass required arguments to select_related call.
This commit adds code to pass all the required arguments to
select_related call for Message objects such that only the
required related fields are fetched from the database.

Previously, we did not pass any arguments to select_related,
so all the directly and indirectly related fields were fetched
when many of them were actually not being used and made the
query unnecessarily complex.
2023-08-10 17:35:43 -07:00
Sahil Batra 7ff5423e21 topic: Pass args to select_related call for Message objects.
This commit adds code to pass all the required arguments to
select_related call for Message objects such that only the
required related fields are fetched from the database.

Previously, we did not pass any arguments to select_related,
so all the directly and indirectly related fields were fetched
when many of them were actually not being used and made the
query unnecessarily complex.
2023-08-10 17:35:43 -07:00
Sahil Batra 36f8aba7db message: Pass args to select_related call for Message objects.
This commit adds code to pass all the required arguments to
select_related call for Message objects such that only the
required related fields are fetched from the database.

Previously, we did not pass any arguments to select_related,
so all the directly and indirectly related fields were fetched
when many of them were actually not being used and made the
query unnecessarily complex.
2023-08-10 17:35:43 -07:00
Sahil Batra 35559ae324 export: Pass "user_profile" as arg to select_related call for Subscription.
This commit adds code to pass all the required arguments to
select_related call for Subscription objects such that only
the required related fields are fetched from the database.

Previously, we did not pass any arguments to select_related,
so all the directly and indirectly related fields were fetched
when many of them were actually not being used and made the
query unnecessarily complex.
2023-08-10 17:35:43 -07:00
Sahil Batra ebd91b152c email_mirror: Pass required args to select_related.
This commit adds code to pass all the required arguments to
select_related call for MissedMessageEmailAddress such that
only the required related fields are fetched from the database.

Previously, we did not pass any arguments to select_related,
so all the directly and indirectly related fields were fetched
when many of them were actually not being used and made the
query unnecessarily complex.
2023-08-10 17:35:43 -07:00