Mirror
/
zulip
mirror of https://github.com/zulip/zulip.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
47,098 Commits 28 Branches 142 Tags 769 MiB
Commit Graph

47098 Commits

Author SHA1 Message Date
Alex Vandiver 04ca2e92f7 caches: Cache link preview data in memcached, not in PostgreSQL. 
The choice to cache these in the database dates back to c93f1d4eda,
with the comment added in da33b72848 while working around the
durability of the "database" cache in local development.

The values were stored in a durable cache, as they needed to be
ensured to persist between when they were inserted in
`get_link_embed_data` and when they were used in
`render_incoming_message` via `link_embed_data_from_cache`.

However, database accesses are not fast compared to memcached, and we
wish to avoid the overhead of the database connection from the
`embed_links` worker.  Specifically, making the connection may not be
thread-safe -- and in low-memory (and Docker) configurations, all
workers run as separate threads in a single process.  This can lead to
stalled database connections in `embed_links` workers, and failed
previews.

Since the previous commit made the durability of the cache no longer
necessary, this will have minimal effect; at worst, posting the same
URL twice, on either side of an upgrade, will result in two preview
fetches of it.
 2022-04-15 14:48:12 -07:00
Alex Vandiver 351bdfaf78 preview: Use cache only as a non-durable cache, not an IPC. 
The `get_link_embed_data` / `link_embed_data_from_cache` pair as
introduced in c93f1d4eda uses the cache
as a temporary store inside of the `embed_links` worker; this means
that it must be durable storage, or the worker will stall and re-fetch
the same links to preview them.

Switch to plumbing through the fetched URL embed data as an parameter
to the Markdown evaluation which uses them, rather than using the
cache as an intermediary.  This frees up the cache to be merely a
non-durable cache.

As a side-effect, this removes get_cache_with_key, and
link_embed_data_from_cache which was its only callsite.
 2022-04-15 14:48:12 -07:00
Alex Vandiver 327ff9ea0f preview: Use a dataclass for the embed data. 
This is significantly cleaner than passing around `Dict[str, Any]` all
of the time.
 2022-04-15 14:48:12 -07:00
Alex Vandiver ede4a88b49 populate_db: Remove unnecessary pre-population of URL cache. 
76deb30312 changed this to not just be the URL, but rather a
prefixed hash of the URL, but failed to update this location which
wrote to it.  This meant that this pre-population step was writing to
the wrong keys in the durable cache, and thus ineffective.

Then, da33b72848 switched the cache to be in-memory, making this
write to the wrong keys in an in-process memory store.  There is no
way to pre-fill this sort of cache, except at server start-up.

Finally, and most fundamentally, 8c0c9ca7a4 then disabled
`inline_url_embed_preview` by default, making the code entirely moot.

Remove the triply-unnecessary code.
 2022-04-15 14:48:12 -07:00
Alex Vandiver aaa58a49db cache: Make the cache_name=None behaviour clearer. 
`django.core.cache.cache` is equal to
`django.core.cache.caches["default"]`; the latter is more
understandable in context.
 2022-04-15 14:48:12 -07:00
Alex Vandiver 661c333377 markdown: Use named parameters to add_a helper. 
This has enough parameters that it benefits from making which is which
explicit.
 2022-04-15 14:48:12 -07:00
Alex Vandiver 452a30305d markdown: Clarify url parameter of "add_a" helper. 2022-04-15 14:48:12 -07:00
Mateusz Mandera fcf82bf047 digest: Don't send emails to deactivated users, even if queued. 2022-04-15 14:32:55 -07:00
Mateusz Mandera 7a13836d26 test_digest: Fix typo in a comment. 2022-04-15 14:32:55 -07:00
Anders Kaseorg cc30ed8ec7 actions: Delete zerver.lib.actions. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:38 -07:00
Anders Kaseorg 729019acdd actions: Move part into zerver.lib.test_classes. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:37 -07:00
Anders Kaseorg e01faebd7e actions: Split out zerver.actions.create_realm. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:37 -07:00
Anders Kaseorg 53f4a395bc actions: Split out zerver.actions.realm_domains. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:37 -07:00
Anders Kaseorg 59f6b090c7 actions: Split out zerver.actions.realm_settings. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:37 -07:00
Anders Kaseorg 12de8d797e actions: Move part into zerver.forms. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:36 -07:00
Anders Kaseorg eda000899b actions: Split out zerver.actions.message_edit. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:36 -07:00
Anders Kaseorg 5d1a5a3877 actions: Split out zerver.actions.muted_users. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:36 -07:00
Anders Kaseorg ec174dfb47 actions: Split out zerver.actions.bots. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:36 -07:00
Anders Kaseorg eb4e9fe1e7 actions: Split out zerver.actions.message_flags. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:36 -07:00
Anders Kaseorg e5500a2226 actions: Split out zerver.actions.reactions. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:35 -07:00
Anders Kaseorg cbad5739ab actions: Split out zerver.actions.create_user. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:35 -07:00
Anders Kaseorg 5fcbc412cf actions: Split out zerver.actions.streams. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:35 -07:00
Anders Kaseorg 975066e3f0 actions: Split out zerver.actions.message_send. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:34 -07:00
Anders Kaseorg ec6355389a actions: Split out zerver.actions.user_settings. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:34 -07:00
Anders Kaseorg d7981dad62 actions: Split out zerver.actions.users. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:34 -07:00
Anders Kaseorg bbce879c81 actions: Split out zerver.actions.custom_profile_fields. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:33 -07:00
Anders Kaseorg f6a06ba6e3 actions: Move part into zerver.lib.bulk_create. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:33 -07:00
Anders Kaseorg c041b68578 actions: Move part into zerver.lib.message. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:33 -07:00
Anders Kaseorg 9dd7e34ab3 actions: Move part into zerver.lib.subscription_info. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:33 -07:00
Anders Kaseorg b7adfb02f6 actions: Split out zerver.actions.presence. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:32 -07:00
Anders Kaseorg ab04068294 actions: Move part into zerver.lib.users. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:32 -07:00
Anders Kaseorg e230ea2598 actions: Split out zerver.actions.uploads. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:32 -07:00
Anders Kaseorg a29f1b39da actions: Move part into zerver.lib.streams. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:32 -07:00
Anders Kaseorg 6168c0110a actions: Split out zerver.actions.user_activity. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:32 -07:00
Anders Kaseorg df4849bb15 actions: Split out zerver.actions.user_topics. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:32 -07:00
Anders Kaseorg 385616f27f actions: Split out zerver.actions.realm_emoji. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:31 -07:00
Anders Kaseorg 8fc5922ebd actions: Split out zerver.actions.realm_export. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:31 -07:00
Anders Kaseorg 3d7aa98c45 actions: Split out zerver.actions.realm_icon. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:31 -07:00
Anders Kaseorg 7f088f3403 actions: Split out zerver.actions.realm_logo. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:31 -07:00
Anders Kaseorg ca8d374e21 actions: Split out zerver.actions.invites. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:31 -07:00
Anders Kaseorg 241463e215 actions: Split out zerver.actions.alert_words. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:31 -07:00
Anders Kaseorg 1ac7496855 actions: Split out zerver.actions.default_streams. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:31 -07:00
Anders Kaseorg 12130da339 actions: Split out zerver.actions.hotspots. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:31 -07:00
Anders Kaseorg 975f5a3c2d actions: Split out zerver.actions.realm_linkifiers. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:31 -07:00
Anders Kaseorg e887abcf41 actions: Split out zerver.actions.realm_playgrounds. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:30 -07:00
Anders Kaseorg 3a135b04d9 actions: Split out zerver.actions.submessage. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:30 -07:00
Anders Kaseorg 62d3b5bfd5 actions: Split out zerver.actions.typing. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:30 -07:00
Anders Kaseorg 372c10f5f3 actions: Split out zerver.actions.user_groups. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:30 -07:00
Anders Kaseorg 90cae59ea6 actions: Split out zerver.actions.video_calls. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:30 -07:00
Anders Kaseorg c136eebb33 actions: Split out zerver.lib.recipient_users. 
Signed-off-by: Anders Kaseorg <anders@zulip.com>
 2022-04-14 17:14:30 -07:00