I move these three functions to lib/cache.py:
to_dict_cache_key_id
to_dict_cache_key
flush_message
This will prepare us for a more significant refactoring that
eventually breaks down some circular dependencies with
Message and bugdown.
Our flush functions update user profile cache entries which can cause
confusing race conditions (see e.g. #1257). To resolve this, we move
all the user_profile flush functions to delete the entry instead of
updating it -- it will then be fetched as part of the next request
that needs to access the user object.
There are still races here, and there is perhaps an argument that a
better fix for this would be to re-fetch the object and then put it
into the cache, but this resolves the main cache correctness problem
we had with the previous implementation.
Fixes: #1322.
Originally this cache was used to transmit data from Django to Tornado
(and also for general message caching purposes), but now nothing
actually reads from this cache, so we can eliminate it.
Due to a cyclic dependency issue, functions having models as parameters
were annotated as Any.
That issue is fixed by importing models inside an `if False:` block,
so that mypy sees them but they are not imported at runtime.
In update_user_profile_caches, the return type in annotation was
marked as Any. Change that to None because, nothing is being returned
in that function.
This changes the type annotations for the cache keys in Zulip to be
consistently text_type, and updates the annotations for values that
are used as cache keys across the codebase.
[Substantially revised by tabbott]
This probably still has some bugs in it, but having mostly complete
annotations for models.py will help a lot for the annotations folks
are adding to other files.
The old code for this lookup was unnecessarily complicated because we
were working around Guardian, where the `is_realm_admin` check was
extremely expensive.
Previously we relied on having two matching list of fields for the
get_active_user_dicts_in_realm, one in the actual code and the other
in the caching system. By unifying these lists to have a single
source, we eliminate a class of caching bugs we might otherwise
regularly introduce.
The function update_user_profile_caches now operates on a list
of user_profiles, so callers like flush_realm() can benefit from
having a single cache_set_many() call. This slightly complicates
the call from flush_user_profile().
(imported from commit e064871d849b873c6ca388f00d4f7afaba1bf222)
For the realm-wide caches of active user dicts and alert words, just
make a single call to cache_delete() when you are deactivating a
realm. Before this change, we were doing O(N) cache_deletes as
part of the code path through flush_user_profile(). Now we just
call update_user_profile_caches() directly to clear the user_profile
caches.
This change also sets us up to turn flush_realm() into a post-save hook.
(imported from commit 699b4ea226ae15fc8c402cb4bc64ff6bdc041fc2)
This is a slight behavior change, as we now flush user_profile
caches for bots as well as humans.
(imported from commit 24c72c44d851ee4c66a67a4728cd6c548faeedcd)
This function updates all the user_profile-related caches
that are keyed on a per-user basis.
(This had some test coverage already.)
(imported from commit 37979400514a7b46a6dcb7e36665b0fee2f3c525)
The do_send_messages() populates the user_presences data structure
for process_new_message(), so that Tornado code never needs to hit
the database or memcached to get the user presence info.
(imported from commit 194aeaead8fa712297a2ee8aff5aa773b92f1207)
This reduces the number of memcached calls we make in our time-
slice-limited tornado event handler.
(imported from commit 8903ce4ac754ba82d57e04d1b0356be7533edee2)
We found that since bugdown processes are threaded, the cost of
doing a db query in a markdown processor is quite high---each
thread must start up a new db connection including a SSL handshake
etc. We should strive to keep our rendering pipeline free of mandatory
DB queries.
(imported from commit 555066bd03da6c681b74ce6137acc264eb41c55d)
This needs to be deployed to both staging and prod at the same
off-peak time (and the schema migration run).
At the time it is deployed, we need to make a few changes directly in
the database:
(1) UPDATE django_content_type set app_label='zerver' where app_label='zephyr';
(2) UPDATE south_migrationhistory set app_name='zerver' where app_name='zephyr';
(imported from commit eb3fd719571740189514ef0b884738cb30df1320)
Steve Howell
Tim Abbott
Umair Khan
Eklavya Sharma
Tim Abbott
Taranjeet Singh
Ashish Kumar
Ryan Moore
Jason Michalski
Steve Howell
Jessica McKellar
Leo Franchi
Jessica McKellar
Tim Abbott