Commit Graph

85 Commits

Author SHA1 Message Date
Tim Abbott 0990246289 avatar: Fix memcached query loop fetching bots.
Similar to the related issue with users, the new avatar storage had
accidentally added database queries in a loop to this code path.
2017-05-09 22:33:27 -07:00
Tim Abbott 8e47dc73bd avatar: Fix loop doing database queries in register.
Due to the refactoring of the avatar URL codepath that added realm IDs
to the URLs, we ended up calling `get_user_profile_by_email` inside
`get_avatar_url`, which in turns was called in a loop over all users
in a realm.

Needless to say, this resulted in a significant performance problem.

We fix this issue by passing in the data needed to compute the avatar
URL, rather than looking it up by email address.
2017-05-09 22:33:27 -07:00
Aditya Bansal 821be4519c pep8: Add compliance with rule E261 to cache.py. 2017-05-07 23:21:50 -07:00
Umair Khan 4d543217ba cache: Take hash of KEY_PREFIX to limit key size.
Key size of Memcached records should be less than 256.
2017-05-05 18:23:40 +05:00
hackerkid bf3b2ac673 Include timezone in user_dict fields.
Tweaked by tabbott to avoid adding timezone to bot dicts, since bots
don't need a timezone.
2017-04-14 10:33:55 -07:00
Rishi Gupta 128c431f14 cache.py: Change realm_alert_words_cache_key to use Realm.string_id. 2017-03-13 14:17:14 -07:00
Raghav Jajodia a3a03bd6a5 mypy: Added Dict, List and Set imports.
Fixed mypy errors associated with the upgrade.
2017-03-04 14:33:44 -08:00
Harshit Bansal 9d5be410af page_params: Modify `bot_list` to hold active as well as inactive bots.
Modify the `bot_list` to hold all the bots owned by an user
irrespective of whether the bot is active or inactive. Also
include the `is_active` field in `active_bot_dict_fields` to
distinguish between inactive and active bots.
2017-02-26 23:56:51 -08:00
Steve Howell fa31ad35c9 Fix display of changed avatars in old messages (page_params).
Our client code will now receive avatar_url in
page_params.people_list during page load, so it will be
able to use more current urls for old messages (the client
already had some logic for that and was just missing the
data).

We also add avatar_url to the realm_user/add event.

When we change the avatar, we make sure to always send a
realm_user/update event (even for bots).

We also needed to add avatar_version and
avatar_source to our active users cache.
2017-02-22 07:57:03 -08:00
Steve Howell 3a04831793 Add avatar_version to active_bot_dict_fields. 2017-02-17 10:19:56 -08:00
Umair Khan c585fa6eb4 change-email: Delete display recipient cache. 2017-02-07 21:49:31 -08:00
Umair Khan 41aa07adb6 change-email: Delete email caches on email change. 2017-02-07 18:43:26 -08:00
Tim Abbott e9158dd520 lint: Clean up E121 PEP-8 rule. 2017-01-23 21:02:39 -08:00
Tim Abbott 42612161df tornado: Remove unused caching code. 2017-01-19 16:36:31 -08:00
Juan Verhook cfa9c2eaf2 mypy: Update zerver directory to use Text 2016-12-29 09:12:15 -08:00
Robert Hönig 0917493588 mypy: Convert zerver/lib to use typing.Text. 2016-12-25 10:33:45 -08:00
Sampriti Panda c0326d1938 Add lint rule to disallow python calls with versions (e.g: python2, python3)
Fixes #2435
2016-12-19 08:00:48 -08:00
Umair Khan 770a899239 Django 1.10: Use single cache prefix for casper tests.
There is a change in Django 1.10 due to which whenever the password
of the user is changed the session hash changes. This change affects
us because we cache user profile objects and these cached objects need
to be refreshed. However, the signal sent by Django in which objects are
refreshed fails to refresh the cache for Tornado because it uses a
different cache prefix.

Note: Backend tests are not affected because they don't rely on Tornado.
2016-12-14 22:40:33 -08:00
Igor Tokarev c93f1d4eda Add oembed/Open Graph/Meta tags data retrieval from inline links.
This change adds support for displaying inline open graph previews for
links posted into Zulip.

It is designed to interact correctly with message editing.

This adds the new settings.INLINE_URL_EMBED_PREVIEW setting to control
whether this feature is enabled.

By default, this setting is currently disabled, so that we can burn it
in for a bit before it impacts users more broadly.

Eventually, we may want to make this manageable via a (set of?)
per-realm settings.  E.g. I can imagine a realm wanting to be able to
enable/disable it for certain URLs.
2016-12-07 17:40:18 -08:00
Bickio 6b0df43463 pep8: Fix E125. 2016-11-30 20:03:29 -08:00
Umair Khan 0536aeba4d Django 1.10: Use same cache prefix for JS tests.
Previously, the key prefix was based on the process id due to which
the JS tests couldn't properly flush user profiles from the cache as
our application spans over multiple processes. This problem becomes
apparent when in json_change_settings view after changing the user_profile
the tornado views continue to get the cached user profile corresponding
to their process id.
2016-11-26 15:10:50 -08:00
Umair Khan d81446805c Django 1.10: Use `caches` object to access cache. 2016-11-04 10:06:00 -07:00
Steve Howell ac994fdd51 Move three functions from models.py to lib/cache.py.
I move these three functions to lib/cache.py:

    to_dict_cache_key_id
    to_dict_cache_key
    flush_message

This will prepare us for a more significant refactoring that
eventually breaks down some circular dependencies with
Message and bugdown.
2016-10-04 11:31:20 -07:00
Tim Abbott 22fd7ba02a avatar: Move avatar hash computations to their own file. 2016-10-02 21:19:10 -07:00
Tim Abbott 8442e9249c Reflow annotation for generic_bulk_cached_fetch.
This is a test of mypy's new support for annotating functions that
take lots of arguments.
2016-09-27 20:36:56 -07:00
Tim Abbott 2e6aad669c cache: Add a basic annotation for cache_with_key. 2016-09-10 11:57:08 -07:00
Umair Khan 8e5e6a06f2 Delete cache entry for user profile.
Our flush functions update user profile cache entries which can cause
confusing race conditions (see e.g. #1257).  To resolve this, we move
all the user_profile flush functions to delete the entry instead of
updating it -- it will then be fetched as part of the next request
that needs to access the user object.

There are still races here, and there is perhaps an argument that a
better fix for this would be to re-fetch the object and then put it
into the cache, but this resolves the main cache correctness problem
we had with the previous implementation.

Fixes: #1322.
2016-07-28 13:43:14 -07:00
Eklavya Sharma 674f6999e1 Improve annotations of decorators. 2016-07-22 11:14:33 -07:00
Tim Abbott 014a13df7c cache: Fix echoing of mkdir command to console. 2016-07-18 14:25:13 -07:00
Taranjeet Singh ba3f9de9a9 zerver/lib/cache.py: Move remote_cache_prefix to var directory.
This commit ensures the var directory exists before its needed in both
development and production environments.
2016-07-18 14:13:02 -07:00
Tim Abbott 72e948d19a Remove now-unused message_cache_key message cache.
Originally this cache was used to transmit data from Django to Tornado
(and also for general message caching purposes), but now nothing
actually reads from this cache, so we can eliminate it.
2016-07-08 17:58:56 -07:00
Eklavya Sharma 7ca1e658b5 zerver/lib/cache.py: Change some TypeVars to Any.
Change ItemT and CompressedItemT to Any.
See https://github.com/python/mypy/issues/1721.
2016-06-27 16:50:50 +05:30
Eklavya Sharma f82b28e835 zerver/lib/cache.py: Fix get_cache_backend's annotation. 2016-06-11 09:12:58 -07:00
Eklavya Sharma 0b2d1c30e9 zerver/lib/cache.py: Replace Any with appropriate models.
Due to a cyclic dependency issue, functions having models as parameters
were annotated as Any.
That issue is fixed by importing models inside an `if False:` block,
so that mypy sees them but they are not imported at runtime.
2016-06-11 09:12:58 -07:00
Eklavya Sharma ff4e95d941 Improve generic_bulk_cached_fetch annotation using TypeVars. 2016-06-11 09:12:42 -07:00
Eklavya Sharma d27a0e162a zerver/lib/cache.py: update_user_profile_caches return type is None.
In update_user_profile_caches, the return type in annotation was
marked as Any.  Change that to None because, nothing is being returned
in that function.
2016-06-11 09:11:52 -07:00
Eklavya Sharma 53084fe03c Use text_type as type of cache keys and update users.
This changes the type annotations for the cache keys in Zulip to be
consistently text_type, and updates the annotations for values that
are used as cache keys across the codebase.
2016-06-11 09:10:34 -07:00
Eklavya Sharma d3b80d94a2 Use appropriate string types and correctly encode/decode them. 2016-06-11 17:34:23 +05:30
Eklavya Sharma 286d23734a zerver/lib/cache.py: Remove unneeded return statements. 2016-06-09 16:57:11 -07:00
Ashish Kumar 31bf6b8259 Type annotation of zerver/models.py
[Substantially revised by tabbott]

This probably still has some bugs in it, but having mostly complete
annotations for models.py will help a lot for the annotations folks
are adding to other files.
2016-06-02 23:28:34 -07:00
Ashish Kumar cad342aff6 Correct annotation of generic_bulk_cached_fetch in zerver/lib/cache.py.
Previously, object_ids was tagged as an int, but it is called from
models.py with a string, so we make it an Any.
2016-06-01 14:00:49 -07:00
Eklavya Sharma 1ea6171179 Fix an annotation in zerver/lib/cache.py.
This is done to make annotations in zerver/lib/actions.py work correctly.
2016-05-25 15:11:48 -07:00
Tim Abbott 2409ac9b2f cache: Add type annotations to active_*_dict_fields. 2016-05-10 11:48:03 -07:00
Tim Abbott 0161d2fddd Cleanup guardian-based complexity in get_realm_user_dicts.
The old code for this lookup was unnecessarily complicated because we
were working around Guardian, where the `is_realm_admin` check was
extremely expensive.
2016-05-09 10:12:35 -07:00
Tim Abbott 2a2cbd60c3 cache: Fix fragile active_bot_dicts_in_realm caching model.
The issue here is similar to that in the previous commit.
2016-05-09 10:12:35 -07:00
Tim Abbott fbc7e977ac cache: Fix fragile active_user_dicts_in_realm caching model.
Previously we relied on having two matching list of fields for the
get_active_user_dicts_in_realm, one in the actual code and the other
in the caching system.  By unifying these lists to have a single
source, we eliminate a class of caching bugs we might otherwise
regularly introduce.
2016-05-09 10:12:35 -07:00
Ashish Kumar 31408d639e Type annotation of zerver/lib/cache.py. 2016-04-29 14:43:48 -07:00
Tim Abbott 552caf661a Caching: Fix 'update_fields' not being present in .delete() 2016-04-20 15:12:53 -07:00
Ryan Moore 18139fd86f s/memcached_prefix/remote_cache_prefix/g 2016-03-31 12:54:29 -07:00
Ryan Moore 1f120aa3a8 s/memcached_total/remote_cache_total/g 2016-03-31 12:54:29 -07:00