Commit Graph

144 Commits

Author SHA1 Message Date
Vaibhav 61371cbe9a user_info: Add `bot_owner_id` to user info dataset.
Modifies the dict with the user info to include the key `bot_owner_id`
so it can be displayed in the user info popover.

Tests concerned with changing bot owner have been modified to have
number of events=2 because while updating the bot info, two events
are fired -- updating the `realm_bot` and `realm_user` since the
key `bot_owner_id` is a part of realm user info.
2019-05-28 12:41:52 -07:00
Puneeth Chaganti 5f50c4d799 cache: Clear realm descriptions when description is changed.
Clear both the rendered and the text descriptions for a realm, when the
description of the realm is changed.
2019-05-17 11:31:14 -07:00
Puneeth Chaganti f084075c2a cache: Clear rendered realm text description when flushing realm.
The commit 2d9c5b3575 introduced this cache key,
and didn't setup clearing of the cache correctly.
2019-05-17 11:31:14 -07:00
Puneeth Chaganti 2d9c5b3575 open graph: Display realm description as open graph description. 2019-04-25 15:29:16 -07:00
Anders Kaseorg 643bd18b9f lint: Fix code that evaded our lint checks for string % non-tuple.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:21:37 -07:00
Puneeth Chaganti 184c0b9870 render_markdown_path: Convert dicts to dict item tuples for caching.
Calls to `render_markdown_path` weren't getting cached since the context
argument is unhashable, and the `ignore_unhashable_lru_cache` decorator ignores
such calls. This commit adds a couple of more decorators - one which converts
dict arguments to the function to a dict items tuple, and another which converts
dict items tuple arguments back to dicts. These two decorators used along with
the `ignore_unhashable_lru_cache` decorator ensure that the calls to
`render_markdown_path` with the context dict argument are also cached.

The time to run zerver.tests.test_urls.PublicURLTest.test_public_urls drops by
about 50% from 8.4s to 4.1s with this commit. The time to run
zerver.tests.test_docs.DocPageTest.test_doc_endpoints drops by about 20% from
3.2s to 2.5s.
2019-04-22 13:53:18 -07:00
Hemanth V. Alluri 261166f96d context_processors: Cache the realm description in default context. 2019-03-18 22:19:18 -07:00
Tim Abbott 51eeb0a3ee cache: Add missing : in test-backend key prefixes.
Previously, these cache keys looked like:
:1:9c26164d3a393e316e0f8210efe270e08710d45astream_by_realm_and_name:...

Now, they look like this:
:1:9c26164d3a393e316e0f8210efe270e08710d45a:stream_by_realm_and_name:...
2019-03-18 10:56:50 -07:00
Vishnu Ks 94ae2dc24e models: Cache currently_used_upload_space_bytes function. 2019-03-04 18:46:13 -08:00
Bennet Sunder 7c5f316cb8 alert_words: Performance improvements in looking for alert_words.
This commit leverages the ahocorasick algorithm to build a set of user_ids
that have their alert_words present in the message. It runs in linear time
of the order of length of the input message as opposed to number of
alert_words. This is after building a ahocorasick Automaton which runs
in O(number of alert_words in entire realm) which is usually cached.
2019-03-01 15:36:39 -08:00
Anders Kaseorg f0ecb93515 zerver core: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:41:24 -08:00
Wyatt Hoodes 8eac361fb5 docs: Refactor BS work with use of cache_with_key.
Refactor the potentially expensive work done by Beautiful Soup into a
function that is called by the alter_content function, so that we can
cache the result.  Saves a significant portion of the runtime of
loading of all of our /help/ and /api/ documentation pages (e.g. 12ms
for /api).

Fixes #11088.

Tweaked by tabbott to use the URL path as the cache key, clean up
argument structure, and use a clearer name for the function.
2019-01-28 15:21:52 -08:00
Tim Abbott a0da4f6d30 python: Clean up various if False blocks.
Most of these are now-unnecessary typing imports; some are just
improved comments for those with other mypy motivations.
2018-12-17 11:14:47 -08:00
Vishnu Ks 8a1794caa3 message: Store the value of first_visible_message_id in Realm table.
This eliminates a bunch of potentially buggy caching code, with no
material negative side effects.
2018-12-12 15:11:17 -08:00
Tim Abbott e603237010 email: Convert accounts code to use delivery_email.
A key part of this is the new helper, get_user_by_delivery_email.  Its
verbose name is important for clarity; it should help avoid blind
copy-pasting of get_user (which we'll also want to rename).
Unfortunately, it requires detailed understanding of the context to
figure out which one to use; each is used in about half of call sites.

Another important note is that this PR doesn't migrate get_user calls
in the tests except where not doing so would cause the tests to fail.
This probably deserves a follow-up refactor to avoid bugs here.
2018-12-06 16:21:38 -08:00
Tim Abbott 209dd5db67 actions: Add a function for changing realm subdomains.
This is initial work, which will help us establish habits of using a
well-tested approach for renaming a Zulip organization (since as part
of https://github.com/zulip/zulip-mobile/issues/3142, we'll likely
need to make this function do more).
2018-11-15 14:39:14 -08:00
Pragati Agrawal d5df0377cc settings_users: Support guest user in admin-user-table.
This supports guest user in the user-info-form-modal as well as in the
role section of the admin-user-table.

With some fixes by Tim Abbott and Shubham Dhama.
2018-10-29 12:33:35 -07:00
Steve Howell 76deb30312 preview: Hash cache keys for preview urls.
We don't want really long urls to lead to truncated
keys, or we could theoretically have two different
urls get mixed up previews.

Also, this suppresses warnings about exceeding the
250 char limit.

Finally, this gives the key a proper prefix.
2018-10-14 09:28:57 -07:00
Yago González f6219745de users: Get all API keys via wrapper method.
Now reading API keys from a user is done with the get_api_key wrapper
method, rather than directly fetching it from the user object.

Also, every place where an action should be done for each API key is now
using get_all_api_keys. This method returns for the moment a single-item
list, containing the specified user's API key.

This commit is the first step towards allowing users have multiple API
keys.
2018-08-08 16:35:17 -07:00
Tim Abbott 6f7e12ea19 docs: Add subsystem documentation for caching. 2018-07-31 17:00:45 -07:00
Anders Kaseorg 195cc78470 zerver/lib/cache.py: Avoid shelling out for mkdir.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-19 10:43:37 -07:00
Shubham Dhama 01555e8772 streams: Handle guest user ids for stream settings changes' events. 2018-06-04 11:35:37 -07:00
Vishnu Ks 372e9740ac events: Add date_joined to user_dict. 2018-05-17 07:49:35 -07:00
Steve Howell 4332fd64f7 Add submessages to message payloads. 2018-05-16 15:13:33 -07:00
Aditya Bansal a68376e2ba zerver/lib: Change use of typing.Text to str. 2018-05-12 15:22:39 -07:00
Tim Abbott 707af5ab56 cache: Remove a now-unnecessary TODO.
We solved the problem the TODO raised by using a different type
annotation syntax, and I'm not sure whether that refactor would
actually improve the code.
2018-03-16 11:32:14 -07:00
neiljp (Neil Pilgrim) 966ca7015f mypy: Finalize migration of cache.py to python3 function annotation.
- Use forward declarations of some types from models.py to avoid cycles.
- Remove cache.py from linter rule exclude list to ensure it stays that way.
2018-03-16 11:29:12 -07:00
neiljp (Neil Pilgrim) 005cb6bd03 mypy: Improve [get_]cache_with_key typing & use py3 annotation. 2018-03-16 11:29:12 -07:00
Tim Abbott 3d8e45f1cb cache: Avoid caching /help/ documentation page content.
This should make it a lot less annoying to edit these pages locally,
without regressing the test performance which motivated the cache.
2018-03-05 09:26:58 -08:00
Umair Khan 0eca2e102d cache: Add ignore_unhashable_lru_cache function.
This is a wrapper over lru_cache function. It adds following features on
top of lru_cache:

    * It will not cache result of functions with unhashable arguments.
    * It will clear cache whenever zerver.lib.cache.KEY_PREFIX changes.
2018-02-09 18:14:08 -08:00
rht 9161f8c39b zerver/lib: Remove u prefix from strings. 2018-02-05 12:12:58 -08:00
Vishnu Ks 036dc53d20 messages: Rename last_visible_message_id to first_visible_message_id. 2018-01-22 19:53:44 -08:00
Vishnu Ks e6d3f8895f messages: Create function to calculate first visible message id. 2018-01-04 08:52:40 -05:00
rht ee546a33a3 zerver/lib: Use python 3 syntax for typing.
Edited by tabbott to improve various line-wrapping decisions.
2017-11-28 17:15:14 -08:00
rht 229a8b38c0 zerver/lib: Use Python 3 syntax for typing for several files.
This adds a number of annotations that had been missed in previous
passes.
2017-11-28 17:02:24 -08:00
Tim Abbott 646ba5b9e5 bulk_get_users: Fix issues with users in multiple realms.
The previous implementation had a subtle caching bug: because it was
sharing its cache with the `get_user_profile_by_email` cache, if a
user happened to have an email in that cache, we'd return it, even
though that user didn't match `base_query`.

This causes `get_cross_realm_users` to no longer have a problematic
caching bug.
2017-11-27 14:34:45 -08:00
rht fef7d6ba09 zerver/lib: Remove u prefix from strings.
License: Apache-2.0
Signed-off-by: rht <rhtbot@protonmail.com>
2017-11-03 15:34:37 -07:00
neiljp (Neil Pilgrim) 304e411944 mypy: Remove unused FuncT TypeVar in cache.py. 2017-10-31 00:02:17 -07:00
Tim Abbott f2e3e779eb mypy: Properly annotate generic_bulk_cached_fetch.
Along with fixing some minor bugs, this requires extracting out the
default functions so that we can do type: ignores on them properly.

While we're at it, we switch to the Python 3 syntax.
2017-10-28 10:07:15 -07:00
Tim Abbott 73c27e1277 cache: Fix type aliasing of cached_objects.
Previously, it was converted from a CompressedItemT to an ItemT
without changing the variable name.
2017-10-28 10:01:44 -07:00
Tim Abbott 94c1da7025 cache: Move generic_bulk_cached_fetch typevars up a bit. 2017-10-28 10:00:43 -07:00
neiljp (Neil Pilgrim) c063ba72a2 mypy: Improve typing of cache_with_key and cache decorators.
Fixes #1348.
2017-10-28 08:57:49 -07:00
Steve Howell df93a99b50 Cache only one row per message.
Before this change, we populated two cache entries for each
message that we sent.  The entries were largely redundant,
with the only difference being whether we sent the content
as raw markdown or as the rendered HTML.

This commit makes it so we only have one cache entry per
message, and it includes both content and rendered_content.

One legacy source on confusion here is that `content`
changes meaning when you're on the front end.  Here is the
situation going forward:

    database:
        content = raw
        rendered_contented = rendered

    cache entry:
        content = raw
        rendered_contented = rendered

    payload for the frontend:
        content = raw (for apply_markdown=False)
        content = rendered (for apply_markdown=True)
2017-10-26 16:35:28 -07:00
Steve Howell 14d2d4e506 Fix bug in flush_user_profile().
Every time we updated a UserProfile object, we were calling
delete_display_recipient_cache(), which churns the cache and
does an extra database hop to find subscriptions.  This was
due to saying `updated_fields` instead of `update_fields`.

This made us prone to cache churn for fields like UserProfile.pointer
that are fairly volatile.

Now we use the helper function changed().  To prevent the
opposite problem, we use all the fields that could invalidate
the cache.
2017-10-25 11:30:56 -07:00
Steve Howell c8875693c8 Extract changed() helper in flush_user_profile().
The verbose style of `changed` is partly to appease mypy.
2017-10-25 11:29:09 -07:00
Steve Howell b94c062368 Make the realm user cache include non-active users.
This is a prepatory commit that adds non-active users to
the realm user cache.  It mostly involves name changes and
removing an `is_active` filter from the relevant DB query.

The only consumer of this cache is `get_raw_user_data`, which
now filters on `is_active` in a dictionary comprehension (but
this will get moved around a bit in a subsequent commit).
2017-10-25 11:18:30 -07:00
rht 035ed93111 zerver/lib: remove `import six`. 2017-09-27 19:10:28 -07:00
rht 2e12fe5e2e zerver/lib: Remove print_function. 2017-09-27 18:05:45 -07:00
rht f43e54d352 zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
Juliana Bacelar 928dd06cc8 linter: Add lint rule banning 'import os.path' 2017-09-22 10:32:21 -07:00