zulip/zerver
Steve Howell 53b5768b14 Double speed of message fetching in get_old_messages_backend().
When we don't already have old messages in cache, we need to
fetch data from the database and create dictionaries for the
cache.  This commit makes that process work in 50ms, instead
of 130ms, for the data set in test_bulk_message_fetching(),
which is 602 records.  Before this commit we had about 132
microseconds of unnecessary churn per message, because we
were fetching DB fields we didn't need and incurring the cost
of the Django ORM.  Now we use values() to get only the columns
we need, and we take advantage of previous commits that make
our code less OO and more function-driven, so we can pass the
values directly to build_message_dict() without having to create
objects.

A couple caveats on this commit:

1) I haven't been able to get good measurements on the overall
effect on get_old_messages_backend().  If you kill the cache to
force DB queries, you introduce noise related to sessions and
user profiles.

2) Look at the long comment in this commit related to
re-rendering messages in this codepath.  The problem precedes
this commit.

(imported from commit dcb64aa9416f0e9583355ddd6dc3adfa746b9fc7)
2013-09-30 18:22:07 -04:00
..
fixtures github: Put pull request comments on the 'commits' stream 2013-09-19 23:44:21 -04:00
lib Extract get_avatar_url(). 2013-09-30 18:22:06 -04:00
management email mirror: ignore decoding errors. 2013-09-27 11:34:57 -04:00
migrations [schema] Add a date_created field to zerver_stream. 2013-09-27 16:23:56 -04:00
templatetags [manual] Rename Django app from zephyr to zerver. 2013-08-06 07:39:36 -04:00
tests/frontend [important] Fix stacktrace in unread.js. 2013-09-29 22:42:33 -04:00
views Double speed of message fetching in get_old_messages_backend(). 2013-09-30 18:22:07 -04:00
worker Log slow queries to zulip so we notice them 2013-09-30 17:41:56 -04:00
__init__.py [manual] Rename Django app from zephyr to zerver. 2013-08-06 07:39:36 -04:00
context_processors.py [manual] Rename Django app from zephyr to zerver. 2013-08-06 07:39:36 -04:00
decorator.py Split out presence and activity interval events from user_activity queue 2013-09-24 20:44:15 -04:00
exceptions.py [manual] Rename Django app from zephyr to zerver. 2013-08-06 07:39:36 -04:00
filters.py Change Humbug => Zulip in name of exception filter module. 2013-08-07 10:00:07 -04:00
finders.py Change Humbug => Zulip in name of finder module. 2013-08-07 10:00:07 -04:00
forms.py Allow MIT to invite coworkers. 2013-09-19 17:14:41 -04:00
handlers.py Change Humbug => Zulip in names of error reporting handlers. 2013-08-07 10:00:07 -04:00
middleware.py Log slow queries to zulip so we notice them 2013-09-30 17:41:56 -04:00
models.py Double speed of message fetching in get_old_messages_backend(). 2013-09-30 18:22:07 -04:00
openid.py [manual] Rename Django app from zephyr to zerver. 2013-08-06 07:39:36 -04:00
retention_policy.py [manual] Rename Django app from zephyr to zerver. 2013-08-06 07:39:36 -04:00
static_header.txt Change Humbug => Zulip in text/comments. 2013-08-07 10:00:07 -04:00
storage.py Change Humbug => Zulip in name of storage module. 2013-08-07 10:00:07 -04:00
tests.py Double speed of message fetching in get_old_messages_backend(). 2013-09-30 18:22:07 -04:00
tornado_callbacks.py Don't cache UserPresence info. 2013-09-17 14:51:56 -04:00
tornadoviews.py Let clients specify how long queues shall live, within limits 2013-08-20 16:19:07 -04:00