Commit Graph

23 Commits

Author SHA1 Message Date
Tim Abbott 7c001822f2 Use bulk requests for updating memcached in get_old_messages.
Otherwise we end up doing 1000 requests to memcached, which can be
quite expensive.

(imported from commit be247f63b5fb88c6f4a45326261b66ea67fe1028)
2013-04-25 14:43:37 -04:00
Tim Abbott 9b8f0fab0f Retrieve message objects from memcached in a bulk request.
On my laptop, this saves about 80 milliseconds per 1000 messages
requested via get_old_messages queries.  Since we only have one
memcached process and it does not run with special priority, this
might have significant impact on load during server restarts.

(imported from commit 06ad13f32f4a6d87a0664c96297ef9843f410ac5)
2013-04-24 10:44:56 -04:00
Tim Abbott 66b3c1fbff Log time spent querying memcached in logs when larger than 5ms.
(imported from commit a4de15026d24526a446b724500d1194dce824d1a)
2013-04-24 10:44:56 -04:00
Luke Faraone 71a91197fa Enable absolute imports.
See PEP 328[1] for details. This feature was introduced in Python 2.5 and
will become mandatory in Python 3.

[1]: http://www.python.org/dev/peps/pep-0328

(imported from commit 7444eeba8a08d5f91b94c7921848f2274979bd76)
2013-04-23 09:51:17 -07:00
Leo Franchi 652b821d64 Add a bunch of statsd logging
(imported from commit 8c17a76fed08c34abab3cc45d76a0fb30774d73a)
2013-04-18 18:05:52 -04:00
Leo Franchi 302cfcd48c Send client information for initial presence and process time differential
(imported from commit 99a51b7cc8b6c51c4e82757a984d07603b2980e3)
2013-04-12 09:11:40 -04:00
Tim Abbott 1cec86eb2d [manual] Remove now-unused User model.
I think all that one needs to do to deploy this commit is on developer
laptops, run `generate-fixtures --force`.

(imported from commit 34916341435fef0875b5a2c7f53c2f5606cd16cd)
2013-04-02 12:57:10 -04:00
Tim Abbott 1443edce00 Access the UserProfile's new email field rather than using User.
This is preparatory for stopping using the User model.

(imported from commit a1b0808c8cc2ddd19a25163f91c4f18620c9ce90)
2013-04-02 12:07:08 -04:00
Tim Abbott e2d010ef2d Fix most unnecessary database queries in huddle creation.
This saves 2 database queries per user in the huddle when sending the
first message to a particular huddle.

(imported from commit f71aa32df846fb4b82651a93ff9608087ffcaa5a)
2013-03-28 07:36:10 -04:00
Jessica McKellar 0c3382fabb Always give hashlib.sha1 and friends bytes.
This fixes an experienced bug where you couldn't subscribe to a stream
with non-ASCII characters (failing with a UnicodeEncodeError), as well
as many other potential bugs.

(imported from commit f084a4b4b597b85935655097a7b5a163811c4d71)
2013-03-23 00:10:10 -04:00
Tim Abbott b38a538f8c Rename userprofile_by_foo cache keys to user_profile_by_foo.
(imported from commit ef398abc48c9b81a3d339ffdce00bae274246d28)
2013-03-18 16:15:11 -04:00
Tim Abbott 9909c8dc19 send_message: Use a memcached for the UserProfile-by-id cache.
This cache should save 2 database queries whenever we send a private
message.  However, previously it was per-process (which meant it was
mostly useless) and also buggy (it never stored anything in the cache,
so that it was completely useless).  Switching this to our standard
memcached setup will address both problems.

(imported from commit 1d807f30704bccf28de33a80523488aedc58a9be)
2013-03-18 16:15:11 -04:00
Tim Abbott 9ae583b910 Use the User/UserProfile caches for Django requests too.
Previously we only used these caches for Tornado requests, because we
were not updating memcached when e.g. the user's pointer changed, and
so functions like update_pointer would not work correctly.

Now that we are updated memcached when the User and UserProfile
objects change, we can use these for all requests.

This saves 2 database queries on every Django request to the server.

(imported from commit aa5bffd885d14bde38b95e80a226bd5ab66f253d)
2013-03-15 18:09:34 -04:00
Tim Abbott 1e6ed16c23 Flush the User cache when saving User objects too.
(imported from commit e595209fe1190e2f3e0d55ad3b9bae0997b91d1a)
2013-03-15 18:09:34 -04:00
Tim Abbott b6f42cb1e1 Update UserProfile objects in memcached when we save them.
(imported from commit 59426e19eeeba131f363a4bc5f74f59b6dbe1066)
2013-03-15 18:09:33 -04:00
Tim Abbott 7e456ddba7 Cache get_status_list output for up to a minute.
This should substantially decrease the amount of server load generated
by the userpresence system.

I tested that this indeed was indeed saving one query on
/json/update_active_status requests on my laptop with 2 users from the
humbughq.com realm logged in.

(imported from commit 03e9d4eb95b9f664d489862684ae162db2076e08)
2013-03-15 16:37:41 -04:00
Tim Abbott 0a9a553b5f Fix use of invalid hash keys with memcached.
Email addresses can contain characters that are not valid for
memcached cache keys.

(imported from commit d7cfd6d5292e4fc8f4e2b19f2c01a4097d878f1b)
2013-03-14 15:07:41 -04:00
Tim Abbott c098520bbd Move the key functions for various caches to cache.py.
(imported from commit b04826533c32516cc2eef3b35263a40385ae7be4)
2013-03-14 15:07:41 -04:00
Tim Abbott 1580386946 Replace db_cache_with_key with a parameter on cache_with_key.
(imported from commit f2c600292888ba384ee4acc97c94f1d6f8bc9657)
2013-03-14 15:07:41 -04:00
Tim Abbott d679a72952 [manual] Cache results of the Twitter API in the database.
This should substantially improve the repeat-rendering time for pages
with large numbers of tweets since we don't need to go all the way to
twitter.com, which can take like a second, to render tweets properly.

To deploy this commit properly, one needs to run

./manage.py createcachetable third_party_api_results

(imported from commit 01b528e61f9dde2ee718bdec0490088907b6017e)
2013-03-11 13:15:55 -04:00
Tim Abbott a058471a56 tornado: Cache messages internally in tornado process.
This allows us to handle the return_messages_immediately part of
get_updates requests without having to talk to the database.

(imported from commit ed0b7742d359efb21a0a4960f4fc25f4337e9ad4)
2013-01-11 16:11:07 -05:00
Zev Benjamin 7bbde14d78 Use functools.wraps on the functions returned by our decorators
This lets Django report the correct view name in errors

(imported from commit b21347e7af39cda439125355f99f4fc63fc3bd2f)
2012-11-08 16:30:57 -05:00
Keegan McAllister 669bb789f8 Add caching decorators
(imported from commit 7a4d9257ea8c6a86a5ffb498f726c4c2eb42bc9f)
2012-09-19 13:01:58 -04:00