Commit Graph

50 Commits

Author SHA1 Message Date
Tim Abbott f06fdbb7a8 Fix Zephyr mirroring loop detection with new events API.
(imported from commit 79454bb26851b2f4a61d6cdf04350386671618a5)
2013-05-03 18:20:34 -04:00
Tim Abbott 5e22778843 Use a function for stopping/restarting time logging for longpolling.
(imported from commit 11b772deaa126fcc7e7605d467022b22d9e98cb0)
2013-04-24 10:44:56 -04:00
Luke Faraone 71a91197fa Enable absolute imports.
See PEP 328[1] for details. This feature was introduced in Python 2.5 and
will become mandatory in Python 3.

[1]: http://www.python.org/dev/peps/pep-0328

(imported from commit 7444eeba8a08d5f91b94c7921848f2274979bd76)
2013-04-23 09:51:17 -07:00
Zev Benjamin 0caec89fc6 Use helper functions instead of importing event_queue.client and event_queue.user_clients directly
(imported from commit 653decf415e0026439dd5d323bed361732dac2e9)
2013-03-28 16:57:48 -04:00
Zev Benjamin 5b20dcf03c Add server-side event filtering based on type
Clients can now request to receive only certain kinds of events,
although they always receive restart events.

(imported from commit 1e72981f8fe763829ab2abde1e35f94cad5c34e4)
2013-03-28 16:57:48 -04:00
Zev Benjamin 7248659c4c Return initial data in register()
(imported from commit 5b510ef072790b9e44652542372e3fc88735c356)
2013-03-28 16:57:48 -04:00
Zev Benjamin 22134946a4 Require clients call register() before get_events() unless they pass dont_block=True
(imported from commit f2aee3b7185ffd4642aeb23891fe681115fb0895)
2013-03-28 16:57:48 -04:00
Zev Benjamin 6cc70d94f6 Add register() call to event system
(imported from commit 0c9fbfec1866591b2169ce2da2bc2af6003f8f31)
2013-03-28 16:57:47 -04:00
Zev Benjamin 8f4eaa63ad Initial event system implementation
This version has several limitations that are addressed in later
commits in this series.

(imported from commit 5d452b312d4204935059c4d602af0b9a8be1a009)
2013-03-28 16:48:14 -04:00
Luke Faraone 09c9d92149 Refactor get_messages into the _backend pattern and add rest_ method.
(imported from commit 9b1ae464a882a6fa6da2a4cfd5a6543f5d2b3e51)
2013-03-28 07:57:36 -07:00
Luke Faraone 2be8f80f47 Make request._client a public member, dropping the '_'.
(imported from commit 5fc7ab57157ceb40c8bca9b0c97a47d8fac6745b)
2013-03-28 07:53:36 -07:00
Zev Benjamin d1b9ab9fd8 Move SERVER_GENERATION definition to settings.py
(imported from commit 4be2b912a63f49f525c66f46db0bc5c466ad364f)
2013-03-27 14:15:03 -04:00
Zev Benjamin 203b0fcc25 Use json_to_bool instead of simplejson.loads
(imported from commit 0d045c606925379450215349a17d6fe04d3f1be9)
2013-03-27 14:15:03 -04:00
Tim Abbott 144987e581 logs: Provide total time passed for longpolled requests.
Also improve display of times passed -- we now use display short times
in milliseconds for easier reading.

(imported from commit 08e1e7e6acbef48453080864946f7602a3395e7c)
2013-03-27 08:19:27 -04:00
Keegan McAllister 23b8833ec5 Handle Tornado callback notify in a more generic, less HTTP-centric way
(imported from commit 85a74d1b40461236c4c95ad688e9796ab50f0bbf)
2013-03-22 15:51:23 -04:00
Keegan McAllister 8db92fd995 Split out the actual Tornado views from the callback handling / notification
(imported from commit 26d40556bf7f545d3606a01d83b16c7265bafb0e)
2013-03-22 15:50:35 -04:00
Tim Abbott 722918aa61 Use humbug_finish() wrapper for finish() with required arguments.
I think this is a little cleaner than the previous approach.

(imported from commit fa6233721f43c94003de643d7801a88a156affd2)
2013-03-15 12:53:07 -04:00
Tim Abbott 549577a928 Pass the original request object to Tornado's finish().
This is preparatory for running the Django response middleware on
our Tornado responses.

(imported from commit 05da8ea9cb663a928b2f98a928f3992aae4f067c)
2013-03-15 12:53:06 -04:00
Zev Benjamin 22457277f4 Move send_with_safety_check functionality into AsyncDjangoHandler
This will mainly be useful in the event system branch, where we want
to actually send a response from a file other than tornadoviews.py.

(imported from commit b7ae9bb9b062215ab44eb5f0a3a72d6baeee1d07)
2013-03-14 18:48:15 -04:00
Tim Abbott cf58ee833a Rename lib/message_cache.py to lib/cache_helpers.py
(imported from commit 591e427aee47aea58b50d6c3fbd6a05cfe9378ea)
2013-03-14 15:07:41 -04:00
Tim Abbott 60c3cf826d restart-server: Fill memcached caches after clearing the cache.
Our previous code could in theory end up clearing the caches it had
just filled, if Tornado's cache filling work happened to be faster
than the memcached flush.

(imported from commit 48174aadad398fb7a7c917a1df765c1261b12a55)
2013-03-14 15:07:41 -04:00
Leo Franchi 4a23959b11 Remove pointer_updater param as it is no longer used
(imported from commit 80cc4a6d4c24660087981b1132ca2ed44c86a424)
2013-03-14 13:35:28 -04:00
Tim Abbott 99d4bc114b Move Zephyr mirroring liveness check to the UserPresence queries.
It's closer to a presence query than an update, and more importantly
this moves this out of Tornado -- previously Tornado was spending at
least 3ms per recipient on messages sent to the MIT realm fetching all
this data to return back to users.  This should save around 100ms per
message sent to a popular stream the MIT realm -- but more
importantly, each such event is 100ms during which Tornado is not
processing other messages.

(imported from commit 134169f0fdcd9f6640fda957edc4a28b07783d8e)
2013-03-13 17:41:29 -04:00
Tim Abbott c38092424e Clean up fetch_table_messages code a bit.
Fixes #792.  I'm really curious what (if anything) this change does to
our Tornado load numbers.

(imported from commit daf81d2dea81f36ec4a3eb19993169b58681e717)
2013-02-12 16:25:45 -05:00
Jessica McKellar f53e31e118 If a web client is requesting message that are too old, force a reload.
If get_updates requests a message ID that is too old, Tornado will
return a 400, and currently the only way to start caring about newer
message IDs is to reload.

(imported from commit 1638d71868475ffd793162afc7a2731cab14bd75)
2013-01-23 11:33:07 -05:00
Tim Abbott 13bbf20260 Restore original 25000 message cache size.
(imported from commit 5eb0da70a8514f75addcc042d1a3e3aa98711e25)
2013-01-17 16:20:18 -05:00
Tim Abbott 03ca42a988 tornado: On user_message/stream_message cache misses, query the database.
This should fix the issues we saw where users who hadn't received a
message in the cache would not be able to receive new messages.

(imported from commit 3834dd5bd5a10c7a603194b540d78afd8a966017)
2013-01-17 16:20:18 -05:00
Tim Abbott d4ad38882d Increase the usermessage cache to 50000 messages.
(imported from commit 3b937a45339c8ab68fe07901518df1b97885afd3)
2013-01-17 13:14:02 -05:00
Tim Abbott 0ab3b3947e Use a smaller cache for stream messages.
This is a hack to work around the fact that the query here takes far
longer than it should.

(imported from commit e6ee00c0636fe85cd946e4e3f2bbb2bda2da1f26)
2013-01-17 11:32:20 -05:00
Tim Abbott 9f93d9f46b tornado: Speed up Stream message cache initialization query.
The stream message cache query was grabbing an overly broad set of
columns and related models (e.g. the sender's user_profile), so that
Tornado would spend too long doing the query.

(imported from commit 6a692bc0e31957d4b9d9528d348c4b07a09eb15c)
2013-01-17 10:54:39 -05:00
Reid Barton ed29f22069 Implement single-stream get_old_messages/get_updates
(imported from commit 22a5c4985f36c38cf2490ecf26da294b502f8e46)
2013-01-16 14:19:05 -05:00
Reid Barton c9e2fed3f9 Preparatory changes to Tornado server for receiving a single stream's messages
Doesn't yet add the API for actually doing so.

(imported from commit a2062d71544441256a41f817388bb0c7ea6891ff)
2013-01-16 14:19:05 -05:00
Tim Abbott 47b2b1cb1c tornado: Fill the message memcached in a child process.
(imported from commit 3a7b4c0f4aad37319f4cfa7892709e99883f6a87)
2013-01-14 12:28:51 -05:00
Tim Abbott bea03548c2 tornado: Don't attempt to populate the memcached on restart.
Later, we can fork off a separate process to do that task, but for
now, we can just rely on the fact that unless the server was just
rebooted, the relevant messages will already be in that cache, and
even if it was, getting the needed messages once each is not a
particularly expensive query.

(imported from commit 6d08eba1b41237fd4e1204e181ce8f227573930d)
2013-01-14 12:28:48 -05:00
Tim Abbott 4df3c44a97 tornado: Fix order_by query during usermessage initialization.
(imported from commit e39556438ccb6487fe6be3e80d9c3b1e4cdf0cb7)
2013-01-11 16:19:31 -05:00
Tim Abbott 3636f22b2b tornado: Rename receive to receive_message.
(imported from commit 336273048c902941bb65755254bf253c0d130bd6)
2013-01-11 16:11:07 -05:00
Tim Abbott e36fd4b817 tornado: Send rendered messages to Tornado via memcached.
(imported from commit c1e8a017fa61b4e3f5c44ad4e5f59e4faf012ca3)
2013-01-11 16:11:07 -05:00
Tim Abbott a058471a56 tornado: Cache messages internally in tornado process.
This allows us to handle the return_messages_immediately part of
get_updates requests without having to talk to the database.

(imported from commit ed0b7742d359efb21a0a4960f4fc25f4337e9ad4)
2013-01-11 16:11:07 -05:00
Tim Abbott 053543e6eb tornado: Remove normal-runtime query in the user pointers code path.
We still need to do an initial query to get the pointers for all users.

(imported from commit 6ee991a648faf46539fd8676890dedb5f2003d05)
2013-01-11 16:11:07 -05:00
Tim Abbott f65ad3d59b Remove obsolete check for empty users string.
This code is from a previous protocol where the users array was posted
as a comma-separate list, rather than encoded via json.

(imported from commit 3a2edfcbf250a7eef305b2e98018c1361cc2fffe)
2013-01-10 16:59:36 -05:00
Tim Abbott 45b9de1e85 tornado: Remove UserProfile database queries from notify_new_message.
(imported from commit a23ce0cf7b8497e2ba7bc048d6fcf373f7ad913d)
2013-01-10 16:59:36 -05:00
Tim Abbott 8fe1edb6b0 tornado: Remove database queries from notify_pointer_update.
(imported from commit 218d5b804a4b4d9f7a1e9305165fd21820c26240)
2013-01-10 16:59:36 -05:00
Tim Abbott c3c59e938c Cache user pointers as they pass through Tornado.
(imported from commit 21a0dc0008ea5d3ac11388d3dd8d8cf8fa8a7e01)
2013-01-10 16:59:36 -05:00
Tim Abbott 62aed7c527 tornado: Move the add_*_callback functions a little earlier.
(imported from commit 2925e345fb9bd44c605350aac631c65a8f0601b6)
2013-01-10 16:59:36 -05:00
Tim Abbott 7274b24018 Callbacks: Reference users by user_profile_id, not user_id.
This is for consistency with the rest of our code dealing with message
delivery, which always uses the user_profile_id.

(imported from commit 5bf10bb9b994b0a98d3a22bd0bd86e542ab8a2ee)
2013-01-10 16:59:36 -05:00
Tim Abbott 21c2317bd2 Move callbacks library to tornadoviews.py.
(imported from commit a4e545d82133ae9dc34e283e3bf268225e35f400)
2013-01-10 16:59:36 -05:00
Tim Abbott b93db35e50 return_messages_immediately: Fix using last=None with dont_block.
Previously, if last was None, we wouldn't check dont_block,
server_generation, or any of the other reasons that get_updates might
return immediately, and just unconditionally entered longpolling mode.

In the process, this reorders return_messages_immediately to have
fewer cases and thus be easier to read.

(imported from commit 67803b8bfc7d9c9c1a4d6916eb2fb62664fb35a9)
2013-01-08 18:17:55 -05:00
Tim Abbott 40a3659844 return_messages_immediately: Remove check for clients with no messages.
This check was a workaround for the fact that the browser client
submitted a "last" value of -1.

(imported from commit a668f6a4e7a0c027f1214166a9bbf40d29b5daeb)
2013-01-08 18:17:55 -05:00
Tim Abbott 550bbe92ea get_updates: Move negative last checking to POST processing.
We shouldn't deploy this change until strictly after we deploy

"Fix website improperly submitting a last value of -1."

or we will break website clients.

(imported from commit 7f682ab0f7060b677f53f0a0073faef216f45d00)
2013-01-08 18:17:55 -05:00
Tim Abbott 83abc08315 Move tornado views to their own file.
(imported from commit e0d9182cbc89a16eedd2846fa17cfba482510565)
2013-01-08 16:53:53 -05:00