Commit Graph

15 Commits

Author SHA1 Message Date
Leo Franchi f64bc59dfb Automatically register consumer once asynchronous connection is completed
(imported from commit 3d860a9b79a42beffbd3b73f49aa2c25986dd3c6)
2013-03-22 15:55:49 -04:00
Leo Franchi 452105e393 Reconnect to rabbitmq in our queue client if the connection is lost
(imported from commit 82ed67f0c0f93e9f79ff65ef55b397815d20aaeb)
2013-03-22 15:55:48 -04:00
Keegan McAllister 4baeaaa532 TornadoQueueClient: Don't stop the Tornado I/O loop on RabbitMQ disconnect
Requires Pika 0.9.6 or later.

(imported from commit 316ab9f2a6b5c2a25ae8584e2604222b33e0dec3)
2013-03-22 15:55:48 -04:00
Keegan McAllister 6ffc2bbe65 TornadoQueueClient: Provide a callback when declaring queues
This is required by Pika 0.9.8.  We need at least 0.9.6 for the next
commit; I had been testing with 0.9.5 previously.  Anyway this way
seems more correct as well.

(imported from commit bfb9e9e78938073001f70c4d28a5e07cc4ebac32)
2013-03-22 15:55:48 -04:00
Keegan McAllister 373d3ece1c queue: Implement TornadoQueueClient
(imported from commit 2803bc91e42aa20d6e6dec87bfadd2f890268d05)
2013-03-22 15:39:07 -04:00
Leo Franchi d4a582fa1c Handle SIGINT and SIGTERM in process_user_activity by closing communication gracefully
(imported from commit dcf813fa8af1aaf5d6cc635268757e40c6c202ab)
2013-02-15 11:05:31 -05:00
Leo Franchi c2074813ca Generate unique consumer tags when connecting to the queues
(imported from commit cd9c78db46ae1a36b46bebf5cb0d226e15f71ba4)
2013-02-15 11:05:31 -05:00
Leo Franchi c348dd3587 Remove get_instance() from SimpleQueueClient
SimpleQueueClient was not thread-safe as the pika method
calls that we are using might disconnect and end up in a bad
state if two threads enqueue messages at the same time.

In order to avoid this, each user of the rabbitmq queue
gets its own instance of a SimpleQueueClient.

(imported from commit 694083b75cd58a60b8de282a8f40eb92a864c5ce)
2013-02-11 18:05:57 -05:00
Keegan McAllister 9edb616feb queue: Add logging
(imported from commit db2f15cf5f6a683f03d08a2e8b8ee4547537daf7)
2013-02-08 13:57:22 -05:00
Keegan McAllister 89c544f21f queue: Refactor SimpleQueueClient initialization
In anticipation of subclassing.

(imported from commit 1ebee932aed1b472bf9a8ab6d9bc30519f587c83)
2013-02-08 13:57:22 -05:00
Keegan McAllister d349de903b queue: Get rid of lazy initialization in SimpleQueue
Instead make it a singleton with a get_instance() class method.

(imported from commit e32cabf77b43361e74a11a23bba3a6d9fb32f82f)
2013-02-08 13:57:22 -05:00
Keegan McAllister b2bc7941dc queue: Use normal instance data in SimpleQueue
Code like this is dangerous:

    class SimpleQueue(object):
        queues = set()

because all instances will share the same 'queues' set object.

We don't really need multiple instances, but neither is there a reason to break
them.  So do the normal Python thing instead.

(imported from commit a56bb8414dd549cfd312c9565df198cf9d20f08a)
2013-02-08 13:57:22 -05:00
Keegan McAllister f14db52f22 queue: Rename SimpleQueue to SimpleQueueClient
It doesn't represent a single queue; that's what the 'queue_name' arguments are
for.

(imported from commit 8382ae6793ca7c3ae4b5aacf2c128113a7a88fb6)
2013-02-08 13:57:22 -05:00
Keegan McAllister 2617bbcdff queue: Remove shebang line
There's no reason to run this file as a script, and it's not marked as
executable.

(imported from commit 52bddaf2b59333c1f37b2cfc90b92a164c2e6d3c)
2013-02-08 13:57:22 -05:00
Tim Abbott e592e71515 [manual] Use rabbitmq queue to process UserActivity.
Before this is deployed, we need to install rabbitmq and pika on the
target server (see the puppet part of this commit for how).

When this is deployed, we need to start the new user activity bot:

./manage.py process_user_activity

in the screen session on the relevant server, or user_activity logs
won't be processed (which will eventually result in all users getting
notifications about how their mirrors are out of date).

(imported from commit 44d605aca0290bef2c94fb99267e15e26b21673b)
2013-01-14 13:28:23 -05:00