Commit Graph

102 Commits

Author SHA1 Message Date
Leo Franchi 49328c2551 Unify rabbitmq usage to use only one queue.
To work around the issue we're having with queue draining between
parallel blocking connections, use the same rabbitmq queue for both
activity and presence events, keyed on a 'type' flag in the message
itself.

(imported from commit 188e8fda1695734e52c5740db2195072cfc81479)
2013-02-15 11:43:17 -05:00
Leo Franchi d4a582fa1c Handle SIGINT and SIGTERM in process_user_activity by closing communication gracefully
(imported from commit dcf813fa8af1aaf5d6cc635268757e40c6c202ab)
2013-02-15 11:05:31 -05:00
Leo Franchi c2074813ca Generate unique consumer tags when connecting to the queues
(imported from commit cd9c78db46ae1a36b46bebf5cb0d226e15f71ba4)
2013-02-15 11:05:31 -05:00
Leo Franchi 5d82dc5ce2 Don't match ** for strong across newlines
(imported from commit e3bc0b409bd99c3707746c23eb73e11342cfe178)
2013-02-14 11:56:53 -05:00
Tim Abbott 2f4b62048e gather_subscriptions: Fix making O(streams) database queries.
This cuts the MIT Zephyr home page load time and also #subscriptions
database-sourced load time for me from about a second to about 50ms on
my laptop.  The home page still takes about 600ms to load due to
templating.

(imported from commit 1cfd8c750301abe6b6a881be4b2857532be947ec)
2013-02-12 16:25:45 -05:00
Leo Franchi 168f8623e4 Add https?: greedy url matching before falling back to our url guesser
(imported from commit 9e6e5a0522e6501b354a56223c2639841d290d4b)
2013-02-12 12:24:15 -05:00
Leo Franchi 7af226398e Only connect to RabbitMQ if it is enabled
(imported from commit 9726595bbb850c903177b5f64612eb4aa82e36ce)
2013-02-12 12:12:28 -05:00
Leo Franchi c348dd3587 Remove get_instance() from SimpleQueueClient
SimpleQueueClient was not thread-safe as the pika method
calls that we are using might disconnect and end up in a bad
state if two threads enqueue messages at the same time.

In order to avoid this, each user of the rabbitmq queue
gets its own instance of a SimpleQueueClient.

(imported from commit 694083b75cd58a60b8de282a8f40eb92a864c5ce)
2013-02-11 18:05:57 -05:00
Leo Franchi 0a0c4bb9a0 [manual] Use rabbitmq for asynchronous presence updating
Note: When deploying, restarting the process-user-activity-commandline script is needed

(imported from commit 63ee795c9c7a7db4a40170cff5636dc1dd0b46a8)
2013-02-11 18:05:57 -05:00
Leo Franchi 31f87481d0 [manual][schema] Add an API for user presence (idle) information
Adds a new db table for storing presences, and an API for setting
an individual user's idleness as well as fetching all idle status
for all users in a realm

(imported from commit 5aad3510d4c90c49470c130d6dfa80f0d36b0057)
2013-02-11 18:05:57 -05:00
Jessica McKellar 53f0b00785 Add a management command to change and log user emails.
(imported from commit 0bf25433a371f7e46ade96019f4a9ceebe395fe5)
2013-02-10 16:45:25 -05:00
Keegan McAllister 9edb616feb queue: Add logging
(imported from commit db2f15cf5f6a683f03d08a2e8b8ee4547537daf7)
2013-02-08 13:57:22 -05:00
Keegan McAllister 89c544f21f queue: Refactor SimpleQueueClient initialization
In anticipation of subclassing.

(imported from commit 1ebee932aed1b472bf9a8ab6d9bc30519f587c83)
2013-02-08 13:57:22 -05:00
Keegan McAllister d349de903b queue: Get rid of lazy initialization in SimpleQueue
Instead make it a singleton with a get_instance() class method.

(imported from commit e32cabf77b43361e74a11a23bba3a6d9fb32f82f)
2013-02-08 13:57:22 -05:00
Keegan McAllister b2bc7941dc queue: Use normal instance data in SimpleQueue
Code like this is dangerous:

    class SimpleQueue(object):
        queues = set()

because all instances will share the same 'queues' set object.

We don't really need multiple instances, but neither is there a reason to break
them.  So do the normal Python thing instead.

(imported from commit a56bb8414dd549cfd312c9565df198cf9d20f08a)
2013-02-08 13:57:22 -05:00
Keegan McAllister f14db52f22 queue: Rename SimpleQueue to SimpleQueueClient
It doesn't represent a single queue; that's what the 'queue_name' arguments are
for.

(imported from commit 8382ae6793ca7c3ae4b5aacf2c128113a7a88fb6)
2013-02-08 13:57:22 -05:00
Keegan McAllister 2617bbcdff queue: Remove shebang line
There's no reason to run this file as a script, and it's not marked as
executable.

(imported from commit 52bddaf2b59333c1f37b2cfc90b92a164c2e6d3c)
2013-02-08 13:57:22 -05:00
Jessica McKellar ad8e9598f6 Log domain in do_activate_user.
(imported from commit 0e39b5ddc395ff245f8e3b0252ea3b33a90860f1)
2013-02-08 13:20:19 -05:00
Jessica McKellar 50bab7af8f Log domain in user creation event.
(imported from commit d5aca898781b4f33a749536181e50f941bd16b59)
2013-02-08 13:20:19 -05:00
Leo Franchi c637b9cf44 Handle chars at end of link better
(imported from commit 53842bc17bbb9cfb555738ee6b72291d7ce61d67)
2013-02-06 11:51:52 -05:00
Waseem Daher 66850b5dcd Make gather_subscriptions pass along whether or not a stream is invite-only.
(imported from commit f71c171185a0552dba4917967d3feffb9c92c4fe)
2013-02-05 17:24:29 -05:00
Luke Faraone ac0dbd0864 Return in_home_view in gather_subscriptions.
This enables us to do client-side filtering on messages by the value of
this setting.

(imported from commit e69a7772dc90c67cb91ea8f630f5b78c902e8fa8)
2013-02-05 15:34:37 -05:00
Luke Faraone e8afaa8b8e Return a dictionary in subscriptions/list instead of a tuple.
This will give us flexibility in the future to add new properties to the
list.

In order to support that, we now do a list comprehension rather than just
returning the gather_subscriptions list in get_stream_colors.

(imported from commit a3c0f749a3320f647440f800105942434da08111)
2013-02-05 15:34:37 -05:00
Leo Franchi 054420fc38 Match .co.uk before .co in linkification
(imported from commit 2461cb4d49eef15431dde30dda646b25dc860a7b)
2013-02-05 13:04:45 -05:00
Jessica McKellar fdb0d15080 Don't let non-subscribers of invite-only streams query the membership.
(imported from commit 01bd8ea089dec96e487e5e82fb38df65703679ae)
2013-02-05 10:12:04 -05:00
Leo Franchi 3569e8e9b2 Sanitize links to prevent XSS, and handle emails
(imported from commit 622396efde50d9f5e3501f5d780c344ad0692662)
2013-02-01 18:42:27 -05:00
Leo Franchi cc58044234 Rework linkify regex to match more urls
(imported from commit 0e1a1df88363374ffbc802f83f43eb0fac8c99ea)
2013-02-01 16:14:29 -05:00
Keegan McAllister d348e0e9d6 bugdown: Disable italics and __foo__-style bold
(imported from commit c35d6980db6c59828514a90eec199a7494625495)
2013-01-31 16:28:57 -05:00
Leo Franchi 071ea3b9a4 Send humbug & email on markdown parse failure
(imported from commit c132ab91f1ff9ffdbe5f740980bf362b16c6bba6)
2013-01-31 14:48:40 -05:00
Keegan McAllister c5a69e5814 timeout: Try to kill the thread once the timeout expires
(imported from commit 25e19a3baa306fb440b27f492506ff231681bb4e)
2013-01-30 11:13:48 -05:00
Keegan McAllister e12f10ec1f bugdown: Spend at most 5 seconds rendering a message
(imported from commit bc092acc8b2b9f8a63af669de06c6f7512ccf8c9)
2013-01-30 11:13:48 -05:00
Leo Franchi 6130d271cf Make fence regex stricter to only match fence-like things
(imported from commit 59d5302b7af09c669423d364d46fb0d1b6a7b080)
2013-01-29 18:02:46 -05:00
Leo Franchi c6f73f7697 Refactor fence block code to never infinite loop
(imported from commit f72cb182e4fc9c4e8003853276d8aa40b454d08f)
2013-01-29 18:02:45 -05:00
Keegan McAllister f2659b7dcd bugdown: Remove dead code
The user of this function was removed in 97dd20ec.

(imported from commit c3823df65172f3799658e36d7ba183f661e9fe65)
2013-01-28 18:37:48 -05:00
Leo Franchi 7e567f527f Move gather_subscriptions to actions.py as it's not a view
(imported from commit cdf94ded51e441fe3515641458023be44c1ba40c)
2013-01-28 17:32:58 -05:00
Leo Franchi c39c6f643c Handle lists that start immediately after paragraphs
(imported from commit 055593d7a324598e133d53db0c33103016426c8f)
2013-01-25 17:13:13 -05:00
Leo Franchi 969d89d070 Don't accept - as valid list delimiters
(imported from commit 287353a29289ee536a59f47f87ff66893bf261ec)
2013-01-25 17:13:13 -05:00
Leo Franchi ba7dcc320c Remove ``` support from code fence
Supporting ``` as a code fence marker complicates the auto-fence
closing, and as per a discussion with Keegan on code-review@, it
is not worth the extra complexity.

(imported from commit 405afb95c4295a02f4677181456caf9d49913ac4)
2013-01-25 17:12:49 -05:00
Leo Franchi c2db0dd8bf Automatically add an end-of-code fence block if missing
(imported from commit 167f0217bbe6709e73dbc06f2894d9e22ee17a77)
2013-01-25 17:02:57 -05:00
Leo Franchi b2505bfc22 Add backend support for invite-only streams
Trying to add a user to an invite-only stream that already
exists will result in in error

(imported from commit 910750580a122cee92096d7e83457cb0b8cce616)
2013-01-24 17:34:21 -05:00
Tim Abbott 748927ad7c Fix transaction management in create_mit_user_if_needed.
(imported from commit 063eaf5180964ad32a6feb3c899a4cc5beea6fe9)
2013-01-24 10:58:51 -05:00
Reid Barton c9e2fed3f9 Preparatory changes to Tornado server for receiving a single stream's messages
Doesn't yet add the API for actually doing so.

(imported from commit a2062d71544441256a41f817388bb0c7ea6891ff)
2013-01-16 14:19:05 -05:00
Keegan McAllister b5a0147e26 Log events to a file named after today's date
We need this so that we can safely expunge old events without interfering with
the running server.  See #414.

(imported from commit 4739e59e36ea69f877c158c13ee752bf6a2dacfe)
2013-01-15 14:37:36 -05:00
Tim Abbott e592e71515 [manual] Use rabbitmq queue to process UserActivity.
Before this is deployed, we need to install rabbitmq and pika on the
target server (see the puppet part of this commit for how).

When this is deployed, we need to start the new user activity bot:

./manage.py process_user_activity

in the screen session on the relevant server, or user_activity logs
won't be processed (which will eventually result in all users getting
notifications about how their mirrors are out of date).

(imported from commit 44d605aca0290bef2c94fb99267e15e26b21673b)
2013-01-14 13:28:23 -05:00
Tim Abbott 6a2bf7c4f3 Move update_user_activity to actions.py.
(imported from commit a81669d01e699fbf809a02f91e6007b7f61e5518)
2013-01-14 13:28:23 -05:00
Luke Faraone 155cd1e821 Reintroduce updated internal_send_message to support sending personals.
internal_send_message now has the ability to send personals as well as
stream messages.

This change was accidentally lost during a rebase.

(imported from commit 153a3929c5c64be82288293c1f0cc02fcc03c08d)
2013-01-14 12:41:18 -05:00
Tim Abbott 47b2b1cb1c tornado: Fill the message memcached in a child process.
(imported from commit 3a7b4c0f4aad37319f4cfa7892709e99883f6a87)
2013-01-14 12:28:51 -05:00
Tim Abbott e36fd4b817 tornado: Send rendered messages to Tornado via memcached.
(imported from commit c1e8a017fa61b4e3f5c44ad4e5f59e4faf012ca3)
2013-01-11 16:11:07 -05:00
Tim Abbott a058471a56 tornado: Cache messages internally in tornado process.
This allows us to handle the return_messages_immediately part of
get_updates requests without having to talk to the database.

(imported from commit ed0b7742d359efb21a0a4960f4fc25f4337e9ad4)
2013-01-11 16:11:07 -05:00
Tim Abbott 1a82741650 Move action functions from models.py to zephyr/lib/actions.py.
(imported from commit 9d577dd53ce7d4c9faf6cc8a56129d684a50811b)
2013-01-11 16:11:03 -05:00