d4a582fa1c
Handle SIGINT and SIGTERM in process_user_activity by closing communication gracefully
...
(imported from commit dcf813fa8af1aaf5d6cc635268757e40c6c202ab)
2013-02-15 11:05:31 -05:00
c2074813ca
Generate unique consumer tags when connecting to the queues
...
(imported from commit cd9c78db46ae1a36b46bebf5cb0d226e15f71ba4)
2013-02-15 11:05:31 -05:00
5d82dc5ce2
Don't match ** for strong across newlines
...
(imported from commit e3bc0b409bd99c3707746c23eb73e11342cfe178)
2013-02-14 11:56:53 -05:00
2f4b62048e
gather_subscriptions: Fix making O(streams) database queries.
...
This cuts the MIT Zephyr home page load time and also #subscriptions
database-sourced load time for me from about a second to about 50ms on
my laptop. The home page still takes about 600ms to load due to
templating.
(imported from commit 1cfd8c750301abe6b6a881be4b2857532be947ec)
2013-02-12 16:25:45 -05:00
168f8623e4
Add https?: greedy url matching before falling back to our url guesser
...
(imported from commit 9e6e5a0522e6501b354a56223c2639841d290d4b)
2013-02-12 12:24:15 -05:00
7af226398e
Only connect to RabbitMQ if it is enabled
...
(imported from commit 9726595bbb850c903177b5f64612eb4aa82e36ce)
2013-02-12 12:12:28 -05:00
c348dd3587
Remove get_instance() from SimpleQueueClient
...
SimpleQueueClient was not thread-safe as the pika method
calls that we are using might disconnect and end up in a bad
state if two threads enqueue messages at the same time.
In order to avoid this, each user of the rabbitmq queue
gets its own instance of a SimpleQueueClient.
(imported from commit 694083b75cd58a60b8de282a8f40eb92a864c5ce)
2013-02-11 18:05:57 -05:00
0a0c4bb9a0
[manual] Use rabbitmq for asynchronous presence updating
...
Note: When deploying, restarting the process-user-activity-commandline script is needed
(imported from commit 63ee795c9c7a7db4a40170cff5636dc1dd0b46a8)
2013-02-11 18:05:57 -05:00
31f87481d0
[manual][schema] Add an API for user presence (idle) information
...
Adds a new db table for storing presences, and an API for setting
an individual user's idleness as well as fetching all idle status
for all users in a realm
(imported from commit 5aad3510d4c90c49470c130d6dfa80f0d36b0057)
2013-02-11 18:05:57 -05:00
53f0b00785
Add a management command to change and log user emails.
...
(imported from commit 0bf25433a371f7e46ade96019f4a9ceebe395fe5)
2013-02-10 16:45:25 -05:00
9edb616feb
queue: Add logging
...
(imported from commit db2f15cf5f6a683f03d08a2e8b8ee4547537daf7)
2013-02-08 13:57:22 -05:00
89c544f21f
queue: Refactor SimpleQueueClient initialization
...
In anticipation of subclassing.
(imported from commit 1ebee932aed1b472bf9a8ab6d9bc30519f587c83)
2013-02-08 13:57:22 -05:00
d349de903b
queue: Get rid of lazy initialization in SimpleQueue
...
Instead make it a singleton with a get_instance() class method.
(imported from commit e32cabf77b43361e74a11a23bba3a6d9fb32f82f)
2013-02-08 13:57:22 -05:00
b2bc7941dc
queue: Use normal instance data in SimpleQueue
...
Code like this is dangerous:
class SimpleQueue(object):
queues = set()
because all instances will share the same 'queues' set object.
We don't really need multiple instances, but neither is there a reason to break
them. So do the normal Python thing instead.
(imported from commit a56bb8414dd549cfd312c9565df198cf9d20f08a)
2013-02-08 13:57:22 -05:00
f14db52f22
queue: Rename SimpleQueue to SimpleQueueClient
...
It doesn't represent a single queue; that's what the 'queue_name' arguments are
for.
(imported from commit 8382ae6793ca7c3ae4b5aacf2c128113a7a88fb6)
2013-02-08 13:57:22 -05:00
2617bbcdff
queue: Remove shebang line
...
There's no reason to run this file as a script, and it's not marked as
executable.
(imported from commit 52bddaf2b59333c1f37b2cfc90b92a164c2e6d3c)
2013-02-08 13:57:22 -05:00
ad8e9598f6
Log domain in do_activate_user.
...
(imported from commit 0e39b5ddc395ff245f8e3b0252ea3b33a90860f1)
2013-02-08 13:20:19 -05:00
50bab7af8f
Log domain in user creation event.
...
(imported from commit d5aca898781b4f33a749536181e50f941bd16b59)
2013-02-08 13:20:19 -05:00
c637b9cf44
Handle chars at end of link better
...
(imported from commit 53842bc17bbb9cfb555738ee6b72291d7ce61d67)
2013-02-06 11:51:52 -05:00
66850b5dcd
Make gather_subscriptions pass along whether or not a stream is invite-only.
...
(imported from commit f71c171185a0552dba4917967d3feffb9c92c4fe)
2013-02-05 17:24:29 -05:00
ac0dbd0864
Return in_home_view in gather_subscriptions.
...
This enables us to do client-side filtering on messages by the value of
this setting.
(imported from commit e69a7772dc90c67cb91ea8f630f5b78c902e8fa8)
2013-02-05 15:34:37 -05:00
e8afaa8b8e
Return a dictionary in subscriptions/list instead of a tuple.
...
This will give us flexibility in the future to add new properties to the
list.
In order to support that, we now do a list comprehension rather than just
returning the gather_subscriptions list in get_stream_colors.
(imported from commit a3c0f749a3320f647440f800105942434da08111)
2013-02-05 15:34:37 -05:00
054420fc38
Match .co.uk before .co in linkification
...
(imported from commit 2461cb4d49eef15431dde30dda646b25dc860a7b)
2013-02-05 13:04:45 -05:00
fdb0d15080
Don't let non-subscribers of invite-only streams query the membership.
...
(imported from commit 01bd8ea089dec96e487e5e82fb38df65703679ae)
2013-02-05 10:12:04 -05:00
3569e8e9b2
Sanitize links to prevent XSS, and handle emails
...
(imported from commit 622396efde50d9f5e3501f5d780c344ad0692662)
2013-02-01 18:42:27 -05:00
cc58044234
Rework linkify regex to match more urls
...
(imported from commit 0e1a1df88363374ffbc802f83f43eb0fac8c99ea)
2013-02-01 16:14:29 -05:00
d348e0e9d6
bugdown: Disable italics and __foo__-style bold
...
(imported from commit c35d6980db6c59828514a90eec199a7494625495)
2013-01-31 16:28:57 -05:00
071ea3b9a4
Send humbug & email on markdown parse failure
...
(imported from commit c132ab91f1ff9ffdbe5f740980bf362b16c6bba6)
2013-01-31 14:48:40 -05:00
c5a69e5814
timeout: Try to kill the thread once the timeout expires
...
(imported from commit 25e19a3baa306fb440b27f492506ff231681bb4e)
2013-01-30 11:13:48 -05:00
e12f10ec1f
bugdown: Spend at most 5 seconds rendering a message
...
(imported from commit bc092acc8b2b9f8a63af669de06c6f7512ccf8c9)
2013-01-30 11:13:48 -05:00
6130d271cf
Make fence regex stricter to only match fence-like things
...
(imported from commit 59d5302b7af09c669423d364d46fb0d1b6a7b080)
2013-01-29 18:02:46 -05:00
c6f73f7697
Refactor fence block code to never infinite loop
...
(imported from commit f72cb182e4fc9c4e8003853276d8aa40b454d08f)
2013-01-29 18:02:45 -05:00
f2659b7dcd
bugdown: Remove dead code
...
The user of this function was removed in 97dd20ec.
(imported from commit c3823df65172f3799658e36d7ba183f661e9fe65)
2013-01-28 18:37:48 -05:00
7e567f527f
Move gather_subscriptions to actions.py as it's not a view
...
(imported from commit cdf94ded51e441fe3515641458023be44c1ba40c)
2013-01-28 17:32:58 -05:00
c39c6f643c
Handle lists that start immediately after paragraphs
...
(imported from commit 055593d7a324598e133d53db0c33103016426c8f)
2013-01-25 17:13:13 -05:00
969d89d070
Don't accept - as valid list delimiters
...
(imported from commit 287353a29289ee536a59f47f87ff66893bf261ec)
2013-01-25 17:13:13 -05:00
ba7dcc320c
Remove ``` support from code fence
...
Supporting ``` as a code fence marker complicates the auto-fence
closing, and as per a discussion with Keegan on code-review@, it
is not worth the extra complexity.
(imported from commit 405afb95c4295a02f4677181456caf9d49913ac4)
2013-01-25 17:12:49 -05:00
c2db0dd8bf
Automatically add an end-of-code fence block if missing
...
(imported from commit 167f0217bbe6709e73dbc06f2894d9e22ee17a77)
2013-01-25 17:02:57 -05:00
b2505bfc22
Add backend support for invite-only streams
...
Trying to add a user to an invite-only stream that already
exists will result in in error
(imported from commit 910750580a122cee92096d7e83457cb0b8cce616)
2013-01-24 17:34:21 -05:00
748927ad7c
Fix transaction management in create_mit_user_if_needed.
...
(imported from commit 063eaf5180964ad32a6feb3c899a4cc5beea6fe9)
2013-01-24 10:58:51 -05:00
c9e2fed3f9
Preparatory changes to Tornado server for receiving a single stream's messages
...
Doesn't yet add the API for actually doing so.
(imported from commit a2062d71544441256a41f817388bb0c7ea6891ff)
2013-01-16 14:19:05 -05:00
b5a0147e26
Log events to a file named after today's date
...
We need this so that we can safely expunge old events without interfering with
the running server. See #414 .
(imported from commit 4739e59e36ea69f877c158c13ee752bf6a2dacfe)
2013-01-15 14:37:36 -05:00
e592e71515
[manual] Use rabbitmq queue to process UserActivity.
...
Before this is deployed, we need to install rabbitmq and pika on the
target server (see the puppet part of this commit for how).
When this is deployed, we need to start the new user activity bot:
./manage.py process_user_activity
in the screen session on the relevant server, or user_activity logs
won't be processed (which will eventually result in all users getting
notifications about how their mirrors are out of date).
(imported from commit 44d605aca0290bef2c94fb99267e15e26b21673b)
2013-01-14 13:28:23 -05:00
6a2bf7c4f3
Move update_user_activity to actions.py.
...
(imported from commit a81669d01e699fbf809a02f91e6007b7f61e5518)
2013-01-14 13:28:23 -05:00
155cd1e821
Reintroduce updated internal_send_message to support sending personals.
...
internal_send_message now has the ability to send personals as well as
stream messages.
This change was accidentally lost during a rebase.
(imported from commit 153a3929c5c64be82288293c1f0cc02fcc03c08d)
2013-01-14 12:41:18 -05:00
47b2b1cb1c
tornado: Fill the message memcached in a child process.
...
(imported from commit 3a7b4c0f4aad37319f4cfa7892709e99883f6a87)
2013-01-14 12:28:51 -05:00
e36fd4b817
tornado: Send rendered messages to Tornado via memcached.
...
(imported from commit c1e8a017fa61b4e3f5c44ad4e5f59e4faf012ca3)
2013-01-11 16:11:07 -05:00
a058471a56
tornado: Cache messages internally in tornado process.
...
This allows us to handle the return_messages_immediately part of
get_updates requests without having to talk to the database.
(imported from commit ed0b7742d359efb21a0a4960f4fc25f4337e9ad4)
2013-01-11 16:11:07 -05:00
1a82741650
Move action functions from models.py to zephyr/lib/actions.py.
...
(imported from commit 9d577dd53ce7d4c9faf6cc8a56129d684a50811b)
2013-01-11 16:11:03 -05:00
761b7cd5fa
Rename zephyr.lib.time to zephyr.lib.timestamp.
...
Otherwise one gets:
AttributeError: 'module' object has no attribute 'time'
when trying to use the time module from inside zephyr.lib.
(imported from commit 645368672a3eff68320278dd480edeed56721fcc)
2013-01-11 15:58:19 -05:00
3c10a2e6a0
Move bulk_create_* methods to bulk_create.py.
...
(imported from commit 30d5f86209f9c8807516f0440276fafeb6590c1a)
2013-01-10 16:59:36 -05:00
7cebbd4a52
Add datetime_to_timestamp library function.
...
(imported from commit 28466ac5c8fc406ec70a61cae5975a784eb0f238)
2012-12-12 15:45:37 -05:00
02d65664b4
Increase MySQL batch size to 10,000.
...
This saves 30 seconds in populate_db runtime on MySQL.
(imported from commit 7fe483bf5f32cfa3d09db8ad7a9be79bd0a2a271)
2012-12-11 17:17:38 -05:00
b283123382
bugdown: Remove special treatment of backslash
...
Fixes #562 .
(imported from commit aa39cf390ef44275c7d5a84ca954c75535d372b1)
2012-12-11 16:54:10 -05:00
970969fc44
Add new timestamp_to_datetime helper function.
...
(imported from commit 6791d009ae2e8371abe2c929e87c816a1981f5fe)
2012-12-05 15:34:19 -05:00
6d9e0095eb
bugdown: Disable reference-based links
...
This is syntax like
Here's [a link][]
[a link]: http://google.com
This is not very useful for short chat-style messages. It will confuse users,
especially because we don't document it. And disabling it saves the effort of
applying the same link fixups as elsewhere.
(imported from commit c23391465486db545302b79c084b4f9cd5cdcc6a)
2012-12-05 13:54:43 -05:00
fb77bdf359
bugdown: Set link target, title attributes on the server
...
For consistency. Fixes #266 .
(imported from commit 63e199a0fe4534df804a82cc98a1fdcf8ccb45da)
2012-12-05 13:54:43 -05:00
80d07f597b
bugdown: Ignore user-specified link title
...
We already override this client-side.
(imported from commit c9569d42d57f206e5fb7d0cc8569b3b578ac1467)
2012-12-05 13:54:43 -05:00
3829c8fd7f
bugdown: Assume http:// for links without a protocol
...
It's unlikely that users want relative links within humbughq.com.
Fixes #447 .
(imported from commit d43a5758e6df448b07f56dc2de28078adaab8aeb)
2012-12-05 13:54:43 -05:00
d8b6cfdc11
bugdown: Fix confusing variable use in LinkPattern
...
(imported from commit 1538911149263340a5ea210c6d804a937f07cd5e)
2012-12-05 13:54:43 -05:00
b577bd54cd
bugdown: Remove code path to bypass sanitize_url
...
(imported from commit 984b87027cb56fe75c2dd9dcf505dde5bc624f8f)
2012-12-05 13:54:43 -05:00
8b3cfc0f1b
bugdown: Use a local copy of markdown.inlinepatterns.LinkPattern
...
With changes for the way things are imported in bugdown.
(imported from commit 11adf7911f7bb945367221f8fa317caa71de3fc4)
2012-12-05 13:54:42 -05:00
ff040faf97
bugdown: Use Markdown.reset() instead of re-creating the whole parser
...
(imported from commit 45a65453f2178a6a73392e2bd1e7d6d03de0e0e7)
2012-11-20 15:10:15 -05:00
fc60543432
bugdown: Remove trailing punctuation from automatic links
...
And allow parentheses etc. within a link, if they're not at the end.
Fixes #401 .
(imported from commit 5261fce74fe381ceece2e7406776cc5bde19deb9)
2012-11-20 15:10:15 -05:00
ba40bd390b
bugdown: Use our local copy of fenced_code
...
And wire it up to our local copy of codehilite. This fixes highlighting in
fenced code blocks, e.g.
~~~~ .js
var x = function () {
return "hi";
};
~~~~
(imported from commit 0efb0c9b98a3acdf55e18bb1918af7960f3425be)
2012-11-19 13:04:40 -05:00
7d08c405f3
Import GitHub version of Markdown fenced_code extension
...
(imported from commit 929de2ba09a5c6dabed20c7f1b3ba319ba42b244)
2012-11-19 13:04:40 -05:00
45cb7c1d37
bugdown: Don't guess code highlighting language
...
Disables highlighting unless a language is specified.
(imported from commit fe5d1a4a8042241336ee7ac01682553f6b35e956)
2012-11-19 13:04:40 -05:00
1ea92c0953
Manage file locking using the 'with' statement
...
This is clearer and more exception-safe.
(imported from commit b67641b05da9dbf8e5a887f398bac81ab5985cf3)
2012-11-16 13:43:44 -05:00
b278db110f
Move json response functions into their own file
...
(imported from commit 91a786849bfa30dcacecef6b8339d8f1a9365156)
2012-11-08 16:30:57 -05:00
7bbde14d78
Use functools.wraps on the functions returned by our decorators
...
This lets Django report the correct view name in errors
(imported from commit b21347e7af39cda439125355f99f4fc63fc3bd2f)
2012-11-08 16:30:57 -05:00
95c39cbc76
bulk_create: Use a (large) batch_size with MySQL.
...
(imported from commit 529fef7dd55bb8a5f4e286f7c896f4b14b0d1b8d)
2012-11-07 15:49:45 -05:00
b87a35c1ff
bugdown: Disable + as a bullet character for an unordered list
...
Fixes #272 .
(imported from commit 8afaf14965ed1f6a4bb3ccfc9d4c2d807148666d)
2012-11-02 14:04:09 -04:00
7a6fe207c5
bugdown: Disable ordered lists
...
These get automatically re-numbered, which will do the wrong thing when people
split their lists across multiple messages.
Fixes #241 .
(imported from commit 7f6f2c36a6ab27cef0a34008f304fc0fe25c8bd0)
2012-11-02 14:04:09 -04:00
1d4f339eae
bugdown: Hide alphanumeric characters in exception logs, for privacy
...
(imported from commit 39481494b7910307f56e566035c1b464c83d196e)
2012-10-25 15:40:01 -04:00
eef027560a
Remove unused imports
...
(imported from commit eb576627ff72e57fee0e3a4c357f51ad74cd6c86)
2012-10-25 15:22:18 -04:00
26c97729d4
bugdown: Generate HTML, not XHTML
...
Our pages are declared as HTML5:
<!DOCTYPE html>
The markdown library only supports HTML4, but that's probably closer than XHTML.
(imported from commit c78be9ae9bccf029def8d94d3647b0ccce8b2252)
2012-10-24 16:02:15 -04:00
4ca720a0cd
bugdown: Install autolink pattern after link pattern
...
This fixes explicit links.
(imported from commit 6867d271344d35c2c8d54b3393219113a095bc4f)
2012-10-22 17:48:48 -04:00
05c08b41c4
bugdown: Disallow ()[]{}<> in linkified URLs.
...
This allows us to handle messages like
my website (available at http://google.com )
(imported from commit 51330507947fc039b6f29f06dfa1c6d21f779aa4)
2012-10-22 17:32:39 -04:00
1c4bbd6a49
bugdown: Start a URL with a word boundary, not a space
...
This fixes the bug where we were eating the preceding space.
(imported from commit 9eca01e6ad828ccc31ad10693339f381333e76a7)
2012-10-22 17:32:39 -04:00
189d42428a
bugdown: Log Markdown parser failures
...
(imported from commit 9e225a32b71edbfd9007cc2fbae32be31896233d)
2012-10-21 23:07:03 -04:00
4a0cf8e608
bugdown: Disable builtin 'autolink' feature
...
This only linkifies inside angle brackets, per
print md.inlinePatterns['autolink'].getCompiledRegExp().pattern
We have our own linkification extension.
(imported from commit 20cab11aaafee075e0caf933d8d197717976988c)
2012-10-21 21:03:18 -04:00
00a4da5f9b
bugdown: Disable automatic mailto links for <foo@bar>
...
This feature was throwing exceptions on input like
<@`x`>
Reported upstream as https://github.com/waylan/Python-Markdown/issues/155
(imported from commit 6b14a65599def0346f76cf017be66dd80f42db49)
2012-10-21 21:03:18 -04:00
4db8587805
bugdown: Decrease code duplication
...
(imported from commit 40158134b60e477d8a353d050fa62c9ded0e9e9f)
2012-10-21 21:03:18 -04:00
649cd15415
bugdown: Linkify using a Markdown extension
...
This prevents trying to linkify inside code blocks.
(imported from commit 97dd20ecee19f41650aa98e68aa8e9908ece5b33)
2012-10-21 21:03:15 -04:00
d72aea25f2
bugdown: Use named match for Gravatar email
...
(imported from commit 778c4b6d754f975c89a91336593d2f62e49249d5)
2012-10-21 20:16:07 -04:00
d68cf3696b
populate_db replay: Use bulk_create to create objects.
...
This cuts the time required to import 38000 messages (with 140000
message receipts) from about 10 minutes to just under 40 seconds on my
laptop with sqlite.
(imported from commit d53b0d1360408c77f38353b58fbb25875262cb40)
2012-10-20 18:26:18 -04:00
61a33f4c4d
bugdown: Disable line numbering of code blocks
...
Pygments renders these using tables, which breaks our client's assumptions
about what <tr>s mean.
(imported from commit 46d37395785e06fb183d17b08afed4c6f5957baa)
2012-10-19 23:54:10 -04:00
dc8bc9e1af
codehilite: Allow the caller to disable line numbering
...
(imported from commit 4257cc1b8b1e2a78cb36d4fdf105a5864c1b20fa)
2012-10-19 23:54:10 -04:00
0edbc80616
Use our local copy of codehilite
...
(imported from commit bc9704a19017061d55ff0e16589d57ee0c46caa5)
2012-10-19 23:50:32 -04:00
5e1654b9b5
Import GitHub version of Markdown codehilite extension
...
(imported from commit 166f85fabf8a80858240c2610437002e49f98278)
2012-10-19 23:30:47 -04:00
086e1b1c33
Make bugdown able to contain sub-modules
...
(imported from commit 6fa5a6cd8f972e3afb21b65bdfca927e824194aa)
2012-10-19 23:30:14 -04:00
c851bc4632
Generate initial API keys in the same manner as passwords
...
(imported from commit 754fa391b441c520082650a0cf3bacf050bf81b3)
2012-10-17 18:23:01 -04:00
cc8a14fcf8
Create accounts with passwords which are deterministic but hard to guess (from the outside)
...
(imported from commit 964610fec6c4690c1e881f2bab252296663c819a)
2012-10-17 18:23:01 -04:00
c31462c278
Allow users to embed Gravatars in messages
...
We'll use this internally for the commit bot. We might eventually disable it
for external users.
(imported from commit 3136cd9faadc6b81355889d2ee6472985da87fbe)
2012-10-17 01:11:33 -04:00
81f0d61c3b
Factor out Gravatar hash calculation
...
(imported from commit 29872722fb4856773d98fc987a1e2d6eb99ad8b2)
2012-10-17 01:09:16 -04:00
ed094b6a84
Markdown: Disable headings using our extension
...
(imported from commit cd40f6bf010ff30775ca2af494b9566843444797)
2012-10-16 11:39:00 -04:00
d8556707aa
Markdown: Disable images
...
(imported from commit 6656b15fa690b463265af6384a73529ee635f688)
2012-10-16 11:35:58 -04:00
fa729c838a
Move our various Markdown hacks into their own file
...
(imported from commit b03a5c64cc95964936c4aba7d667807969e35d21)
2012-10-15 16:16:09 -04:00
d91ec7056f
Add a Django query helper
...
(imported from commit cf1f87eb4fb2b6f44ec0ebafc4cad8dcc9bc5e5c)
2012-09-28 19:33:15 -04:00
669bb789f8
Add caching decorators
...
(imported from commit 7a4d9257ea8c6a86a5ffb498f726c4c2eb42bc9f)
2012-09-19 13:01:58 -04:00
c6fcbef861
Add simple parallel computation library.
...
(imported from commit bd17f16c1645f5bb20e0b5e0799948c7f05fede0)
2012-09-14 13:32:25 -04:00
Leo Franchi
Tim Abbott
Jessica McKellar
Keegan McAllister
Waseem Daher
Luke Faraone
Leo Franchi
Reid Barton
Zev Benjamin