Commit Graph

125 Commits

Author SHA1 Message Date
Tim Abbott d679a72952 [manual] Cache results of the Twitter API in the database.
This should substantially improve the repeat-rendering time for pages
with large numbers of tweets since we don't need to go all the way to
twitter.com, which can take like a second, to render tweets properly.

To deploy this commit properly, one needs to run

./manage.py createcachetable third_party_api_results

(imported from commit 01b528e61f9dde2ee718bdec0490088907b6017e)
2013-03-11 13:15:55 -04:00
Waseem Daher 29d504149c twitter: Only render the first Twitter link we see, not all of them.
(imported from commit 68582224962da14795d91a68197db1001ae7b75c)
2013-03-08 17:45:58 -05:00
Tim Abbott f37c6e7a1a do_create_realm: Use internal_send_message.
(imported from commit 92edff133d16f72cf954671392e413066fb22955)
2013-03-08 17:24:33 -05:00
Tim Abbott a6c461e3e0 internal_send_message: Render content before sending.
(imported from commit 00b80c71ccf5b1388d64cf8b6a2cb21b818ece4a)
2013-03-08 17:24:33 -05:00
Tim Abbott 254bc9f361 Attempt to render messages before accepting them from the user.
This fixes trac #407.

(imported from commit e33647eec32266790f864d14ad377d51956d2a6f)
2013-03-08 17:24:33 -05:00
Tim Abbott 2f022ebb34 bugdown: Allow caller of convert to tell whether we succeeded.
(imported from commit 9e3cab73e0d9e9ad31ba770210e535584cca3bb2)
2013-03-08 17:24:33 -05:00
Waseem Daher 0e2e754c3d [manual] Use Twitter API v1.1 for inline previews.
This commit adds a dependency on python-twitter,
whose upstream is at https://github.com/bear/python-twitter,
and which for now needs to manually be installed on our
servers from the Debian package in sid.

(imported from commit 80cd9f4f59a6f0de6b75ac95e412c69e2a2e2490)
2013-03-08 17:10:06 -05:00
Waseem Daher 1df648baa9 Add inline preview of Twitter links.
This uses the unauthed v1 of the Twitter API, which is going to go
away soon, but it's fine as an interim measure.

(imported from commit 709a250271321f5479854a363875c9da43e6382d)
2013-03-08 14:31:18 -05:00
Tim Abbott 4ab5cabb80 bugdown: Split out iteration through a tree into helper function.
(imported from commit ba73832c61eccdefe45f8e0d935216e6cc431176)
2013-03-08 14:31:17 -05:00
Waseem Daher 4ea7ac102f Allow internal_send_message to send cross-realm stream messages.
Prior to this change, any stream message sent by internal_send_message
could only be in the realm of the sender.

This was a problem most notably for... the tutorial bot, with the
hilarious consequence that the tutorial worked fine in humbughq.com,
but failed to start anywhere else.

(imported from commit 33a904a28e3a57e1a2cf9172c2e2a75b50967a50)
2013-03-06 23:04:57 -05:00
Reid Barton bc0dbbb566 Detect image-ness by end of path component, not end of entire URL
(imported from commit 9dd2e9c7273d3c7d071a8e5f82014e28b2a99ee4)
2013-03-06 19:02:29 -05:00
Tim Abbott acab720502 Add support for making Tornado dump its stack via SIGUSR1/2.
With this change,

pkill -SIGUSR1 -f runtornado

will dump the stack and SIGUSR2 will enable an interactive debugging
session.

This fixes #613 for Tornado which was the original motive for that
ticket; I'm not sure whether we want to do this for our Django
processes as well, but it would be easy to do so if we did.

(imported from commit a7de7c6070f4bf0404bed6f434e6a6b291d66a26)
2013-03-06 14:19:32 -05:00
Jessica McKellar ac305ffc1d Add a management command to deactivate a user.
(imported from commit 14ef58bdd2cd20c05c68cd53cf911711d3bdb5fd)
2013-03-06 11:57:04 -05:00
Waseem Daher 0dee6235cc Don't image-ify :( and other smileys 😞
(imported from commit 25323f19572c24da36267064a1ca820d80c9db79)
2013-03-05 16:22:51 -05:00
Waseem Daher 9f0877a7f2 Add emoji support 👍 🎩 💩 😀
(imported from commit 5f15ddb4492aad7cc4a9b3249acac8b8edc0a177)
2013-03-04 11:08:00 -05:00
Waseem Daher 1650dba322 Intelligently preview Dropbox image links.
Even though they look like images, they're not -- you need to
append ?dl=1 to get the image version.

(imported from commit 2a05e7c58f475c908687110d9191f8709425c660)
2013-03-04 10:38:42 -05:00
Tim Abbott 02bf89c3c3 Add inline previews for images and youtube videos.
(imported from commit 13711a7821318c8bff7c483e874756a63d761c15)
2013-03-01 14:21:35 -05:00
Tim Abbott 1e9a85ff05 Fix use of case-sensitive comparisons on email addresses.
(imported from commit d420169640a9f9c034b3d9ded207e583691f6652)
2013-02-28 17:49:57 -05:00
Jeff Arnold fcd033e33e [schema] Save enter_sends on the server in the database.
(imported from commit 4d82f6aaf5918f155a930253c9cc334dbcc0d97a)
2013-02-27 17:25:29 -05:00
Keegan McAllister 29b98e5546 Display invalid links as plain text rather than linkifying with empty href
Fixes #1001.

(imported from commit 116df72f37e45dd87c34cd836918dd559630fe76)
2013-02-27 11:29:05 -05:00
Keegan McAllister 75704ad1e8 Consistently use #!/usr/bin/env python
At Ksplice we used /usr/bin/python because we shipped dependencies as Debian /
Red Hat packages, which would be installed against the system Python.  We were
also very careful to use only Python 2.3 features so that even old system
Python would still work.

None of that is true at Humbug.  We expect users to install dependencies
themselves, so it's more likely that the Python in $PATH is correct.  On OS X
in particular, it's common to have five broken Python installs and there's no
expectation that /usr/bin/python is the right one.

The files which aren't marked executable are not interesting to run as scripts,
so we just remove the line there.  (In general it's common to have libraries
that can also be executed, to run test cases or whatever, but that's not the
case here.)

(imported from commit 437d4aee2c6e66601ad3334eefd50749cce2eca6)
2013-02-20 16:02:30 -05:00
Tim Abbott 9ef3754682 Use bulk_create to create sent messages.
Previously we were doing 100 queries for new messages being sent to
the main hacker school channels; they were faster than many similar
instances because they were all done within 1 transaction, but still,
send_message_backend would be spending up to 200ms (and 148 queries)
querying the database with the previous code on prod; this new version
should do a fixed number of database queries per message.

(imported from commit 3799e63aebb6f017932ddb0fe1f6209281c0ddcf)
2013-02-19 17:36:22 -05:00
Tim Abbott d364e450ea Move create_user to its own file.
(imported from commit b3dc11d3730634b35256dca940d479a81441c062)
2013-02-19 17:36:22 -05:00
Leo Franchi 49328c2551 Unify rabbitmq usage to use only one queue.
To work around the issue we're having with queue draining between
parallel blocking connections, use the same rabbitmq queue for both
activity and presence events, keyed on a 'type' flag in the message
itself.

(imported from commit 188e8fda1695734e52c5740db2195072cfc81479)
2013-02-15 11:43:17 -05:00
Leo Franchi d4a582fa1c Handle SIGINT and SIGTERM in process_user_activity by closing communication gracefully
(imported from commit dcf813fa8af1aaf5d6cc635268757e40c6c202ab)
2013-02-15 11:05:31 -05:00
Leo Franchi c2074813ca Generate unique consumer tags when connecting to the queues
(imported from commit cd9c78db46ae1a36b46bebf5cb0d226e15f71ba4)
2013-02-15 11:05:31 -05:00
Leo Franchi 5d82dc5ce2 Don't match ** for strong across newlines
(imported from commit e3bc0b409bd99c3707746c23eb73e11342cfe178)
2013-02-14 11:56:53 -05:00
Tim Abbott 2f4b62048e gather_subscriptions: Fix making O(streams) database queries.
This cuts the MIT Zephyr home page load time and also #subscriptions
database-sourced load time for me from about a second to about 50ms on
my laptop.  The home page still takes about 600ms to load due to
templating.

(imported from commit 1cfd8c750301abe6b6a881be4b2857532be947ec)
2013-02-12 16:25:45 -05:00
Leo Franchi 168f8623e4 Add https?: greedy url matching before falling back to our url guesser
(imported from commit 9e6e5a0522e6501b354a56223c2639841d290d4b)
2013-02-12 12:24:15 -05:00
Leo Franchi 7af226398e Only connect to RabbitMQ if it is enabled
(imported from commit 9726595bbb850c903177b5f64612eb4aa82e36ce)
2013-02-12 12:12:28 -05:00
Leo Franchi c348dd3587 Remove get_instance() from SimpleQueueClient
SimpleQueueClient was not thread-safe as the pika method
calls that we are using might disconnect and end up in a bad
state if two threads enqueue messages at the same time.

In order to avoid this, each user of the rabbitmq queue
gets its own instance of a SimpleQueueClient.

(imported from commit 694083b75cd58a60b8de282a8f40eb92a864c5ce)
2013-02-11 18:05:57 -05:00
Leo Franchi 0a0c4bb9a0 [manual] Use rabbitmq for asynchronous presence updating
Note: When deploying, restarting the process-user-activity-commandline script is needed

(imported from commit 63ee795c9c7a7db4a40170cff5636dc1dd0b46a8)
2013-02-11 18:05:57 -05:00
Leo Franchi 31f87481d0 [manual][schema] Add an API for user presence (idle) information
Adds a new db table for storing presences, and an API for setting
an individual user's idleness as well as fetching all idle status
for all users in a realm

(imported from commit 5aad3510d4c90c49470c130d6dfa80f0d36b0057)
2013-02-11 18:05:57 -05:00
Jessica McKellar 53f0b00785 Add a management command to change and log user emails.
(imported from commit 0bf25433a371f7e46ade96019f4a9ceebe395fe5)
2013-02-10 16:45:25 -05:00
Keegan McAllister 9edb616feb queue: Add logging
(imported from commit db2f15cf5f6a683f03d08a2e8b8ee4547537daf7)
2013-02-08 13:57:22 -05:00
Keegan McAllister 89c544f21f queue: Refactor SimpleQueueClient initialization
In anticipation of subclassing.

(imported from commit 1ebee932aed1b472bf9a8ab6d9bc30519f587c83)
2013-02-08 13:57:22 -05:00
Keegan McAllister d349de903b queue: Get rid of lazy initialization in SimpleQueue
Instead make it a singleton with a get_instance() class method.

(imported from commit e32cabf77b43361e74a11a23bba3a6d9fb32f82f)
2013-02-08 13:57:22 -05:00
Keegan McAllister b2bc7941dc queue: Use normal instance data in SimpleQueue
Code like this is dangerous:

    class SimpleQueue(object):
        queues = set()

because all instances will share the same 'queues' set object.

We don't really need multiple instances, but neither is there a reason to break
them.  So do the normal Python thing instead.

(imported from commit a56bb8414dd549cfd312c9565df198cf9d20f08a)
2013-02-08 13:57:22 -05:00
Keegan McAllister f14db52f22 queue: Rename SimpleQueue to SimpleQueueClient
It doesn't represent a single queue; that's what the 'queue_name' arguments are
for.

(imported from commit 8382ae6793ca7c3ae4b5aacf2c128113a7a88fb6)
2013-02-08 13:57:22 -05:00
Keegan McAllister 2617bbcdff queue: Remove shebang line
There's no reason to run this file as a script, and it's not marked as
executable.

(imported from commit 52bddaf2b59333c1f37b2cfc90b92a164c2e6d3c)
2013-02-08 13:57:22 -05:00
Jessica McKellar ad8e9598f6 Log domain in do_activate_user.
(imported from commit 0e39b5ddc395ff245f8e3b0252ea3b33a90860f1)
2013-02-08 13:20:19 -05:00
Jessica McKellar 50bab7af8f Log domain in user creation event.
(imported from commit d5aca898781b4f33a749536181e50f941bd16b59)
2013-02-08 13:20:19 -05:00
Leo Franchi c637b9cf44 Handle chars at end of link better
(imported from commit 53842bc17bbb9cfb555738ee6b72291d7ce61d67)
2013-02-06 11:51:52 -05:00
Waseem Daher 66850b5dcd Make gather_subscriptions pass along whether or not a stream is invite-only.
(imported from commit f71c171185a0552dba4917967d3feffb9c92c4fe)
2013-02-05 17:24:29 -05:00
Luke Faraone ac0dbd0864 Return in_home_view in gather_subscriptions.
This enables us to do client-side filtering on messages by the value of
this setting.

(imported from commit e69a7772dc90c67cb91ea8f630f5b78c902e8fa8)
2013-02-05 15:34:37 -05:00
Luke Faraone e8afaa8b8e Return a dictionary in subscriptions/list instead of a tuple.
This will give us flexibility in the future to add new properties to the
list.

In order to support that, we now do a list comprehension rather than just
returning the gather_subscriptions list in get_stream_colors.

(imported from commit a3c0f749a3320f647440f800105942434da08111)
2013-02-05 15:34:37 -05:00
Leo Franchi 054420fc38 Match .co.uk before .co in linkification
(imported from commit 2461cb4d49eef15431dde30dda646b25dc860a7b)
2013-02-05 13:04:45 -05:00
Jessica McKellar fdb0d15080 Don't let non-subscribers of invite-only streams query the membership.
(imported from commit 01bd8ea089dec96e487e5e82fb38df65703679ae)
2013-02-05 10:12:04 -05:00
Leo Franchi 3569e8e9b2 Sanitize links to prevent XSS, and handle emails
(imported from commit 622396efde50d9f5e3501f5d780c344ad0692662)
2013-02-01 18:42:27 -05:00
Leo Franchi cc58044234 Rework linkify regex to match more urls
(imported from commit 0e1a1df88363374ffbc802f83f43eb0fac8c99ea)
2013-02-01 16:14:29 -05:00