Commit Graph

2664 Commits

Author SHA1 Message Date
Zev Benjamin 2da4c0a4bf Remove extraneous SERVER_GENERATION definition
It looks like this was supposed to be moved in e0d9182c but was
copied, instead.

(imported from commit 094f43e9cfd4dde380a3d1ef79b1c273b54ce1a3)
2013-03-27 14:15:03 -04:00
Tim Abbott 4502e563d1 Reduce code duplication for coding filling memcached caches.
(imported from commit e7e053c6fa27fd7a1143696196e9ee974ec91c8f)
2013-03-27 12:59:24 -04:00
Tim Abbott 8fbc98601b Populate cache of Huddle objects on server restart.
(imported from commit 3214cfa70d0de01c42028a9d025f7607f609081b)
2013-03-27 12:59:24 -04:00
Tim Abbott 8b2cb6472b Cache the lookup of Huddle objects by hash.
This is used in the send_message code path.

(imported from commit 4d9a4d079fe5b10b94c11016e9ec8c5601addc2c)
2013-03-27 12:59:24 -04:00
Tim Abbott b2e76f50b9 Populate cache of Client objects on server restart.
(imported from commit 6d0ef309dadcd6c24fc9e17b0859becb053adccf)
2013-03-27 12:59:23 -04:00
Tim Abbott 36f09729e6 Run the memcached cache filling jobs in separate processes.
This will significantly improve the speed with which the memcached
caches are filled.

(imported from commit 445fec710a00b3d19b7962beddf89d4a75322eab)
2013-03-27 12:59:23 -04:00
Tim Abbott a774598a10 Populate cache of Recipient objects on server restart.
(imported from commit 752851c7c3ee565201dd59fb0455c82a8885d42d)
2013-03-27 12:59:23 -04:00
Tim Abbott cc065a3c90 Populate cache of Stream objects on server restart.
(imported from commit 131eeddc5ebe5cc0db2554514a730d2750078012)
2013-03-27 12:59:23 -04:00
Leo Franchi 79102044f4 Don't create a pika-connecting queue client if we have no RabbitMQ
(imported from commit 680e867abb61644cbc62a4280b4e47b03eb5abe7)
2013-03-27 11:06:10 -04:00
Tim Abbott 6fd4bf38c0 logs: Don't display database time when there were no queries.
(imported from commit 71297c920b63578ebc2770954345bfb9d4d8937f)
2013-03-27 08:19:27 -04:00
Tim Abbott 144987e581 logs: Provide total time passed for longpolled requests.
Also improve display of times passed -- we now use display short times
in milliseconds for easier reading.

(imported from commit 08e1e7e6acbef48453080864946f7602a3395e7c)
2013-03-27 08:19:27 -04:00
Tim Abbott 887389d16c logs: Output timestamps in milliseconds when less than a second.
(imported from commit 150038059a1d7c677004edf27d4af2e48bf41285)
2013-03-27 08:19:27 -04:00
Tim Abbott f3ad2d7a6b Centralize dispatch logic for which queue processor to use.
Previous we had around 4 copies of the logic for deciding whether we
should publish data via a SimpleQueueClient queue, a
TornadoQueueClient queue, or to directly handle the operation, which
resulted in their getting out of sync and buggy (see e.g. the previous
commit).

We need to add a lock around adding things to the queue to work around
a bug with pika's BlockingConnection.

I should note that the previous logic in some places had a bunch of
tests of the form "elif settings.TEST_SUITE" for doing the work that
would have been done by the queue processor directly; these should
have just been "else" clauses -- since we generally want that code to
run on development environments whether or not the test suite is
currently running.

(imported from commit 16bdbed4fff04b1bda6fde3b16bee7359917720b)
2013-03-27 08:19:27 -04:00
Tim Abbott 16574806f5 Don't initialize SimpleQueueClient inside Tornado.
Previously we had several files which initialized SimpleQueueClient()
for sending items to the UserActivity queue, even though those code
paths aren't used outside Tornado.  This resulted in slower Tornado
startup times.

(imported from commit ad97021ec18d3927233744037c548c22db33c321)
2013-03-27 08:19:27 -04:00
Tim Abbott 1df99937a9 Tornado: Fill the Tornado UserMessage cache using manual query.
The actual database query that we use to fill the UserMessage cache
only takes a few hundred milliseconds to run; however the process of
iterating through the results would take 3-5 seconds because the
Django ORM is not very efficient for small tables where we're only
interested in the integer values in a couple columns.

So we can save most of that Tornado startup time by just doing this
one query manually; I left the original query next to it in a comment
so it is easy to keep it all up to date as we change our product.

(imported from commit ac4675bcdda5d812ebfbe211450c85ee2787ee66)
2013-03-27 08:19:27 -04:00
Tim Abbott 4076704476 [django 1.5] models: Always return a string from __repr__() methods.
See http://bugs.python.org/issue5876 for an explanation for why this
is needed -- basically __repr__() needs to return a string, not a
unicode object in Python 2.

This causes problems on Django 1.5 because the more expressive
exception code in model.objects.get() will crash with a __repr__()
containing non-ascii unicode characters.

(imported from commit f44085e67d9d14629b821a29bbf65738f1794d6c)
2013-03-27 08:19:26 -04:00
Tim Abbott fde2243dbc [django 1.5] Update read flags handling to use new Django 1.5 syntax.
(imported from commit af1e2608b6d6d3677bf6266980d8dc77739aa4ba)
2013-03-27 08:19:26 -04:00
Leo Franchi dbb2157457 Make home view unread count only count messages newer than the pointer
(imported from commit 0f699ed23f59380b04606bd30600373956f89ae4)
2013-03-26 18:34:16 -04:00
Leo Franchi 9de2442bd9 Add infomation in JIRA integration in our integrations page
(imported from commit 2aef274c630cc918a6fde8fa33a4598ba8ee7f3f)
2013-03-26 14:14:47 -04:00
Jessica McKellar 297bc7f09b Don't hide the floating recipient bar when scrolling.
We made this change for performance reasons that don't exist now that
we only render a small portion of your messages, and it causes a
distracting flicker when you scroll through messages slowly.

(imported from commit 33379320f6b90d93ec8beac17323b287f8bb2485)
2013-03-26 14:06:27 -04:00
Jessica McKellar 9451451672 tutorial: streamline some sections, and bold user actions.
This may help with compliance.

(imported from commit 8ef250b07a0a1a7023bc74f89a46875c88fadef7)
2013-03-26 14:06:25 -04:00
Jessica McKellar 76dade3b6e tutorial: remove code formatting examples and just link to documentation.
Those examples make the tutorial feel much longer, and they aren't
relevant to people who aren't using Humbug to talk about code.

(imported from commit c3213775d26cf533b3d9bde691de08a53d427939)
2013-03-26 14:05:45 -04:00
Jessica McKellar 2fd56b3745 tutorial: de-emphasize that we don't auto-scroll.
It's not so black and white in a world where we auto-scroll at the
bottom, and we've observed that people trying Humbug over-focus on it.

(imported from commit 2057643f179d5d1666cb33438c5a513977197b37)
2013-03-26 14:05:45 -04:00
Jessica McKellar eaa2cc702e Autoscroll when you are at the bottom of your message feed.
(imported from commit 7ca92b13c874346f846aea8200a638a7914fe7d6)
2013-03-26 14:05:45 -04:00
Keegan McAllister 7309d57067 Output page_params as a single JSON object
This is a lot cleaner, and also cuts about 50-70 ms off of page load time in
local testing (with lots of users), presumably because there's less work to be
done by the slow Django template engine.

(imported from commit 257b700238ee5d9a4ae00a53011ed5bce018124c)
2013-03-26 13:51:21 -04:00
Keegan McAllister 6d9aca5f1a Move "page parameters" (email, enter_sends, etc) into a single object
(imported from commit 842b2371bf6364982f1358f1cd2d91118c7fb2bf)
2013-03-26 13:50:03 -04:00
Keegan McAllister 38ebefe68f Remove unused variables
(imported from commit c18d43165c1dbdee28107ef21e26e474b819c653)
2013-03-26 13:50:03 -04:00
Jessica McKellar cc8ee68514 tests: assert that text objects are instances of basestring, since they might be str or unicode.
This fixes tests that have been failing for me for, well, months, that
I've been ignoring:

======================================================================
FAIL: test_successful_subscriptions_list (zephyr.tests.SubscriptionAPITest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jesstess/dev/humbug/zephyr/tests.py", line 631, in test_successful_subscriptions_list
    self.assertIsInstance(stream['name'], str)
AssertionError: u'Denmark' is not an instance of <type 'str'>

======================================================================
FAIL: test_get_stream_colors (zephyr.tests.SubscriptionPropertiesTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jesstess/dev/humbug/zephyr/tests.py", line 515, in test_get_stream_colors
    self.assertIsInstance(color,  str)
AssertionError: u'#c2c2c2' is not an instance of <type 'str'>

----------------------------------------------------------------------

The more comprehensive fix to this is going through both our API and
JSON calls and ensuring that we always return unicode objects,
documenting that, and then testing that more specifically. For now, at
least have passing tests.

(imported from commit ed1875ea1f66c1f1e89f80502c0d6abb323dc489)
2013-03-23 00:19:38 -04:00
Jessica McKellar 53ec385d0b create_user: ask for realm as part of user creation.
With open realms, you can't infer the realm from the e-mail address.

(imported from commit c05afa586f121a38673f9e62281cf26c8590570b)
2013-03-23 00:19:37 -04:00
Jessica McKellar f87ffd0703 create_user: fix missing spaces in options error message.
(imported from commit 25ea284b17dfb3160aca336e31203991f70994d8)
2013-03-23 00:19:37 -04:00
Jessica McKellar e5f1dad409 create_user: remove unused imports.
(imported from commit b2dd10f73fb0285c2c92ab56b1111cf9e859cedf)
2013-03-23 00:19:37 -04:00
Jessica McKellar ea11ce4ae6 create_user: Add Terms of Service warning.
(imported from commit bdb31aebad397e98db77d3ea95ef17ac9dd4737f)
2013-03-23 00:19:37 -04:00
Jessica McKellar 0c3382fabb Always give hashlib.sha1 and friends bytes.
This fixes an experienced bug where you couldn't subscribe to a stream
with non-ASCII characters (failing with a UnicodeEncodeError), as well
as many other potential bugs.

(imported from commit f084a4b4b597b85935655097a7b5a163811c4d71)
2013-03-23 00:10:10 -04:00
Jessica McKellar 55240e3de2 tests: test inviting someone to a non-ASCII stream.
(imported from commit d1788cf4712800c1b3445ad47510ac488e7935b0)
2013-03-23 00:10:10 -04:00
Jessica McKellar c4ceb2c276 tests: test inviting a new user with non-ASCII streams.
(imported from commit 049ae5a77c87eaaf0313b4b83d1dd038b14dcc22)
2013-03-23 00:10:09 -04:00
Jessica McKellar 9b89d7dc19 tests: add tests for sending non-ASCII messages.
(imported from commit 16093b2a080e7b00149129a9b53f308d8688def2)
2013-03-23 00:10:09 -04:00
Jessica McKellar 531ad235df tests: exercise non-ASCII login information.
(imported from commit 51a79a637a2d624c78627c84cd93d10cd700dff5)
2013-03-23 00:10:09 -04:00
Leo Franchi 378a8939ec Use TornadoQueueClient for activity queue when in the tornado process
(imported from commit 93ca4295403786ae26a2f70be17410674e6d053c)
2013-03-22 15:55:49 -04:00
Leo Franchi f64bc59dfb Automatically register consumer once asynchronous connection is completed
(imported from commit 3d860a9b79a42beffbd3b73f49aa2c25986dd3c6)
2013-03-22 15:55:49 -04:00
Leo Franchi 452105e393 Reconnect to rabbitmq in our queue client if the connection is lost
(imported from commit 82ed67f0c0f93e9f79ff65ef55b397815d20aaeb)
2013-03-22 15:55:48 -04:00
Keegan McAllister 4baeaaa532 TornadoQueueClient: Don't stop the Tornado I/O loop on RabbitMQ disconnect
Requires Pika 0.9.6 or later.

(imported from commit 316ab9f2a6b5c2a25ae8584e2604222b33e0dec3)
2013-03-22 15:55:48 -04:00
Keegan McAllister 6ffc2bbe65 TornadoQueueClient: Provide a callback when declaring queues
This is required by Pika 0.9.8.  We need at least 0.9.6 for the next
commit; I had been testing with 0.9.5 previously.  Anyway this way
seems more correct as well.

(imported from commit bfb9e9e78938073001f70c4d28a5e07cc4ebac32)
2013-03-22 15:55:48 -04:00
Leo Franchi c346abe6e2 Send Tornado callback notifications via RabbitMQ
(imported from commit 83814d8c6b78fe725aedea9d98fb588ed31123e7)
2013-03-22 15:55:40 -04:00
Keegan McAllister 23b8833ec5 Handle Tornado callback notify in a more generic, less HTTP-centric way
(imported from commit 85a74d1b40461236c4c95ad688e9796ab50f0bbf)
2013-03-22 15:51:23 -04:00
Keegan McAllister 8db92fd995 Split out the actual Tornado views from the callback handling / notification
(imported from commit 26d40556bf7f545d3606a01d83b16c7265bafb0e)
2013-03-22 15:50:35 -04:00
Keegan McAllister 591816d80e Use TornadoQueueClient in runtornado
(imported from commit cc4d5505987dc2826d177ff93d037b17f78072b8)
2013-03-22 15:39:07 -04:00
Keegan McAllister 373d3ece1c queue: Implement TornadoQueueClient
(imported from commit 2803bc91e42aa20d6e6dec87bfadd2f890268d05)
2013-03-22 15:39:07 -04:00
Waseem Daher 6760e68f94 integrations: Document Capistrano integration.
(imported from commit e26774e3657a71e8f21bf8ef2b1b1a9c289707da)
2013-03-22 14:37:54 -04:00
Leo Franchi 7837e36d0c Clean up some code by consolidating
(imported from commit 3978da70286174ad3666d08bcc8db2eb28446399)
2013-03-22 09:49:54 -04:00
Jessica McKellar 4f1bf1b2f2 realm_stats: restrict some usage stats to only active users.
They are more meaningful this way -- the fact that your bots that
never log in or inactive users don't have colored streams shouldn't
impact that statistic.

(imported from commit b39debda338cbbad06957bc969b42862a888026a)
2013-03-21 17:38:20 -04:00