Commit Graph

652 Commits

Author SHA1 Message Date
Tim Abbott f1b5d69622 decorators: Replace POST with REQ everywhere.
It's strictly more functional, and having a single arguments
extraction decorator makes our codebase less confusing.

(imported from commit 2a5618c04b486268a462a24a1481ac030f15eac4)
2013-05-09 10:35:49 -04:00
Leo Franchi 4173862833 Send notification with missed message to users if they received a PM while offline
This is configurable in the user's settings on Humbug

(imported from commit 80bf6d7a6f0dc3811117548e2225865db8b533d9)
2013-05-09 10:35:48 -04:00
Tim Abbott 74148518b4 events: Fix the requesting client type being always internal.
(imported from commit b03bee006547ee811c2f0af66d82dfe6bc4331fc)
2013-05-09 10:35:47 -04:00
Leo Franchi 0da2f51ddb Calculate presence information with delta between server timestamp
(imported from commit b4b367ed620878e6d01287cb1aaae0730c7475c8)
2013-05-06 13:52:23 -04:00
Luke Faraone 6bc2d21b87 Make audible notifications a tunable.
This decouples from Chrome notifications, which gives us cross-platform
support in at least modern browsers.

We log this action so its replayable in our message logs.

This implements the model change indicated by the previous schema commit.

(imported from commit b21213cdde54f43670bbb0bf1f607147fc732b38)
2013-05-03 15:08:49 -07:00
Leo Franchi 33eef54c5b Re-send unread message counts if XHR fails
(imported from commit 062e76ca7db177540ed4e9ab0d331b9ecc2ee5f1)
2013-05-03 11:14:46 -04:00
Tim Abbott bfaa698ea6 get_old_messages: Don't fetch messages from database unless we have to.
Previously, we were fetching Message.objects.select_related() from the
database, even if we actually ended up fetching the message dicts from
memcached and thus not actually using them.  Especially in the cached
case, this resulted in a lot of overhead where the Django ORM put
together Message objects with lots of data in them that were never
used.  This commit switches the model to only fetch the full message
objects from the database for those messages which are not found in
the memcached caches.

Here are the timings for get_old_messages before this patch was applied:

(cached)
127ms (db: 42ms/2q) /json/get_old_messages (starnine@mit.edu via website)
385ms (db: 105ms/1q) /json/get_old_messages (starnine@mit.edu via website)

(uncached)
315ms (mem: 6ms/41) (db: 90ms/22q) /json/get_old_messages (starnine@mit.edu via website)
507ms (db: 94ms/14q) /json/get_old_messages (starnine@mit.edu via website)

Here are the timings for get_old_messages after this patch was applied:

(cached)
 80ms (db: 9ms/2q) /json/get_old_messages (starnine@mit.edu via website)
133ms (db: 4ms/1q) /json/get_old_messages (starnine@mit.edu via website)

(uncached)
230ms (mem: 9ms/41) (db: 48ms/23q) /json/get_old_messages (starnine@mit.edu via website)
385ms (db: 55ms/15q) /json/get_old_messages (starnine@mit.edu via website)

(imported from commit c4748513392a906393314aa7cd41d98a69865411)
2013-05-02 15:32:23 -04:00
Zev Benjamin e1c8d2f50f Return match_subject and match_content from messages_in_narrow()
This fixes a bug where if you were narrowed to a search and received
a new message that belonged in that search, the message would appear
to have an empty subject and content.

(imported from commit fe1dbf584d3659d57c5b70c7eb45cb22bbc9732f)
2013-05-01 21:52:04 -04:00
Steve Howell 7dd64eb157 Handle github sending empty string for the stream.
(imported from commit 2fd85db1828be44ef63920d5df347b5f85acb573)
2013-05-01 17:39:54 -04:00
Steve Howell be1e18f864 Support branches whitelist for github
(imported from commit 066bb8ee028778cb39b43afc9737fd2117c91928)
2013-05-01 17:19:53 -04:00
Steve Howell b645c67994 Use stream from github webhook call (if supplied).
(imported from commit 4f57c4dec8ab5e833583a2b5912a92e8a2bd34c0)
2013-05-01 17:19:53 -04:00
Tim Abbott 9f0fc7c031 blueslip: Include window.location.href in error reports.
This can be useful for debugging what sort of narrow is happening in
addition to the URI decoding bug we're currently experiencing.

(imported from commit 0cb55fec4ac1afa986c747eb79236b4300c9e636)
2013-05-01 16:10:35 -04:00
Steve Howell 30f825ebfe Handle bad api_key for jira integration properly.
(imported from commit e6063431e81434faa5f32ac8f91a08f86bd46997)
2013-04-30 18:26:34 -04:00
Steve Howell fbad47ec28 api_key is url parm for jira webhook
(imported from commit 24624a9fcd7e6fdc15d23c2874a04e1465c3f3cf)
2013-04-30 18:14:11 -04:00
Tim Abbott f36d51edeb Update Zephyr Mirroring liveness check for new REST API.
(imported from commit d968fde21bd90510ea7bb7f85ecb9b97b41689f7)
2013-04-30 14:28:38 -04:00
Steve Howell 4450bbcbf8 @csrf_exempt decorator for api_jira_webhook
(imported from commit 6c4ef97b2312b6721bd50605efdf51f7affb514c)
2013-04-30 13:24:47 -04:00
Steve Howell 23f0d0bcb7 support stream parameter for jira to override default stream
(imported from commit b3be7b837f326968f9742c25ba04bdaaf6476b75)
2013-04-30 13:24:47 -04:00
Zev Benjamin fca8f84c14 [schema] Return highlighted subject and content from get_old_messages() when doing a search
We HTML-escape the subject in Postgres to avoid a server round-trip.
Unlike the rendered_content, which is already escaped and cached on
zephyr_message, we normally escape subjects client-side.  Escaping in
Django would require fetching the messages that match the query,
escaping the subjects, and then making a second query to Postgres to
insert the markup.  We could instead fetch the messages with subjects
marked up using non-HTML (some unique string) that is later converted
into the correct markup either in Django or client-side, but then the
escaping problem would just be with some random string instead of
HTML.  Since the function is pretty simple, doing the escaping in
Postgres itself is the least painful option.

(imported from commit 004931d8e496697c18650aee97b1a74c55a04cb2)
2013-04-30 11:40:27 -04:00
Zev Benjamin 8f17f99de2 Construct ts_queries using the new Humbug search configuration
(imported from commit 813ae86e9ea5f8af3ec2abd7d506cd707e699cdf)
2013-04-27 20:06:26 -04:00
Zev Benjamin aeea631bd2 Add JSON query for checking which of a set of messages are in a narrow
(imported from commit b1320cf0e1404d6b0f3dbf3a5b32b29287c698d7)
2013-04-26 17:45:22 -04:00
Zev Benjamin 6cdc3f67df Only fetch an extra message in get_old_messages if a narrow isn't specified
In the case where we're getting old messages for a narrowed view, the
anchor message id might not actually be in the result set so there's
no reason to fetch an extra message.

(imported from commit e610d1f2cb95be3ff9fce6dc95e40c560bc5bf84)
2013-04-26 17:39:08 -04:00
Tim Abbott 7c001822f2 Use bulk requests for updating memcached in get_old_messages.
Otherwise we end up doing 1000 requests to memcached, which can be
quite expensive.

(imported from commit be247f63b5fb88c6f4a45326261b66ea67fe1028)
2013-04-25 14:43:37 -04:00
Luke Faraone 3bb3bbb1ee Redirect to /accounts/register relative to the current domain.
This ensures that we don't always send you to hhq.c when you use OpenID.

(imported from commit ab4c4cfa201740fbddcaa2e51bc15bfe977e221d)
2013-04-24 14:20:07 -07:00
Luke Faraone c48ff1784c Implement OpenID signups.
This allows users on signup-eligible domains to sign up for Humbug using
Google Apps.

As part of this, we wrap the openid done view in our own code in order to
handle the "Unknown user" error. Therein, we create a PreregistrationUser
and then shunt the user through the rest of the confirmation process, pre-
filling in their name.

(imported from commit 066d9a1021384a6da2662352e62a701451bd6f44)
2013-04-24 12:03:42 -07:00
Zev Benjamin 75bbda1dad Add lower message id bound when marking messages as read for the mobile unread count hack
Having a message ID range significantly improves the query
performance because the number of messages Postgres has to consider
is much smaller.

(imported from commit 9b007457712f1c1502d526abea1b6fd742bd911d)
2013-04-24 11:30:24 -04:00
Tim Abbott 9b8f0fab0f Retrieve message objects from memcached in a bulk request.
On my laptop, this saves about 80 milliseconds per 1000 messages
requested via get_old_messages queries.  Since we only have one
memcached process and it does not run with special priority, this
might have significant impact on load during server restarts.

(imported from commit 06ad13f32f4a6d87a0664c96297ef9843f410ac5)
2013-04-24 10:44:56 -04:00
Luke Faraone 71a91197fa Enable absolute imports.
See PEP 328[1] for details. This feature was introduced in Python 2.5 and
will become mandatory in Python 3.

[1]: http://www.python.org/dev/peps/pep-0328

(imported from commit 7444eeba8a08d5f91b94c7921848f2274979bd76)
2013-04-23 09:51:17 -07:00
Leo Franchi 3c993db1cd Tweak format of signup statsd log
(imported from commit eecc741f76894f2403b8302b8a6114fba0b06c5d)
2013-04-22 11:07:42 -04:00
Leo Franchi 14fd888a08 Handle mime-type if uploaded image specifies it
(imported from commit 716908a528fd21daedd4df4d7a71f7606acce671)
2013-04-19 18:04:49 -04:00
Leo Franchi 082ea8642e [manual] Add support for parsing Pivotal XML activity
This adds a dependency on the python package 'defusedxml'

(imported from commit bd5c7655e636ab1aa945e619e7a0063187bafb83)
2013-04-19 13:46:27 -04:00
Leo Franchi 652b821d64 Add a bunch of statsd logging
(imported from commit 8c17a76fed08c34abab3cc45d76a0fb30774d73a)
2013-04-18 18:05:52 -04:00
Tim Abbott 9e85e6e0b9 Fix ordering constraint on get_old_messages query to UserMessage table.
Amazingly, this saves about 250ms on every get_old_messages query in
my testing on postgres.humbughq.com (previously, we were scanning all
rows in the zephyr_usermessage table rather than using an index).

(imported from commit 566a5ef0bbf3c2198fa9e0b63d34e38ac9c57d18)
2013-04-18 17:15:29 -04:00
Luke Faraone 546a376d1c Move API endpoint docs onto their own page.
(imported from commit 0472c9051ba75e8c8ea4f80941ca9ef3b7ad949e)
2013-04-18 09:57:57 -07:00
Luke Faraone ea7965d9bb Rework API documentation to cover all supported calls.
We now generate our API documentation page based off of JSON source.

(imported from commit 0e8a91ef4278684dbcad89cad39a1977672245fc)
2013-04-18 09:57:56 -07:00
Leo Franchi fb2b3ae21a Handle multiple preregistration user objects when choosing streams
(imported from commit 52faa0256a719bed8a8ccc120f8177cce20450e2)
2013-04-17 15:48:30 -04:00
Jessica McKellar a87e41d66c Make inviting users to your realm case-insensitive.
Before this commit, foo@Bar.com couldn't be invited to bar.com.

(imported from commit 04b16f66a6ec9e157bdc7523972b37432358f5cd)
2013-04-17 13:27:48 -04:00
Tim Abbott 34977a4017 Use a generator comprehension for history messages.
(imported from commit 1f3079cb33ea1feae0290e02778cb383726fd8c9)
2013-04-12 15:43:42 -04:00
Tim Abbott 5ab44f6993 Add note to invite notification on historical messages.
(imported from commit ea3da4bd5b3e461b37b1d5af685d96bf5ecdb847)
2013-04-12 15:43:42 -04:00
Tim Abbott 6f22fb706d get_old_messages: Drop old streams parameter from lurk mode.
Its functionality is now subsumed in what happens if you specify a
stream narrow.

(imported from commit 01fbd1de6ac20a35314511cd065ca36be766c2a1)
2013-04-12 15:43:42 -04:00
Tim Abbott 5e50ba09ff Backend support for historical messages.
(imported from commit d5e3674c66ae2fb54901ffb59baf72e7afcd1e68)
2013-04-12 15:43:41 -04:00
Tim Abbott f933d19826 get_old_messages: Add support for querying just Message table.
This works by rather than hardcoding e.g. "message__recipient",
using (prefix + "recipient") where prefix is either "message__" or "".

(imported from commit 3a27d6499bc869d6dd389b074cb7d7cf286760aa)
2013-04-12 15:43:41 -04:00
Leo Franchi 302cfcd48c Send client information for initial presence and process time differential
(imported from commit 99a51b7cc8b6c51c4e82757a984d07603b2980e3)
2013-04-12 09:11:40 -04:00
Leo Franchi 5d4b2305fe Send presence updates when a new user logs in for the first time, and when returning from inactive
This commit will incorrectly list past-online users as active, a shortcoming that is
addressed in the next commit

(imported from commit b018767df686f88c0ca939c067c573e4d7cea357)
2013-04-12 09:11:40 -04:00
Luke Faraone 1482966b41 Correctly set Content-Type for uploaded files.
Boto usually handles this for us, but can't do autodetection like it
normally would because the file path we tell Boto isn't the original name
of the file.

(imported from commit 1ad4b04baf39be8887c86f7238438580651874ff)
2013-04-11 13:52:10 -07:00
Jessica McKellar 560636e372 views: remove unused imports.
(imported from commit 68f66fe79dfb72671e6a7c79bb98646ec729c701)
2013-04-10 16:57:51 -04:00
Jessica McKellar 7175dc534a Send invitation e-mails asynchronously through RabbitMQ.
This avoids 10s of seconds of delay when you invite several people at
once through the web UI.

(imported from commit 75acdbdb04caf62bbb08affc7796330246d8a00e)
2013-04-10 16:57:49 -04:00
Zev Benjamin f6a6a6b220 Add per-stream desktop notifications
(imported from commit b4a0576847b3aec1495f017ca9805febe80c9275)
2013-04-10 16:11:27 -04:00
Zev Benjamin 7e532a02ad Simplify subscription property views
This also changes the API for GET /json/subscriptions/property to
only retrieve the property for a particular stream instead of
returning all streams and their properties.  We weren't using this
functionality anywhere and the change makes the API more consistent.

(imported from commit 2799aec2550fd0558e2282beb19734d60801bdb8)
2013-04-10 16:11:27 -04:00
Zev Benjamin a2010871e3 Make subscription properties less free-form
(imported from commit eda607c2abfa51d2dadddc7b9ecba3e2d0b5be4d)
2013-04-10 16:11:27 -04:00
Tim Abbott 1b11eeb2bc Simplify the default_subscriptions code path.
(imported from commit 62894a5949621465fcfd8d25372316d7ab495252)
2013-04-09 11:58:07 -04:00