Commit Graph

4918 Commits

Author SHA1 Message Date
Tim Abbott 0b9e9bca64 gather_subscriptions: Fix some whitespace.
(imported from commit a36c21c1acf2c2fad4aebb78ab1ac07ab67ff285)
2013-06-25 16:34:44 -04:00
Tim Abbott 9924faaabd [manual] API get_public_streams: Return streams in a dictionary.
This way we can return properties of the streams other than just their
names in future versions of the API without breaking old clients.

The manual step required is to deploy the updated version of
sync-public-streams on zmirror.humbughq.com when we deploy this code
to prod.

(imported from commit 42b86d8daa5729f52c9961dd912c5776a25ab0b4)
2013-06-25 16:34:44 -04:00
Tim Abbott 45c692c1e0 tests: Simplify helper_check_subs_before_and_after_add API.
The previous API was very redundant, which meant that any refactoring
would result in lots of unnecessary changes.

(imported from commit e04f6cbd87c8f65d4eebbe6972d26998faa28a56)
2013-06-25 16:34:44 -04:00
Tim Abbott f618c4d2b3 tests: Switch to using common_subscribe_to_streams.
(imported from commit f172e9e39d211b7f31d95d6aa83528f631808edb)
2013-06-25 16:34:44 -04:00
Tim Abbott 21b3ec3e48 tests: Move common_subscribe_to_streams to AuthedTestCase.
(imported from commit 2451d31444933e2bd0cca61b06fdaa7ace572aac)
2013-06-25 16:34:44 -04:00
Tim Abbott f880b1a072 tests: Clean up arguments to make_random_stream_names.
(imported from commit 442e0059f047b1334f455a9b15071dd43bb59d33)
2013-06-25 16:34:44 -04:00
Tim Abbott ce50812bca API get_members: Return a dictionary of users, not a list of tuples.
I believe this should require no special work on deploy, since some
grepping of logs suggests we are not currently using this API query.

(imported from commit 240086f900c6680cbc90bf6a2f334a9e1f172df6)
2013-06-25 16:34:43 -04:00
Tim Abbott 400db86008 [manual] nginx: Pass post-rewrite URIs to FastCGI.
This requires us to do a puppet apply when it is deployed to each of
staging and prod.

(imported from commit eed631ce10340e7fe3252cd8a4f05fd59ef3c942)
2013-06-25 16:34:43 -04:00
Tim Abbott ae89b25d69 nginx: Add fastcgi_params to puppet.
(imported from commit 12e6b02cd2cb411ab83a29a486053df6dff9ebb8)
2013-06-25 16:34:43 -04:00
Kevin Mehall a06860ca68 Don't perform an additional SELECT just to get a count for statsd.
len() uses the QuerySet as an iterator, which performs a SELECT for all
of those rows.

(imported from commit d362fef375c5270bedade2a7b50aea7b1a559d1e)
2013-06-25 15:25:25 -04:00
Kevin Mehall 953e195be1 Refactor update_message_flags to do the query in the django process.
Trac #1398.

(imported from commit c001747d8e6d78a12fe535c36a81c12592976840)
2013-06-25 15:25:25 -04:00
Tim Abbott c4dc0afbf3 Clean up unnecessary UserMessage.flags_dict().
(imported from commit ece196da6c98db3bcc7079e8d44b557b3aac0edb)
2013-06-25 14:06:09 -04:00
Tim Abbott d6e04ce2d6 Fix @-mentions.
Trying to check whether a Django model object is inside a set of other
Django models is not correct in general, e.g.:

UserProfile.objects.only("id").get(id=17) in set([UserProfile.objects.get(id=17)])

returns False.

This bug appears twice in the function, once when computing which
users were mentioned and again when pushing the flags through to
Tornado.

(imported from commit b09ed550258f9df2611e1b0a60f87c48a51830f8)
2013-06-25 14:05:20 -04:00
Tim Abbott d537bf568c tests: Add a backend test for @-mentions.
(imported from commit da4e241c445d080b58e131d95d7d9fe89ecf31f0)
2013-06-25 13:46:53 -04:00
Tim Abbott f7363af887 Don't send desktop notifications for messages you sent to a stream.
Previously, we were only checking whether the message was sent by you
for PMs, not for streams that you marked as notifiable.

(imported from commit 668e102e885aa347cceff376c1873c1c62aff997)
2013-06-25 13:46:48 -04:00
Jeff Arnold c8e34c2bfa Make the first subject autocomplete option be what the user typed
(imported from commit af9f5cfce09a071eb461f93c8c47128897162de2)
2013-06-25 10:28:14 -04:00
Steve Howell 8363f9bfc6 Add mode to run only fast tests.
(imported from commit acb569a54551bdd31aad53eb47191771ae670493)
2013-06-25 08:46:02 -04:00
Steve Howell 660ee6571d Disable embedly everywhere.
(Before it had been disabled only on prod/staging, but we are
removing it everywhere, motivated by making tests run faster.
In particular, the call to embedly_client.is_supported() was
expensive, as it went over the Internet.)

(imported from commit ea12bf6e7ae84ce7e8023a0d314ecc4c07cbc0a8)
2013-06-25 08:46:02 -04:00
Steve Howell 67058cc26f Enable back door for clearing user's rate-limit history.
(This is helpful to make the tests run faster.)

(imported from commit 48c1beaed8cca76976bd6f54224c33460b55ceac)
2013-06-25 08:46:02 -04:00
Scott Feeney 9ca7d58cac Create minified_assets.json in the deployment root
This file is a dict mapping filenames like "min/api.js" to
"min/api.HEX_HASH_HERE.js". This will be useful later for figuring out
which hashed version of a generated file belongs to a specific deploy,
for reuse in future deploys with the same source files.

(imported from commit d03c408b37a4e2d494abd4b1577a70b5ff5c2e34)
2013-06-24 15:55:04 -04:00
Scott Feeney 6b8bc74b4e Replace SITE_ROOT with DEPLOY_ROOT
Whereas `SITE_ROOT` referred to the directory where settings.py is
located, *all* actual uses of `SITE_ROOT` were joining it with `..` to
get the root of the git checkout, a much more useful value.

`DEPLOY_ROOT` now represents the root of the git checkout.

(imported from commit 351437f9a5801e5c7c08a3a97619e863144e5cc8)
2013-06-24 15:55:03 -04:00
Scott Feeney fcb39a262c Add note that DEBUG must be off for cache-busting
(imported from commit 2f14af77fcf95835f7f31b15048f417e8a5682a4)
2013-06-24 15:55:03 -04:00
Tim Abbott 04f1b1da66 Don't flush the UserPresence cache for a realm unless status changed.
Previously we had an issue that every other update_active_status
request for a particular realm would result in doing the expensive
query to compute the list of active users in that realm.  It turned
out this was because on every update_active_status request, we'd queue
an event that would have the effect of clearing the cache, even if
nobody's tatus changed.  This fixes that issue, by only clearing the
cache for a realm if someone's status actually changed (or the 60s
timeout expires).

(imported from commit d5b829fe255a31c8cecb58458738f1e72a2cf6de)
2013-06-24 14:34:51 -04:00
Tim Abbott c87d182102 Avoid saving user_presence twice when creating a new object.
(imported from commit 72af41b4e0b85f9a8605f5040c321c80e16e8a97)
2013-06-24 14:34:51 -04:00
Tim Abbott 8f0fbfb562 Fix message_is_notifiable to not include messages sent by user.
(imported from commit 196696da1350bba176985acce4abe36d5cedd24f)
2013-06-24 13:56:45 -04:00
Kevin Mehall da26a6a4fd Merge http_link and autolink regexes, as they share many workarounds.
(imported from commit 62615af5bf6d48620ec3f9e2523fecbc1ff01441)
2013-06-24 13:41:01 -04:00
Kevin Mehall c5c47fc921 Simplify and comment AutoLink regex.
Override python-markdown to compile with re.VERBOSE.

(imported from commit 77a797c79fb3f6c11948f431715c8bb624dc8bc7)
2013-06-24 13:41:01 -04:00
Scott Feeney 72a3112317 Use www.humbughq.com for API download link
wget users report certificate errors on the API download, apparently
due to wget's lack of support for TLS SNI until recently. Adding www
to the download link should work around this problem. (Trac #1400)

(imported from commit 517375eb22532304a774d66549c50691504ecc93)
2013-06-24 13:19:37 -04:00
Tim Abbott d2d1613153 send_message: Don't fetch unnecessary data on recipients' UserProfiles.
(imported from commit 4e4b5bdb510610cf7198bfb01fc465fe15bd834d)
2013-06-24 11:42:17 -04:00
Tim Abbott edc4f51b00 do_update_message: Check max message and subject lengths.
(imported from commit 48e8b21c7ca122afc40fcd1482d26185f50cadab)
2013-06-24 11:42:17 -04:00
Kevin Mehall bab2c2f93a Don't print an error when nodejs is at /usr/bin/node
(imported from commit d9d7b1d258825c4ea9804eae71dffe54eedc3c1c)
2013-06-24 10:46:36 -04:00
Kevin Mehall fb678e330b Make test-all's check for trailing whitespace give the line number
of the failure.

(imported from commit 721a19145d52c285e134b82c4b0f6e7faa3c64db)
2013-06-24 10:42:16 -04:00
Tim Abbott 8b4d406083 get_old_messages: Use a raw database query for empty narrows.
This improves the performance of the get_old_messages queries when
loading the home page by about a factor of 2.5x.

(imported from commit 581840a6ed7b391c9d9fde67f368fce816e567e2)
2013-06-24 10:17:10 -04:00
Tim Abbott a8d86d5fb2 get_old_messages: Refactor handling of UserMessage flags.
This is in preparation for having a case in which we query the
database directly to get the message flags, without going through a
UserMessage object.

(imported from commit d5218974680b0c4b028a84f3aae1c8242ceb08ce)
2013-06-24 10:17:10 -04:00
Tim Abbott e5e0544816 get_old_messages: Clean up code duplication in database queries.
(imported from commit c35aa71fc848c77f9c5c8edd8cc8acaa9481df74)
2013-06-24 10:17:10 -04:00
Tim Abbott 0cc9960ff5 compress message dictionaries in memcached using zlib.
This decreases the average size of the message dicts in memcached by
about half, without any significant change in the overall performance
of the query.  Since these message dicts are a significant fraction of
what we put in memcached, this seems like a worthwhile optimization.

(imported from commit 3896328074aa4344b8ac7c7ba7685f0a167ec7ad)
2013-06-24 10:17:10 -04:00
Tim Abbott 631a9048f7 Send message dicts to memcached as JSON dumps.
memcached stores objects sent to it using pickling, which is very
slow.  We work around this by sending memcached strings (i.e. JSON
dumps); pickling doesn't slow things down too much if all it is
getting is a string.

(imported from commit 0f0e534182eccb76c5731198e05a9324a1cef316)
2013-06-24 10:17:10 -04:00
Tim Abbott 222ef672b5 [manual] Use ujson instead of simplejson.
This saves something like 15ms on our 1000 message get_old_messages
queries, and will save even more when we start sending JSON dumps into
our memcached system.

We need to install python-ujson on servers and dev instances before
pushing this to prod.

(imported from commit 373690b7c056d00d2299a7588a33f025104bfbca)
2013-06-24 10:17:10 -04:00
Tim Abbott 678dd502ef Add command-line tool to profile get_old_messages requests.
(imported from commit bd7fc27b0c6fc1ae4f82bb74763736f9163b90bf)
2013-06-24 10:17:09 -04:00
Zev Benjamin 90f4b1f227 Preserve the hash when reloading
Previously, if you were narrowed when we reloaded you, you'd be
kicked out to the home view.

(imported from commit c00f1b92c9a4f559998463e141a402b69873fd56)
2013-06-21 16:24:12 -04:00
Kevin Mehall 4f5b4fdb67 Bugdown: Elipsis followed by TLD should not be linkified. Trac #1388.
(imported from commit 3929e4bc68872e5f5ed58f35dd1b4eb53a2e5990)
2013-06-21 15:33:17 -04:00
Waseem Daher 0c01274989 Revert "Slight cleanup of registration page."
This reverts commit 91b4577c20afaf3ee9f5bed7f4e3451020960cf5.
Upon discussing with acrefoot, my new conclusion is that
the page should only look this way if you're going through
the workflow outlined in Trac #1391 -- otherwise it should
remain the way it is.

(imported from commit 57a1989d73526a357b47a4ac68dc911e8f2531cf)
2013-06-21 14:31:40 -04:00
Zev Benjamin bf9ad1874e puppet: Run pg_backup_and_purge.py nightly
(imported from commit 98555b970533adc3c60377490074607da9f78efb)
2013-06-21 14:08:57 -04:00
Zev Benjamin 15d13f8f40 puppet: Add script for doing Postgres base backups and purging old backups
(imported from commit 93a92729b2e964e054aa1af7bcb8a0bae3fd1b33)
2013-06-21 14:08:57 -04:00
Zev Benjamin 33b3b1fa62 puppet: Switch which S3 bucket we backup Postgres to
The old bucket was versioned and didn't allow deletes.  This was
great for paranoia, but not so great for being able to delete old
backups.

(imported from commit be79b5c582ca5ee466cdfea6d3093b6d5ba0e23d)
2013-06-21 14:08:57 -04:00
Zev Benjamin 1b6514b89f puppet: Use the correct Postgres archive command
I hadn't changed it previously out of paranoia in the case we had a
faulty failover and had two masters both uploading to the same place.
However, I now don't think this can happen, as recovery completion
will cause Postgres to start a new timeline.

(imported from commit d58f1aa306eff4f6fd950664ff658539c1249bdf)
2013-06-21 14:08:57 -04:00
Zev Benjamin bf82fadc95 puppet: Move /tmp to local storage on Postgres master servers
(imported from commit eae0a31faad6d95c8e2b55c11481aa19d7e108f2)
2013-06-21 14:08:57 -04:00
Waseem Daher 316d4e9220 Slight cleanup of registration page.
(In anticipation of the implementation of Trac #1391.)

(imported from commit 91b4577c20afaf3ee9f5bed7f4e3451020960cf5)
2013-06-21 11:57:48 -04:00
Michael McCanna 425a247569 Fix path injection for tools/review
(imported from commit 25bac426b71a55f29faa232dcac1d957f2941182)
2013-06-21 11:44:40 -04:00
acrefoot a37b08db58 In-Browser Notifications
Some browsers don't support desktop notifications. Some people haven't allowed permissions
for humbug to have desktop notifications. This is a poor man's attempt at desktop notifications.

We are adding the bootstrap-notify package (http://nijikokun.github.io/bootstrap-notify/)

Resolves Trac #1336.

(imported from commit 6a54f7d1875e765dabd32d94ada8ebe4474a3d71)
2013-06-21 11:44:25 -04:00