Commit Graph

3331 Commits

Author SHA1 Message Date
Leo Franchi c5d28a78a2 Don't send offline email notifications to bots
(imported from commit f7547f49580bf905c05a4fa2aa064fb82b7b4036)
2013-05-13 12:10:14 -04:00
Leo Franchi 7f04c8f9b9 Set a Reply-To header in our missed emails, and from noreply@
Otherwise, warn about not replying to this email.

(imported from commit bdcd8f510a8636b25c3c28aaef146a6e916a7d9f)
2013-05-13 11:59:35 -04:00
Tim Abbott f87e62a8f3 Show unread counts for the home view in the left sidebar.
(imported from commit e56da93c45dff30ae150b0da676bab2278d25718)
2013-05-13 11:58:58 -04:00
Tim Abbott 14feac48e7 All PMs are in the home view.
(imported from commit c5e1a06fe39322f6065f2e56ef00b5f2f0e8aaf0)
2013-05-13 11:58:58 -04:00
Tim Abbott 8f29a39991 Remove the newer_than_pointer_count hack in unread accounting.
This was a workaround for a number of other bugs we had, but at this
point just serves to make debugging more difficult.

(imported from commit 6662b7854c265bd8016f6c8ce75a095731211a45)
2013-05-13 11:58:58 -04:00
Tim Abbott b7e682b157 Fix home view unread counts being 1 lower than they should be.
(imported from commit c9f9a623e862eaf9cb38626318df855ef7bad993)
2013-05-13 11:58:58 -04:00
Tim Abbott 91f8768fb7 Fix private message unread counts never displaying.
(imported from commit fda193ceba36dba2cd7b0694e15d1b927365dc26)
2013-05-13 11:58:58 -04:00
Zev Benjamin 43794983e7 Fix a bug where the stream list would not get resorted if you unsubscribed and then subscribed from a stream
(imported from commit 9bde14f49d05062478fb5f356c24b29bedcdf3fc)
2013-05-13 11:44:56 -04:00
Leo Franchi 29413867b7 Use just one cache key for all tweet lookups
Since we log to statsd our cache time lookups by cache key, using a unique
tweet id for each lookup was just filling up our cache without being useful.

Also, log database cache lookups in a further namespace to distinguish between
memcached caches

(imported from commit a2a16b777fb7ab8cd066feee7344f9c8a3c107f5)
2013-05-11 15:51:40 -04:00
Zev Benjamin 5508fb5f91 Add restrictions on which streams you can send to
Users can send to any stream except invite-only streams that they
aren't subscribed to.  Bots can send to any stream except invite-only
streams that neither they nor their owner is subscribed to.

(imported from commit 623d34d249d923611ca7ca781b5b55205cd3e548)
2013-05-11 02:38:32 -04:00
Zev Benjamin 7b3f5ed14c Add interface for creating and viewing bots
We really should not be storing bot API keys in the DOM and should
require some sort of additional authentication before showing them,
but this seems reasonable for a first pass.

(imported from commit c7d75aa52e21894bf53917457e771c18de38bbcc)
2013-05-11 02:38:32 -04:00
Zev Benjamin c9114eef9d Add json queries for creating and fetching user bots
(imported from commit f745a705dedca66cf671ef19d7bc5f46ce70a306)
2013-05-11 02:38:32 -04:00
Zev Benjamin 73be68fbb9 [schema] Add support for bot users
(imported from commit 634a8211b41fd0040c95b51b96a88d3517fa8cf4)
2013-05-11 02:38:32 -04:00
Luke Faraone e18d76f4dd Readd a default for enable_sounds in json_change_settings.
Previously a default was missing from this function, which resulted in
users being unable to change their settings if they tried to disable
sounds.

(imported from commit 2dae67dcb2e8cb986abb6dee9659be2192993dd9)
2013-05-10 15:15:57 -07:00
Jessica McKellar 237b1077ed Factor out recurring message flag update AJAX request.
(imported from commit 7c0986b33ce57f7a1c58500a0006daeaaea0c87b)
2013-05-10 18:00:57 -04:00
Jessica McKellar c7555c6cf9 Add support for completely collapsing messages through the info popover.
(imported from commit 0a01c4cc8f99019233659c175d73826e16cc95ee)
2013-05-10 18:00:57 -04:00
Tim Abbott d45f840140 Send automated humbug error notifications to the 'errors' stream.
(imported from commit 6aa1ec59b2902b0f806b4caf77cbf76e6b7dfbec)
2013-05-10 17:52:45 -04:00
Tim Abbott 97840da0d2 Fix memcached accounting for bulk set/gets.
After this change, the memcached time consumed by doing
get_old_messages for 200 and 1000 messages respectively now look like
this:

200  63ms (mem: 6ms/3) (db: 4ms/2q) /json/get_old_messages
200 178ms (mem: 67ms/2) (db: 6ms/1q) /json/get_old_messages

which might help explain where the time is going on prod for some of
our slower queries.

(imported from commit b8fe83b175914b6796922a65a1c5537f4e7a9429)
2013-05-10 17:52:45 -04:00
Waseem Daher e0f1949cb4 Grow the composebox when we click on it.
The idea here being: if there's only one line, it discourages
me from writing a long message (and also makes me think that
enter will send).

(imported from commit 424d8d305d1965ce3199ce3227dac94b395945bc)
2013-05-10 16:22:17 -04:00
Steve Howell 898a69f42a Fix Pagination when user hits PgUp/PgDn/Space. (ticket 1237)
This commit takes control of keyboard-based pagination away
from the browser, so that we can use the effective viewport
size as the amount to page, as well as keeping a little bit
of overlap from page to page.  There had been issues with
pagination for a while, but the introduction of the always-open
composebox particularly aggravated the situation.

(imported from commit 45b9b7d5a6b322230c9d55e1be0b763dbce06e2e)
2013-05-10 15:36:24 -04:00
Jessica McKellar 7ed8f2420a Align Enter sends message with checkbox.
This also saves us a little bit of vertical space.

(imported from commit b7740f93259eb4678bac9370f1090d2c02954be7)
2013-05-10 15:28:14 -04:00
Jessica McKellar 7e0dc7ab30 Make the compose box take less vertical space.
(imported from commit ca28485d3def93e9c8b3e4a257f05ff4fb74ea18)
2013-05-10 15:28:14 -04:00
Jessica McKellar dd837c3c9b Don't let closing a modal clear your compose content.
This is the easy way out -- our compose logic needs a refactor
eventually.

(imported from commit c82fc31cadbe9edbd50aba6f8446b4ba9a311055)
2013-05-10 15:28:14 -04:00
Steve Howell b35c102815 Fix typo regression on top_navbar (minor).
(imported from commit 69ac7d32db0eb382db30448c89a50319562a5881)
2013-05-10 14:13:24 -04:00
Steve Howell 85ded308e9 (Refactoring) Extracted get_new_heights() out of resizehandler()
(imported from commit 13707f0d188676b96d4cad1d091b447435a22229)
2013-05-10 13:31:47 -04:00
Leo Franchi c28968c848 Only send huddles once per email
(imported from commit 5dc9e209346d14f3c5af1d07e626d771c7d3a95d)
2013-05-09 15:50:05 -04:00
Luke Faraone 3c08781764 Limit the number of URLs we parse at once to 5.
(imported from commit 876c0d0713b4a4d328e4a69e46c984694f5bce1b)
2013-05-09 10:44:08 -04:00
Luke Faraone 2c44ed1096 Log URLs that cause Embedly timeouts.
(imported from commit 84fdca163f6fefae5adc5ea4849d4d707aa7049e)
2013-05-09 10:44:08 -04:00
Luke Faraone f8d443c3c9 Time out embed.ly calls after 2.5 seconds.
(imported from commit 903b800a8444ff5c0b463dba17b2653e1775ae81)
2013-05-09 10:44:08 -04:00
Waseem Daher 47ac589f87 Make inline previews smaller.
(imported from commit 228f6f6f296877810824f98ea8001e47cf955cf1)
2013-05-09 10:44:08 -04:00
Luke Faraone 0513d9760a Batch-process and cache responses from the Embedly API
(imported from commit c427e2bf8f0f32dc632b05f41fa18ab10d0b6d03)
2013-05-09 10:44:08 -04:00
Luke Faraone 9fec3cd3c0 Explicitly skip Twitter links.
(imported from commit 2452dd4d92febbfc2b83ab0e09d7158f4bda76e3)
2013-05-09 10:44:08 -04:00
Luke Faraone fac83ed2b7 [manual] Add support for Embedly Extract
For sites that are supported, we now grab thumbnails for images + video
embed code for videos and use them in lieu of our existing embed code.

We also embed rich non-script content.

Special casing is done so that we don't embed images twice.

Some testcases were modified to avoid triggering Embed.ly

The manual step is to install python-embedly.

(imported from commit d725bab91675c61953116c5ca741055fce49724e)
2013-05-09 10:42:49 -04:00
Luke Faraone 8601c1670f bugdown: Factor out creating a link into its own method
(imported from commit 869cec9927570c4126b78f90aeedc2d5b542d097)
2013-05-09 10:42:19 -04:00
Luke Faraone 598c3a00f2 Comment change: link to a StackOverflow answer directly
(imported from commit a106b02e734956cab88445d159be18a092ae6d46)
2013-05-09 10:41:33 -04:00
Luke Faraone a470ec2e71 Pass the full stack trace in markdown processor errors.
(imported from commit fb157ab44fbf1465d1529ded4af4e04b2c7e9c55)
2013-05-09 10:41:32 -04:00
Leo Franchi 1267ffe542 Show unread counts next to expanded subject names
(imported from commit 6de1145cdec8d7e68395c1f9a7a95cd59136d805)
2013-05-09 10:41:32 -04:00
Tim Abbott 4968b804fa Change to the home tab when clicking stream filters in left sidebar.
(imported from commit c0cd7e233a29e2b2f2c348b8f977ea0d18e56fba)
2013-05-09 10:35:49 -04:00
Allen Rabinovich 2327bcd5ca Resetting .subscription_name line-height and adding padding-top to adjust vertical position
(imported from commit 067ac448dee9ff2fa4795ade1f99f8425d4bf28c)
2013-05-09 10:35:49 -04:00
Tim Abbott 8d6551e37d Send notifications when a message appears offscreen even if focused.
Previously we never sent desktop notifications when the browser was
focused, even if the message appeared offscreen.  After this commit
there are only a few cases when PM or other notifiable message doesn't
trigger a desktop notification:

(1) You sent it yourself
(2) It was onscreen when it arrived while your Humbug window had focus

(imported from commit e381c02c0e6794594d6934f57249a11ba2a88210)
2013-05-09 10:35:49 -04:00
Tim Abbott b1f75e8d3f Clean up notifications.received_messages.
This is trying to make the logic flow clear -- e.g. we check once, at
the beginning, for whether the message is notifiable, and the checks
for whether the various notification settings are enabled are more
parallel.

(imported from commit a68c71a53055191bc16682a85f739ed8e40ddeae)
2013-05-09 10:35:49 -04:00
Tim Abbott f1b5d69622 decorators: Replace POST with REQ everywhere.
It's strictly more functional, and having a single arguments
extraction decorator makes our codebase less confusing.

(imported from commit 2a5618c04b486268a462a24a1481ac030f15eac4)
2013-05-09 10:35:49 -04:00
Tim Abbott aa6dae7915 Fix improper use of zephyr.decorators.POST in a GET view.
(imported from commit 10cd186f2b0a4e6d89fe5cac2faf2b7eaa6b60be)
2013-05-09 10:35:49 -04:00
Tim Abbott 4ab4772758 Only register a Tornado rabbitmq handler if we're using rabbitmq.
(imported from commit 726934bc9acab3b19720f6b5ab1d0736ddd96832)
2013-05-09 10:35:49 -04:00
Steve Howell 92d6e80d57 Fix bug with uploading the same file twice using the clip icon.
See #1234 for details.  When you upload files the old-school way
(no drag&drop), there was a bug where you couldn't upload the same
file twice, due to us intercepting the change event and not clearing
out the file list when we were done.  Tested on Chrome, but uses
a known IE workaround.

(imported from commit 8120c2e8bce41f3964f4f5c21aad3a85df0e433d)
2013-05-09 10:35:48 -04:00
Leo Franchi 246b4a8459 Fix humbug url
(imported from commit b0fb11acd5f957a71387fe997d292fd6a9d17f39)
2013-05-09 10:35:48 -04:00
Leo Franchi cb8e1147e2 Log which missedmessage events are received
(imported from commit b0b662e97fd9199644c7cf1ac7bc11e6c4d25376)
2013-05-09 10:35:48 -04:00
Leo Franchi 23847545eb Update settings backend tests to handle new frontent config options
(imported from commit a17f931d51e11edec978c249ae116c734fb8d267)
2013-05-09 10:35:48 -04:00
Leo Franchi 065e6de420 Add management command to handle & email notifications about missed messages
(imported from commit 5b1479bc91d559e96467aa09405d823f9b2bbdf6)
2013-05-09 10:35:48 -04:00
Leo Franchi 4173862833 Send notification with missed message to users if they received a PM while offline
This is configurable in the user's settings on Humbug

(imported from commit 80bf6d7a6f0dc3811117548e2225865db8b533d9)
2013-05-09 10:35:48 -04:00
Leo Franchi 06f33a2bde Add code to generate emails for messages that a user missed
(imported from commit 36d3dd9bebbdc9d2af0b0f14acd7f0f6550be4b4)
2013-05-09 10:35:48 -04:00
Leo Franchi 29f857cca6 [schema] Add database fields for offline email notifications and timestamp
(imported from commit ef2ef9cb6c7b02d4a47c0061c58a40a37d91c353)
2013-05-09 10:35:48 -04:00
Leo Franchi 26fd7649f7 Add queue support for draining a queue completely
(imported from commit 50cc52f8b5b74b274024222596c6f2bd27832c89)
2013-05-09 10:35:48 -04:00
Steve Howell 13bc2e1859 Handle "REQUEST ENTITY TOO LARGE" during file uploads.
The filedrop library has a few canned errors, but it seems to mostly
let server errors come through.  We try to trap 413s to give a more
descriptive error than "unknown," but this is just a bandaid fix,
and we should see what's wrong with our prod configuration.

(imported from commit eac26406866d80340f24dbdca9f34408ddb92462)
2013-05-09 10:35:48 -04:00
Waseem Daher 567d04d386 Remove Glyphicons third-party dependency.
(imported from commit 4f5f4e915370f3631c558f8fd7feefc79c357e50)
2013-05-09 10:35:47 -04:00
Waseem Daher 0bcdee759e Use FontAwesome gear for settings icon.
(imported from commit fbcfd7fb94d7df34fec5dc49ab7deb366e6cf4d9)
2013-05-09 10:35:47 -04:00
Tim Abbott 74148518b4 events: Fix the requesting client type being always internal.
(imported from commit b03bee006547ee811c2f0af66d82dfe6bc4331fc)
2013-05-09 10:35:47 -04:00
Zev Benjamin 50fa22a3f9 Add utility for profiling iteration constructs
(imported from commit d4e55353475b86ba06da2d776fce2453cd33cc84)
2013-05-09 10:35:47 -04:00
Zev Benjamin 1b5fb31b2c Make viewport a module that caches height and width values
The .height() and .width() functions are actually pretty expensive for
the number of times we call them.  The viewport height and width
don't change often, though, so we can just cache them and recalculate
them on window resize.

(imported from commit 129fb8c058144125e2974f6b7967cd9f1a5c9ead)
2013-05-09 10:35:47 -04:00
Zev Benjamin 4de79cf723 Make within_viewport take the row offset and row height
The .height() and .width() functions are actually pretty expensive
for the number of times we call them.  The callers of within_viewport
already know the offset and height of the row, so we just pass them
in so the values don't have to be recalculated.

(imported from commit d1c077bd87463d695f0bbe337b6a8b04ac2d17ce)
2013-05-09 10:35:47 -04:00
Zev Benjamin 02dc15f426 Only re-render the streams list when the sort order has changed
(imported from commit 8122785097e2470e9e85f1f4c40bf1cd63264fa4)
2013-05-09 10:35:47 -04:00
Zev Benjamin 466beef6fe Optimize sort_narrow_list
The optimizations are:
* Sort over the list of subscriptions instead of the DOM li elements.
  This requires storing the li elements for each sub on the sub object.
* Do a bulk insert of the li elements instead of doing them one by one.

(imported from commit 1a987799930fc677e25f0bc2dcf66f83a4ac3163)
2013-05-09 10:35:47 -04:00
Zev Benjamin 45ba7c41e7 Don't generate a new favicon if we don't have to
(imported from commit 30e88dfddbdcd2343ce53270fece767d924c48f0)
2013-05-09 10:35:47 -04:00
Zev Benjamin ad6de0d401 Don't export several functions from stream_list.js unnecessarily
(imported from commit d8649259e07494fc70e5529e0cf627fdacc120a1)
2013-05-09 10:35:47 -04:00
Zev Benjamin 1f2529ba39 Do stream side-bar manipulation in event handlers
(imported from commit 27392dc6d05c6fa581c0370488e6baba0b11ce0f)
2013-05-09 10:35:46 -04:00
Zev Benjamin 7becd3e504 Move stream sidebar functions to their own module
(imported from commit 50771c1590eb3a0dbb151bf6a798bd14a4418857)
2013-05-09 10:35:46 -04:00
Zev Benjamin ceac6b0613 Fire events during different stages of subs.js subscription management
We now fire three events:
* subscription_add_done - fired when subs.js has finished handling a
  subscription_add event (all structures are set up, etc.)
* subscription_remove_done - fired when subs.js has finished handling a
  subscription_remove event
* sub_obj_created - fired when subs.js has created a sub object. This
  happens both when a new subscription is added and at page startup for
  all existing subscriptions

These events are fired whenever sub objects are created, even when
not tied to a subscription event.

(imported from commit a4863451f37e7fdbad480696b388ea788b01d6b9)
2013-05-09 10:35:46 -04:00
Zev Benjamin dc82d60c13 Remove no longer true comment
(imported from commit d06ce85ff76a8c9bd5691c004e9a77182ac295dd)
2013-05-09 10:35:46 -04:00
Waseem Daher 4892895d6b Additional always-visible-composebox fixups.
* Start a compose when we do a file upload
* Restore the "Formatting" and "Feedback" links.
* Dismiss composebox error messages when we defocus composebox

Realistically, the "correct" way to do this is not to have to
explicitly manage the composebox's state, as we do now -- it should
just be 100% visible and ready to send any time you click 'send'; it
shouldn't need to have first been composebox.start()ed.

(imported from commit 7f1725c229ed968a9b5500b25d600306173182a0)
2013-05-09 10:35:46 -04:00
Waseem Daher 8cb983ef5a Darken stream names in left sidebar.
(imported from commit 0ccc7940d7dabe31813d27fd7f77b7c75a11ab1c)
2013-05-09 10:35:46 -04:00
Steve Howell cee31a4f4e Fix unescaped angle brackets in search typeahead.
This was a pretty recent regression in the logic for
highlighting search typeaheads.

(imported from commit ba226f9cc7d6b09744ea07ef516fee280f23e048)
2013-05-09 10:35:46 -04:00
Allen Rabinovich 23c729a904 Switched to Source Sans Pro as the base font for the app and made a number of other cosmetic changes.
What changed:
   * Vector icons swapped in for the left sidebar buttons and filters
   * Lighter font weight in the stream filters list
   * Round color swatches in the stream filters list, with an inner shadow
   * Tighter line height in the individual messages in the message pane
   * Fixed button widths in the left sidebar (so the buttons are equal in width)

(imported from commit 337dc4a3d8e29945cfc8cfb9524ac76a7b038ad8)
2013-05-09 10:35:46 -04:00
Steve Howell 58c1110635 Put streams lower in typeaheads and kill off some dead code related to search
(imported from commit c7d50f44a4de6cc5049e51ef4b671cc49902c466)
2013-05-09 10:35:46 -04:00
Steve Howell 77d2d669de Fix highlighting for stream typeahead in compose box.
Recently the typeahead for streams in the compose box was modified so
that streams only matched queries when the query was a prefix to the
stream.  When that change was made, the old highlighting behavior
had been mistakenly left in place.  This commit fixes the highlighting.

(imported from commit b7ec33daba46978df58eb91306686a4f1a57c7fa)
2013-05-09 10:35:46 -04:00
Waseem Daher 328da1fa07 Assorted always-open-composebox fixes.
* Properly resize compose area when we cancel out of it.
* Re-enable clicking on 'reply' in popover.

(The issue with the latter is that clicking on "Reply" started
a reply and then bubbled up and triggered our code that canceled
a reply because you clicked out of the composebox.)

(imported from commit 25d0ea58b72d2ee246217baf3eb9cac58fc858f5)
2013-05-09 10:35:46 -04:00
Waseem Daher 9384870ffa Always-open composebox.
Really, the "correct" way to do this is to undo "scrolltheworld", and
then just have a compose div that always lives underneath the message
list div. (This will also allow us to deal much more reasonably with
the whole "Is the composebox in focus" thing.)

In the interest of prototyping something more rapidly, though, we
adopt the somewhat more hackish approach, with the understanding that
much of it will probably be simplified later.

(imported from commit e2754be155c522b6dac28e7b84c62bd2030217c8)
2013-05-09 10:35:46 -04:00
Steve Howell 96100bb42a Highlight search typeaheads for streams
(imported from commit e8725fe8c4f4985d46e48fdab5003e3e9def9bd1)
2013-05-09 10:35:46 -04:00
Steve Howell 667a47755e Highlight search typeaheads for senders
(imported from commit 80e16d47507b0c23103aa1e574961cc5e29cac6e)
2013-05-09 10:35:46 -04:00
Steve Howell e47b98df38 Highlight search typeaheads for private messages
(imported from commit e3c5525255c70e507f33f24fc305d4a268c87e42)
2013-05-09 10:35:46 -04:00
Tim Abbott d45527864c bugdown: Include server name in markdown failure emails.
(imported from commit 5b090480a66ca04f7b065215c8c5da698b6c5358)
2013-05-09 10:35:45 -04:00
Steve Howell 8a8e7c22f1 Don't try to rebuild subjects for an unsubscribed stream.
(imported from commit 2fea180bd5c989cb9aac843c837200acb10ddbc6)
2013-05-09 10:35:45 -04:00
Steve Howell 64ba2864bc Match names w/prefix and turn off highlighting.
(imported from commit 4ebc4aba5d05173ad6b8176d47fdf062e23c1441)
2013-05-09 10:35:45 -04:00
Steve Howell 0f75cbd30f Match any word in stream for typeahead queries
(imported from commit bd6ee91ae880c7fb3dd118cd2f024820ced73466)
2013-05-09 10:35:45 -04:00
Steve Howell de8106d923 Refine search (sort people by # of pms, limit streams to prefix-only).
Also minor configuration changes and no need to clone list.

(imported from commit 9f9c7259ed479fc29cd66fe260d9d81017e5f48f)
2013-05-09 10:35:45 -04:00
Steve Howell 0164d9f757 Search typeahead has more suggestions and sorting.
(imported from commit 5386be57e7f0b820e1a9cbd166c9c43e5e72215b)
2013-05-09 10:35:45 -04:00
Tim Abbott 77aad147f1 Enable historical messages for all future realms.
(imported from commit b8f13a996ae00644f808297a87c6a92c89aaf4ab)
2013-05-07 11:47:57 -04:00
Waseem Daher a556265cf8 Only highlight "Humbug" when we actually mouse over it.
This demonstrates the important box-model difference between
margin and padding :)

(imported from commit 068414f9066da8175f07a9517a02e93cd2deebc7)
2013-05-07 11:47:13 -04:00
Zev Benjamin 26ea7e0897 Build the recent subjects list on demand
We previously kept the lists in the DOM for all streams and updated
them all when new messages arrived.  This was very expensive for
large numbers of streams, so we now just build the subject lists on
demand.

(imported from commit 937ad4322ce2014200aeae8645f79875f6af576e)
2013-05-07 11:47:13 -04:00
Zev Benjamin 8702d37028 Do narrow-related side-bar manipulation in event handlers
This commit also fixes a bug where "starred messages" wouldn't get
bolded when you narrowed to starred messages.  However, it also
introduces a regression where subjects aren't highlighted correctly
on load to a narrow which will be fixed shortly.

(imported from commit 411575d92762e41d04c1baf126c0ab1dfb4225a5)
2013-05-07 11:47:13 -04:00
Zev Benjamin ca95167da1 Move several initialization calls to after we've set up event handlers.
This will matter shortly as hashchange.initialize can call
narrow.activate(), which fires an event handler.

Really, I have no idea why we have these initialize() methods anyway
and we don't just do initialization on document.ready.

(imported from commit 3a6a80e1426b03439b95cae3f142a4b1c43125e9)
2013-05-07 11:47:13 -04:00
Zev Benjamin 43829b4288 Fire an event on narrow and unnarrow
(imported from commit f3e5e5435ea425c4d3887b43824e53a1c817fa6f)
2013-05-07 11:47:13 -04:00
Zev Benjamin beba14a86c Only call add_message_metadata once when receiving messages in a narrowed view
We memoize add_message_metadata by checking if the message is already
in the all_msg_list.  Therefore, we need to add messages to that
message list before we add it to the narrowed_msg_list.

(imported from commit 4346179376ef6f982162c02c6152a0d294bfb2c0)
2013-05-07 11:47:12 -04:00
Zev Benjamin 96129b1cf0 Speed up string comparison for sorting
The String.localeCompare function is really slow, at least partially
because it creates a locale-aware collator object each time.  So now,
when we can, we create and cache a locale-aware collator object.
However, this is not supported on most browsers, so we fall back to a
non-locale-aware comparison.  This is not ideal, but for now we are
mostly working with English-speaking customers.

(imported from commit 51aa02e3b9fe4a0ef0cb084874fe26e91c57f65e)
2013-05-07 11:47:12 -04:00
Leo Franchi 5e70253bf9 Only remove a filter when unsubscribing if subscribed
Addionally, print out a blueslip error instead of dying
if a stream id is accessed when there is no stream to get

(imported from commit 0d6466ca79312a4fb9a235f313303ac5246afb35)
2013-05-06 13:52:35 -04:00
Leo Franchi 57d5bcf1f8 Only update unread counts if you're subscribed to that stream
(imported from commit 100047ad6ca1818cdf1c19dfafd0b2e06d95e380)
2013-05-06 13:52:30 -04:00
Leo Franchi 0da2f51ddb Calculate presence information with delta between server timestamp
(imported from commit b4b367ed620878e6d01287cb1aaae0730c7475c8)
2013-05-06 13:52:23 -04:00
Jessica McKellar f6c4e6a22e Add a first pass at a features page.
(imported from commit a1ffd7b010085f842f8fc6e8af15358e913beb41)
2013-05-05 22:46:30 -04:00
Tim Abbott f06fdbb7a8 Fix Zephyr mirroring loop detection with new events API.
(imported from commit 79454bb26851b2f4a61d6cdf04350386671618a5)
2013-05-03 18:20:34 -04:00
Luke Faraone 6bc2d21b87 Make audible notifications a tunable.
This decouples from Chrome notifications, which gives us cross-platform
support in at least modern browsers.

We log this action so its replayable in our message logs.

This implements the model change indicated by the previous schema commit.

(imported from commit b21213cdde54f43670bbb0bf1f607147fc732b38)
2013-05-03 15:08:49 -07:00
Luke Faraone 69c9ed6c0d [schema] Add a enable_sounds field to UserProfile
(imported from commit 293808410a66cc840d9e6092db6976372a2b2459)
2013-05-03 15:08:49 -07:00
Luke Faraone 734b91c8d4 Play a sound when you recieve a desktop notification in the app.
We test if the user supports sound in their browser, then determine which
sort of sound their browser supports.

When, whenever we show a desktop notification we also play a sound.

(imported from commit dae41e70a6e4f6ed60ffedaac546d77baee52675)
2013-05-03 15:08:48 -07:00
Steve Howell 6304a30257 Handle multiple casings of same subject in subject list. (#1227)
(imported from commit f1f675cd96a6d26c5b039d8f41c9373bc365cb45)
2013-05-03 14:28:23 -04:00
Tim Abbott 54ae352e83 Fix narrowing to a stream/subject pair when not subscribed.
(imported from commit 0ea2893dc6026df5316dd927f7d740c74dc2c02e)
2013-05-03 11:40:06 -04:00
Luke Faraone b77ca230ab Fix JS lint error in composebox_typeahead.js
(imported from commit a0008e895e439b37658d31449587d20763e52aa7)
2013-05-03 08:20:45 -07:00
Leo Franchi 9362731e4d Mark messages behind pointer as read when moving pointer
(imported from commit 7e1d3d7124aec239a8fdd24da4aab682a41ff435)
2013-05-03 11:14:47 -04:00
Leo Franchi 33eef54c5b Re-send unread message counts if XHR fails
(imported from commit 062e76ca7db177540ed4e9ab0d331b9ecc2ee5f1)
2013-05-03 11:14:46 -04:00
Leo Franchi 2e00749b31 Also collect 10min active user stats
(imported from commit 63b0dbd94022e6d35ceb28952c30a4c25be6a88f)
2013-05-03 10:36:00 -04:00
Steve Howell fa5a7fbf0b Force prefix match on stream typeahead in compose box.
(Also allow up to 3 matches.)

(imported from commit 88d707cedc88ab55685e6b03dbd7c87caa05057d)
2013-05-03 10:12:43 -04:00
Jeff Arnold 105fdc076b Reserve height=100 for images so that collapsing takes them into account
Addresses #1204

(imported from commit 9a1f4b13ab14c4b8fc4eab50178b9c28f477fcd7)
2013-05-02 18:53:34 -04:00
Jeff Arnold b33405d32c Make the terms page printable
(imported from commit 76e2e7e8f136c9cc1e5cc2ec6ee37bdf7212837b)
2013-05-02 16:10:03 -04:00
Tim Abbott 6b85eaf16f Only mark messages in all_msg_list as unread.
(imported from commit 8c76ee8be918beeea44e70cba702fa45abff2297)
2013-05-02 15:35:46 -04:00
Tim Abbott f6fb62ca93 Fix narrowing to stream names that don't exist.
(imported from commit 2d0ff1ffdf32eef6df56d6e9ab00c9c6e4e7ee6a)
2013-05-02 15:35:46 -04:00
Tim Abbott 03fe751005 Clarify confusing narrow-by-stream selection of active stream.
(imported from commit a90059ed62c47ac03bca0fa306adeca3ce577d43)
2013-05-02 15:35:46 -04:00
Tim Abbott 525a20de2e parse_narrow: Handle malformed narrow URLs gracefully.
Since they can't be parsed, probably the best thing to do is to send
the user to the home tab; we could add in showing an error message but
then we'd need a way to clear the error message -- better to just have
this work.

(imported from commit 67c0475ff06eb0431621eef60b9c50287a158232)
2013-05-02 15:35:46 -04:00
Tim Abbott 2c9c5f6093 Clean up code duplication when calling add_narrow_filter.
(imported from commit da397f015427e7c5b01fa1e89ba119ff87cda168)
2013-05-02 15:35:46 -04:00
Tim Abbott e7d04ff60f Fix encoding of narrows with '.' in the URIs.
(imported from commit fa624c0f810817b8093aeb4e3fb4329cad4cf9fb)
2013-05-02 15:35:45 -04:00
Tim Abbott bfaa698ea6 get_old_messages: Don't fetch messages from database unless we have to.
Previously, we were fetching Message.objects.select_related() from the
database, even if we actually ended up fetching the message dicts from
memcached and thus not actually using them.  Especially in the cached
case, this resulted in a lot of overhead where the Django ORM put
together Message objects with lots of data in them that were never
used.  This commit switches the model to only fetch the full message
objects from the database for those messages which are not found in
the memcached caches.

Here are the timings for get_old_messages before this patch was applied:

(cached)
127ms (db: 42ms/2q) /json/get_old_messages (starnine@mit.edu via website)
385ms (db: 105ms/1q) /json/get_old_messages (starnine@mit.edu via website)

(uncached)
315ms (mem: 6ms/41) (db: 90ms/22q) /json/get_old_messages (starnine@mit.edu via website)
507ms (db: 94ms/14q) /json/get_old_messages (starnine@mit.edu via website)

Here are the timings for get_old_messages after this patch was applied:

(cached)
 80ms (db: 9ms/2q) /json/get_old_messages (starnine@mit.edu via website)
133ms (db: 4ms/1q) /json/get_old_messages (starnine@mit.edu via website)

(uncached)
230ms (mem: 9ms/41) (db: 48ms/23q) /json/get_old_messages (starnine@mit.edu via website)
385ms (db: 55ms/15q) /json/get_old_messages (starnine@mit.edu via website)

(imported from commit c4748513392a906393314aa7cd41d98a69865411)
2013-05-02 15:32:23 -04:00
Tim Abbott 4f6c46c090 Add sender's domain to display recipient structures.
(imported from commit 6ed2d869b36ad03eaa17eded57e7e46cf638dfd7)
2013-05-02 15:31:10 -04:00
Steve Howell c02540bfa6 Show lock icon in compose box for invite-only streams (811)
(imported from commit c563fa605d850f39c3b3f0ae9530700f1d1b6e73)
2013-05-02 13:39:34 -04:00
Zev Benjamin ea55ed5ebb Use .attr() instead of .data() to access stream and subject names from the sidebar
The .data() method tries to coerce the value of the attribute into a
Javascript type, which is not what we want when the stream name looks
like a number or some other Javascript type.

(imported from commit a5f639d2ef98435cec6beacf3837fc185474a955)
2013-05-02 12:50:24 -04:00
Zev Benjamin 3b6713d545 Improve message_range's running time
We also now check that start != -1, as that is probably a mistake.

(imported from commit 7065ed40150fda7b373d137129c0944a8fa03fc1)
2013-05-02 12:03:18 -04:00
Zev Benjamin bea819e039 Don't mark all loaded messages before the pointer as read on page load
On page load, the scroll_finished function was being called and
scroll_start_message was -1.  This caused us to mark all messages
that we loaded through the messages initially visible as read.  This
was particularly problematic because message_range iterates over all
message ids between its two arguments.

(imported from commit d93209d466797939cc9dbdbe76d25a5b20195bd2)
2013-05-02 12:03:18 -04:00
Zev Benjamin 7c672e85a9 Look up stream sidebar elements by id
Previously we were doing quadratic work in the number of streams
because we had to iterate over all <li> elements every time we added
a new one.

(imported from commit 60cb97f77d161e9d8c3072157fa9c57c58f7af52)
2013-05-02 12:03:18 -04:00
Zev Benjamin 16cd33f756 Only pick a new color if the new subscription doesn't already have one
(imported from commit 7ccba8ab3ee52e81a8e553a43dce4c8bfc17f1fc)
2013-05-02 12:03:18 -04:00
Zev Benjamin 419f49c8be Cache the available colors instead of recomputing it each time we pick a new one
Since we pick a new color every time we add a new subscription and
recomputing the available colors was linear in the number of
subscriptions, we were doing quadratic work on page load.

(imported from commit 647ff3cb82f405755711da47701f005e7bc0023e)
2013-05-02 12:03:18 -04:00
Zev Benjamin 54bd4516ed Only update recent subjects after all messages in a batch have been processed
We were previously doing this on every message.  Because
update_recent_subjects is linear in the number of streams in the
sidebar, this became very slow when we enabled the streams sidebar
for the MIT realm.

(imported from commit 95cd71d83bbcc08cc6c5c79ca567b5d6b9b17173)
2013-05-02 12:03:18 -04:00
Zev Benjamin a640da16df Sort the streams side-bar only after all streams have been added
We were previously calling sort_narrow_list after each stream was was
added.  Because it is linear in the current length of the sidebar
list, we were doing quadratic work on page load.  When we enabled the
streams sidebar on the MIT realm, this became problematic because of
the number of subscriptions Zephyr users have.

(imported from commit d60ddc638f0a81fbce08eecd6671e9ea6ca38515)
2013-05-02 12:03:18 -04:00
Zev Benjamin ec3c0fffb7 Remove no longer true comment
(imported from commit 84c4e6bfbf16460efe5703e1a4213383a27fbe79)
2013-05-02 12:03:17 -04:00
Waseem Daher 5887d9c576 Condense messages in a fail-safe way.
Messages are now explicitly condensed by our JS, which means that if
we run into some bug where our JS doesn't run, you still see the whole
message (rather than getting a clipped message).

(As of this commit, this can happen when you, e.g. are on the
Settings page and someone sends you a message.)

(imported from commit f3bec97800ea1852c80203e73552ee545fcc7e8a)
2013-05-02 09:52:26 -04:00
Jessica McKellar 4a01b7a95c realm_stats: report on features like starred messages and home view.
(imported from commit ee026ee48f9038c0cb9456bd3b0e1a89460b4968)
2013-05-01 21:59:29 -04:00
Jessica McKellar cc7fddedbc Add a new stream_stats management command.
(imported from commit 7a8e94459ac7402f27ae45598eb113496672b9bf)
2013-05-01 21:59:29 -04:00
Jessica McKellar f4dd598330 realm_stats: since we now auto-color, remove colorized streams reporting.
(imported from commit a051f3077dd0746bfca7944c2369fd90c1db80f7)
2013-05-01 21:59:28 -04:00
Jessica McKellar ce218dec48 realm_stats: only count active (not deactivated) users.
(imported from commit e35838d5bb6260efb33290ab63fdbd10a3d9d879)
2013-05-01 21:59:28 -04:00
Jessica McKellar 5fb65712af realm_stats: only include streams with subscribers (ie not tutorial streams).
(imported from commit 870ce51df191611569bdd7c2509cc679748ea201)
2013-05-01 21:59:28 -04:00
Zev Benjamin e1c8d2f50f Return match_subject and match_content from messages_in_narrow()
This fixes a bug where if you were narrowed to a search and received
a new message that belonged in that search, the message would appear
to have an empty subject and content.

(imported from commit fe1dbf584d3659d57c5b70c7eb45cb22bbc9732f)
2013-05-01 21:52:04 -04:00
Jessica McKellar cc321636b1 Add a management command to bankrupt users.
(imported from commit 58fbd08fc31a69c9ee7fb73b9302d44eb87db1fa)
2013-05-01 21:16:40 -04:00
Steve Howell 7dd64eb157 Handle github sending empty string for the stream.
(imported from commit 2fd85db1828be44ef63920d5df347b5f85acb573)
2013-05-01 17:39:54 -04:00
Leo Franchi d0b8a2fd21 Mark messages as read when using the End key
(imported from commit b2495cb27b1362d037e786db7f108540f2ce655b)
2013-05-01 17:26:19 -04:00
Leo Franchi de44f08772 Fix typo and send proper dbtime to statsd
(imported from commit 87e982de71e005ed110200b2f9afeec488dcfc51)
2013-05-01 17:24:38 -04:00
Steve Howell be1e18f864 Support branches whitelist for github
(imported from commit 066bb8ee028778cb39b43afc9737fd2117c91928)
2013-05-01 17:19:53 -04:00
Steve Howell b645c67994 Use stream from github webhook call (if supplied).
(imported from commit 4f57c4dec8ab5e833583a2b5912a92e8a2bd34c0)
2013-05-01 17:19:53 -04:00
Waseem Daher 435098d001 Process message condensing in narrow.activate rather than hashchange.
Previously, we were having this problem where:
* You narrow to something
* That causes message_list.js:process_collapsing to run on all of the
  elements in the view, which changes some of their sizes
* That causes the pane to scroll and either push the content up or
  down, depending (since stuff on top of where you were is now a
  different size)
* That triggers keep_pointer_in_view, which moves your pointer

Moving process_collapsing into narrow.activate doesn't obviously
fix any of this, but it does seem to mitigate the issue a bit.

In particular, we (a) process it less frequently, and (b) process it
immediately after we show the narrowed view table, which seems to
reduce the raciness of the overall experience.

This does, however, introduce a regression:
* If you receive a long message when you're on
  #settings, e.g., and then go back to Home,
  the message does not properly get a [More] appended
  to it.

(imported from commit b1440d656cc7b71eca8af736f2f7b3aa7e0cca14)
2013-05-01 16:56:03 -04:00
Tim Abbott 9f0fc7c031 blueslip: Include window.location.href in error reports.
This can be useful for debugging what sort of narrow is happening in
addition to the URI decoding bug we're currently experiencing.

(imported from commit 0cb55fec4ac1afa986c747eb79236b4300c9e636)
2013-05-01 16:10:35 -04:00
Leo Franchi 5ef7c4e6db Add a management command for active user stats
(imported from commit a4227858b422c48e272700880e0c21889c7ce566)
2013-05-01 11:17:18 -04:00
Tim Abbott 57c5ea365a Enable the left sidebar for the MIT realm.
(imported from commit d69eba7fd7a95dd88892706c3d36485c71831864)
2013-04-30 18:38:13 -04:00
Tim Abbott 203fc55a7c Sort the narrow list better when there are more than 40 streams.
This shouldn't have any effect in normal realms, but for realms like
mit.edu that have large numbers of inactive streams, it will sort all
the streams that have had a recent message at the top (aka those that
aren't effectively inactive).

(imported from commit 027ce258d04b6fd58705e49f769dec7e0639bb38)
2013-04-30 18:38:13 -04:00
Steve Howell 30f825ebfe Handle bad api_key for jira integration properly.
(imported from commit e6063431e81434faa5f32ac8f91a08f86bd46997)
2013-04-30 18:26:34 -04:00
Steve Howell fbad47ec28 api_key is url parm for jira webhook
(imported from commit 24624a9fcd7e6fdc15d23c2874a04e1465c3f3cf)
2013-04-30 18:14:11 -04:00
Tim Abbott f36d51edeb Update Zephyr Mirroring liveness check for new REST API.
(imported from commit d968fde21bd90510ea7bb7f85ecb9b97b41689f7)
2013-04-30 14:28:38 -04:00
Steve Howell 4450bbcbf8 @csrf_exempt decorator for api_jira_webhook
(imported from commit 6c4ef97b2312b6721bd50605efdf51f7affb514c)
2013-04-30 13:24:47 -04:00
Steve Howell 23f0d0bcb7 support stream parameter for jira to override default stream
(imported from commit b3be7b837f326968f9742c25ba04bdaaf6476b75)
2013-04-30 13:24:47 -04:00
Tim Abbott 5b51705451 Fix logging of requesting user with REST API.
Fixes #1155.

(imported from commit b5becb7418ce9577a6bbaa20dcb68a02f1928b9f)
2013-04-30 11:54:16 -04:00
Zev Benjamin c08a86aeb9 Use server-highlighted subject and content when narrowed to a search
(imported from commit 0579193da040db77f9c7937d3714cb9ffeaf7ed8)
2013-04-30 11:40:27 -04:00
Zev Benjamin fca8f84c14 [schema] Return highlighted subject and content from get_old_messages() when doing a search
We HTML-escape the subject in Postgres to avoid a server round-trip.
Unlike the rendered_content, which is already escaped and cached on
zephyr_message, we normally escape subjects client-side.  Escaping in
Django would require fetching the messages that match the query,
escaping the subjects, and then making a second query to Postgres to
insert the markup.  We could instead fetch the messages with subjects
marked up using non-HTML (some unique string) that is later converted
into the correct markup either in Django or client-side, but then the
escaping problem would just be with some random string instead of
HTML.  Since the function is pretty simple, doing the escaping in
Postgres itself is the least painful option.

(imported from commit 004931d8e496697c18650aee97b1a74c55a04cb2)
2013-04-30 11:40:27 -04:00
Tim Abbott 1986b65c6a Enable historical messages for customer33.invalid.
(imported from commit ed95813f20ba29b425be4d90d6a54beb22ec81ad)
2013-04-30 11:26:20 -04:00
Tim Abbott e3bb1bc8ec bugdown: Fix tweet ID extraction from twitter urls.
(imported from commit 88b9882527a5317bf30bcc5f0d1255e819ea149c)
2013-04-30 10:43:17 -04:00
Zev Benjamin 1c761f7ae3 [schema] [manual] Rebuild search_tsvector and change its update trigger
In addition to changing the trigger that updates
zephyr_message.search_tsvector to use our new text search
configuration, it also now builds the tsvector on rendered_content
instead of content and fires on update of only the subject or
rendered_content columns.

This migration is expected to take a long time.  The
checkpoint_segments parameter in postgresql.conf should be
temporarily raised (probably to 32) while it is running.

(imported from commit 4535438bb33ce1db2a74ecbe91efc52afdb568f1)
2013-04-29 13:58:20 -04:00
Zev Benjamin 8f17f99de2 Construct ts_queries using the new Humbug search configuration
(imported from commit 813ae86e9ea5f8af3ec2abd7d506cd707e699cdf)
2013-04-27 20:06:26 -04:00
Zev Benjamin 2aadf6fc6e [schema] [manual] Create a Postgres text search configuration for use with Humbug
Text search was not that great partially because Postgres wasn't
using a ispell dictionary (Postgres term) before.  We now pull in
Hunspell and use its dictionary and affix rules.

It is Ok to run with this new configuration before updating our full
text column and index that will be coming in the next few commits.

Manual steps for deploy:
1) On both postgres0 and postgres1 (both before moving on to step 2),
   install the hunspell-en-us package
2) On staging, run migration 0022
3) On both postgres0 and postgres1, copy the appropriate postgresql.conf
   file over
4) On both postgres0 and postgres1, run `pg_ctlcluster 9.1 main reload`

(imported from commit 706bf0f6ecc46c712cea10b73c34fd9d1dfd4767)
2013-04-27 20:06:26 -04:00
Zev Benjamin 8b3e800bf5 Use the filter on MessageLists when adding messages
(imported from commit c9070e5413352aef612b4763c35a4c72f4ecb852)
2013-04-26 17:45:25 -04:00
Zev Benjamin 8474675076 Attach Filters to MessageLists
This should allow us to stop special-casing the narrowed message list
as much.

(imported from commit 1eb7216fbd8aa16b796c239a189d6ce0008344f9)
2013-04-26 17:45:25 -04:00
Zev Benjamin 4f4e982ed1 Don't do narrows with search operators on the client
(imported from commit 98e6a06f5c10a384e6295b3281d23a061ecdecab)
2013-04-26 17:45:25 -04:00
Zev Benjamin aeea631bd2 Add JSON query for checking which of a set of messages are in a narrow
(imported from commit b1320cf0e1404d6b0f3dbf3a5b32b29287c698d7)
2013-04-26 17:45:22 -04:00
Zev Benjamin 3b8b8a7d4e Fix 'in' search operator
(imported from commit b42c9ef8d00c92ea117f124b933791478098c5b7)
2013-04-26 17:39:08 -04:00
Zev Benjamin c5c0a2ab45 Canonicalize Filter operators early
(imported from commit c251fab94970c127708c1b40b25e57d4afeb7ce9)
2013-04-26 17:39:08 -04:00
Zev Benjamin 0c15db4aff First steps towards making narrow.js more object-oriented
There's still a lot to do here.  For example, the external code
should probably go through the new Filter object directly instead of
indirectly through the narrow module.

(imported from commit 22dcd31cdebd51453f1658af52a4432b2fe7a4cb)
2013-04-26 17:39:08 -04:00
Zev Benjamin 6cdc3f67df Only fetch an extra message in get_old_messages if a narrow isn't specified
In the case where we're getting old messages for a narrowed view, the
anchor message id might not actually be in the result set so there's
no reason to fetch an extra message.

(imported from commit e610d1f2cb95be3ff9fce6dc95e40c560bc5bf84)
2013-04-26 17:39:08 -04:00
Allen Rabinovich 6970f2deee Corrected the removal code for temporary copy div
(imported from commit 9e3963a3ce4e7039464ab3c8de5939361d7802e5)
2013-04-25 15:11:56 -07:00
Jeff Arnold 88348a8dc7 Prevent XSS vulnerability by avoiding treating message text as HTML
Addresses #1205

(imported from commit 229c65ae48d509bf3b71ed7cbfcc1fbeb60d14c5)
2013-04-25 17:33:04 -04:00
Allen Rabinovich 417299bdfd Modified the "notvisible" CSS class to account for edge cases.
In particular, I added absolute positioning and hidden overflow,
which ensures that if an element has a persistent min-width
(like a file input field apparently does), it doesn't affect its
parent.

(imported from commit 72e7a5bee2775fb6f229899ba849292eee76aa4a)
2013-04-25 14:30:56 -07:00
Tim Abbott 8760c1c7be get_display_recipient: Use an in-memory cache as well as memcached.
(imported from commit 825cb77b29cd635add252e4b9497feeb7ed7e177)
2013-04-25 17:02:20 -04:00
Tim Abbott 7c001822f2 Use bulk requests for updating memcached in get_old_messages.
Otherwise we end up doing 1000 requests to memcached, which can be
quite expensive.

(imported from commit be247f63b5fb88c6f4a45326261b66ea67fe1028)
2013-04-25 14:43:37 -04:00
Zev Benjamin 3e1ec5d7c9 Increase size of initial message cache fetch and exclude tabbott/extra's messages
(imported from commit 59544aa3adfb05f50ca69e56f37f57944dfa0b81)
2013-04-25 13:33:51 -04:00
Zev Benjamin a1634b12d3 Increase efficiency of initial message cache query
In repeated trials, the initial data fetch used to take about 1100ms.
In practice, it was often taking >2000ms, probably due to caching
effects.  This commit cuts the time down to about 300ms in repeated
trials.

Note that the semantics are changed slightly in that we may no longer
get exactly 25000 messages.  However, holes in the message_id
sequence are currently very rare or non-existent so this shouldn't be
a problem and we don't care about the exact number of messages
anyway.

I believe the problem was that the query planner was unable to
effectively use the LIMIT clause to figure out that only a small
subset of zephyr_message was going to be needed.  Thus, it planned
for operating on the entire table and decided it could not use a more
efficient plan because work_mem, although large, would not be large
enough to execute the query over all of zephyr_message.

The original query was:

SELECT "zephyr_message"."id", "zephyr_message"."sender_id", "zephyr_message"."recipient_id", "zephyr_message"."subject", "zephyr_message"."content", "zephyr_message"."rendered_content", "zephyr_message"."rendered_content_version", "zephyr_message"."pub_date", "zephyr_message"."sending_client_id", "zephyr_userprofile"."id", "zephyr_userprofile"."password", "zephyr_userprofile"."last_login", "zephyr_userprofile"."email", "zephyr_userprofile"."is_staff", "zephyr_userprofile"."is_active", "zephyr_userprofile"."date_joined", "zephyr_userprofile"."full_name", "zephyr_userprofile"."short_name", "zephyr_userprofile"."pointer", "zephyr_userprofile"."last_pointer_updater", "zephyr_userprofile"."realm_id", "zephyr_userprofile"."api_key", "zephyr_userprofile"."enable_desktop_notifications", "zephyr_userprofile"."enter_sends", "zephyr_userprofile"."tutorial_status", "zephyr_realm"."id", "zephyr_realm"."domain", "zephyr_realm"."restricted_to_domain", "zephyr_recipient"."id", "zephyr_recipient"."type_id", "zephyr_recipient"."type", "zephyr_client"."id", "zephyr_client"."name" FROM "zephyr_message" INNER JOIN "zephyr_userprofile" ON ( "zephyr_message"."sender_id" = "zephyr_userprofile"."id" ) INNER JOIN "zephyr_realm" ON ( "zephyr_userprofile"."realm_id" = "zephyr_realm"."id" ) INNER JOIN "zephyr_recipient" ON ( "zephyr_message"."recipient_id" = "zephyr_recipient"."id" ) INNER JOIN "zephyr_client" ON ( "zephyr_message"."sending_client_id" = "zephyr_client"."id" ) ORDER BY "zephyr_message"."id" DESC LIMIT 25000;

with query plan:
 Limit  (cost=0.00..27120.95 rows=25000 width=362) (actual time=0.051..1121.282 rows=25000 loops=1)
   ->  Nested Loop  (cost=0.00..5330872.99 rows=4913981 width=362) (actual time=0.048..1081.014 rows=25000 loops=1)
         ->  Nested Loop  (cost=0.00..3932643.31 rows=4913981 width=344) (actual time=0.042..926.398 rows=25000 loops=1)
               ->  Nested Loop  (cost=0.00..2550275.29 rows=4913981 width=334) (actual time=0.035..752.524 rows=25000 loops=1)
                     Join Filter: (zephyr_message.sending_client_id = zephyr_client.id)
                     ->  Nested Loop  (cost=0.00..1739467.29 rows=4913981 width=320) (actual time=0.024..217.348 rows=25000 loops=1)
                           ->  Index Scan Backward using zephyr_message_pkey on zephyr_message  (cost=0.00..362510.09 rows=4913981 width=156) (actual time=0.014..42.097 rows=25000 loops=1)
                           ->  Index Scan using zephyr_userprofile_pkey on zephyr_userprofile  (cost=0.00..0.27 rows=1 width=164) (actual time=0.003..0.004 rows=1 loops=25000)
                                 Index Cond: (id = zephyr_message.sender_id)
                     ->  Materialize  (cost=0.00..1.17 rows=11 width=14) (actual time=0.001..0.010 rows=11 loops=25000)
                           ->  Seq Scan on zephyr_client  (cost=0.00..1.11 rows=11 width=14) (actual time=0.002..0.010 rows=11 loops=1)
               ->  Index Scan using zephyr_recipient_pkey on zephyr_recipient  (cost=0.00..0.27 rows=1 width=10) (actual time=0.002..0.003 rows=1 loops=25000)
                     Index Cond: (id = zephyr_message.recipient_id)
         ->  Index Scan using zephyr_realm_pkey on zephyr_realm  (cost=0.00..0.27 rows=1 width=18) (actual time=0.002..0.003 rows=1 loops=25000)
               Index Cond: (id = zephyr_userprofile.realm_id)
 Total runtime: 1141.408 ms

In the new code, we do two queries:

SELECT "zephyr_message"."id" FROM "zephyr_message" ORDER BY "zephyr_message"."id" DESC LIMIT 1

followed by:

SELECT "zephyr_message"."id", "zephyr_message"."sender_id", "zephyr_message"."recipient_id", "zephyr_message"."subject", "zephyr_message"."content", "zephyr_message"."rendered_content", "zephyr_message"."rendered_content_version", "zephyr_message"."pub_date", "zephyr_message"."sending_client_id", "zephyr_userprofile"."id", "zephyr_userprofile"."password", "zephyr_userprofile"."last_login", "zephyr_userprofile"."email", "zephyr_userprofile"."is_staff", "zephyr_userprofile"."is_active", "zephyr_userprofile"."date_joined", "zephyr_userprofile"."full_name", "zephyr_userprofile"."short_name", "zephyr_userprofile"."pointer", "zephyr_userprofile"."last_pointer_updater", "zephyr_userprofile"."realm_id", "zephyr_userprofile"."api_key", "zephyr_userprofile"."enable_desktop_notifications", "zephyr_userprofile"."enter_sends", "zephyr_userprofile"."tutorial_status", "zephyr_realm"."id", "zephyr_realm"."domain", "zephyr_realm"."restricted_to_domain", "zephyr_recipient"."id", "zephyr_recipient"."type_id", "zephyr_recipient"."type", "zephyr_client"."id", "zephyr_client"."name" FROM "zephyr_message" INNER JOIN "zephyr_userprofile" ON ( "zephyr_message"."sender_id" = "zephyr_userprofile"."id" ) INNER JOIN "zephyr_realm" ON ( "zephyr_userprofile"."realm_id" = "zephyr_realm"."id" ) INNER JOIN "zephyr_recipient" ON ( "zephyr_message"."recipient_id" = "zephyr_recipient"."id" ) INNER JOIN "zephyr_client" ON ( "zephyr_message"."sending_client_id" = "zephyr_client"."id" ) WHERE "zephyr_message"."id" > 4941883

with the message id filled in as the result of the first query.  The
new query differs from the original only in that its ORDER BY and
LIMIT clauses are replaced by a WHERE clause.  The second query has
query plan:

 Hash Join  (cost=709.30..28048.18 rows=20544 width=365) (actual time=41.678..279.261 rows=25041 loops=1)
   Hash Cond: (zephyr_message.recipient_id = zephyr_recipient.id)
   ->  Hash Join  (cost=102.98..27056.66 rows=20544 width=355) (actual time=3.686..190.730 rows=25041 loops=1)
         Hash Cond: (zephyr_message.sending_client_id = zephyr_client.id)
         ->  Hash Join  (cost=101.73..26772.94 rows=20544 width=341) (actual time=3.649..143.695 rows=25041 loops=1)
               Hash Cond: (zephyr_userprofile.realm_id = zephyr_realm.id)
               ->  Hash Join  (cost=99.99..26488.71 rows=20544 width=323) (actual time=3.578..96.746 rows=25041 loops=1)
                     Hash Cond: (zephyr_message.sender_id = zephyr_userprofile.id)
                     ->  Index Scan using zephyr_message_pkey on zephyr_message  (cost=0.00..26106.24 rows=20544 width=159) (actual time=0.017..41.980 rows=25041 loops=1)
                           Index Cond: (id > 4941883)
                     ->  Hash  (cost=83.33..83.33 rows=1333 width=164) (actual time=3.548..3.548 rows=1333 loops=1)
                           Buckets: 1024  Batches: 1  Memory Usage: 275kB
                           ->  Seq Scan on zephyr_userprofile  (cost=0.00..83.33 rows=1333 width=164) (actual time=0.006..1.646 rows=1333 loops=1)
               ->  Hash  (cost=1.33..1.33 rows=33 width=18) (actual time=0.064..0.064 rows=33 loops=1)
                     Buckets: 1024  Batches: 1  Memory Usage: 2kB
                     ->  Seq Scan on zephyr_realm  (cost=0.00..1.33 rows=33 width=18) (actual time=0.003..0.033 rows=33 loops=1)
         ->  Hash  (cost=1.11..1.11 rows=11 width=14) (actual time=0.027..0.027 rows=11 loops=1)
               Buckets: 1024  Batches: 1  Memory Usage: 1kB
               ->  Seq Scan on zephyr_client  (cost=0.00..1.11 rows=11 width=14) (actual time=0.003..0.013 rows=11 loops=1)
   ->  Hash  (cost=335.03..335.03 rows=21703 width=10) (actual time=37.974..37.974 rows=21761 loops=1)
         Buckets: 4096  Batches: 1  Memory Usage: 893kB
         ->  Seq Scan on zephyr_recipient  (cost=0.00..335.03 rows=21703 width=10) (actual time=0.004..18.443 rows=21761 loops=1)
 Total runtime: 299.300 ms

(imported from commit b2a70cccc47be7970df407c6be00eccd2e8be82a)
2013-04-25 13:25:15 -04:00
Waseem Daher ccc5363f3c Make text-underneath-compose-box all nicely line up together.
(imported from commit d4676d95ccad18f65a29447647581b41d53dfa9a)
2013-04-24 17:49:45 -04:00
Luke Faraone 3bb3bbb1ee Redirect to /accounts/register relative to the current domain.
This ensures that we don't always send you to hhq.c when you use OpenID.

(imported from commit ab4c4cfa201740fbddcaa2e51bc15bfe977e221d)
2013-04-24 14:20:07 -07:00
Michael McCanna ce1389dc35 Remove double subscription when "creating" stream
When you create a stream that you'd previously created (then unsubscribed from),
it was possible to end up in the subscribers list twice. Once came from loading
the subscribers list from the backend, and once came from a bit of mark_subscribed
logic that only gets called if you've subscribed to that stream at least once before
in the current session.

resolves trac #1196

(imported from commit e47ff139a9c25b1b8689ea6795dfad96ae8d2591)
2013-04-24 16:21:14 -04:00
Waseem Daher bd702b777f Truncate long names in the presence list, rather than scrolling.
(imported from commit 6ba1487d5e2d2ac1435248fec5f161deec685180)
2013-04-24 15:53:54 -04:00
Allen Rabinovich f863a9b567 Added a check to filedrop plugin to check whether pasted content has strings.
If the pasted content has strings, we don't upload included files and instead
allow the default behavior to take place. This deals with a quirky behavior of
pastes from MS Word, which in addition to the formatted string content also
includes a thumbnail of it. Images still paste as usual.

(imported from commit 60c4f8dd90ac2e8e38940fb302cc9d1ebeecfdf3)
2013-04-24 12:33:16 -07:00
Luke Faraone b9fe130107 Add a comment to emphasise why we shouldn't break the HomepageForm.
(imported from commit 1b4d59e0c868e8350d2a9eb46cdfdaed1f34e69f)
2013-04-24 12:03:42 -07:00
Luke Faraone c48ff1784c Implement OpenID signups.
This allows users on signup-eligible domains to sign up for Humbug using
Google Apps.

As part of this, we wrap the openid done view in our own code in order to
handle the "Unknown user" error. Therein, we create a PreregistrationUser
and then shunt the user through the rest of the confirmation process, pre-
filling in their name.

(imported from commit 066d9a1021384a6da2662352e62a701451bd6f44)
2013-04-24 12:03:42 -07:00
Allen Rabinovich f8397e42c2 Adding an "Attach files" button to the message composition pane.
Changes include:
   * New markup for the button in compose.html
   * A hidden file input field in compose.html
   * Added reference to the file input field in filedrop
     initialization in compose.js
   * A feature test and a click event binding for
     the "Attach files" button in ui.js
   * New paperclip icon reference in fonts.css
   * New general hidden display classes in zephyr.css
   * New composition pane button classes in zephyr.css

Fixes to the "Attach files" button commit e673bda...

Changes include:
   * Fixed the feature test for (new XMLHttpRequest).upload so
     it works in Firefox.
   * Renamed .button to .message-control-button
   * Removed stray newlines

(imported from commit c1f0834b74fd7120ec27db64ec380ffb3fa34633)
2013-04-24 08:59:13 -07:00
Zev Benjamin 75bbda1dad Add lower message id bound when marking messages as read for the mobile unread count hack
Having a message ID range significantly improves the query
performance because the number of messages Postgres has to consider
is much smaller.

(imported from commit 9b007457712f1c1502d526abea1b6fd742bd911d)
2013-04-24 11:30:24 -04:00
Tim Abbott 102988e430 Refill the Session cache after restarting the server.
The fact that we were dumping this cache and not refilling it seems to
be one of the causes of Tornado restarts being a lot slower on prod
than on local systems.

(imported from commit a32a759f4dfb591706ede1cce2d38f5c3704193c)
2013-04-24 10:44:56 -04:00
Tim Abbott 1c57956bf6 Avoid an unnecessary load_old_messages call on page load.
Previously, our check for whether we needed to call load_old_messages
a second time on page load to get up to the present caused us to
basically always do such a call.

(imported from commit b599041e8c0853b4c8c9ab2def6679142302523e)
2013-04-24 10:44:56 -04:00
Tim Abbott 9b8f0fab0f Retrieve message objects from memcached in a bulk request.
On my laptop, this saves about 80 milliseconds per 1000 messages
requested via get_old_messages queries.  Since we only have one
memcached process and it does not run with special priority, this
might have significant impact on load during server restarts.

(imported from commit 06ad13f32f4a6d87a0664c96297ef9843f410ac5)
2013-04-24 10:44:56 -04:00
Tim Abbott 66b3c1fbff Log time spent querying memcached in logs when larger than 5ms.
(imported from commit a4de15026d24526a446b724500d1194dce824d1a)
2013-04-24 10:44:56 -04:00
Tim Abbott 5e22778843 Use a function for stopping/restarting time logging for longpolling.
(imported from commit 11b772deaa126fcc7e7605d467022b22d9e98cb0)
2013-04-24 10:44:56 -04:00
Waseem Daher 5f28cf8e02 Restore tutorial's ability to see what you send.
The internal format of 'message' had changed, so prior to this commit,
the tutorial was receiving (a) internally inconsistent, and (b)
not-what-it-expected versions of the message.

(imported from commit 233b934e6b600bd59125d133fdf7443fd8f6bbf8)
2013-04-24 10:35:25 -04:00
Waseem Daher eab43f28f4 Properly compute the name of our tutorial stream server-side.
It's subtle, but the slice was in the wrong place and wasn't
actually truncating the stream name at all, so the client and
server disagreed about where the tutorial messages should go.

(It might be the case that we should accept the tutorial stream
name from the client directly, rather than computing it in two
places.)

(imported from commit 8273223f182e8ad36eaea1cbf75e1426fcfdfbab)
2013-04-24 10:32:30 -04:00
Waseem Daher c05c391c31 Use '===' instead of '==' to make the linter happy.
(imported from commit 6fee035ab4531d64ece6a49501f275c64e0f2265)
2013-04-24 10:32:30 -04:00
Waseem Daher 5951c7eb00 Don't send tutorial timeout message if we're no longer running.
If the system was waiting for you to reply and you replied 'exit', the
tutorial would stop -- but our thing that was waiting for you to reply
would continue waiting. It would eventually timeout and send you the
heartbroken "I didn't hear from you so I stopped waiting" message.

Chances are, you were unsubscribed so you didn't see it, but we
should still just not send it.

(imported from commit 694e442bc29b32efd59f08b4b8b5f573768aea21)
2013-04-24 10:28:33 -04:00
Jeff Arnold a251b12ec7 Don't trigger two events when our app changes the hash
(imported from commit c17a82b043c10927de149b105c6ceea5175b6f4f)
2013-04-23 16:13:26 -04:00
Jeff Arnold 0b9e509f41 If the user expanded a message, keep it expanded when we narrow
(imported from commit cbff6a8629df25b0b5b90bf6e10e9ea35e9a72ef)
2013-04-23 16:13:26 -04:00
Jeff Arnold 0c657f6fc5 Process collapsing every time we show new content
(imported from commit 21899206e9012023d3fec08f3990474fa38047e7)
2013-04-23 16:13:26 -04:00
Jeff Arnold 1cf946969d Fix name of hashchange.zephyr event in changehash
tabbott noticed this mistake during some pair programming

(imported from commit 3b5885f50929e3ed0062eef4e5f3aba0a14d52bc)
2013-04-23 16:13:26 -04:00
Tim Abbott c4e4b481bd Increase the threshhold for triggering collapsing./
(imported from commit c09b5ac43b4521bcbb2ab3feb5334fc9cf9ba1a5)
2013-04-23 16:13:26 -04:00
Jeff Arnold 3f68ff3ac3 Don't collapse messages that would only be collapsed by a small amount
(imported from commit 2cdd1ef1c731c7e6ac122f431ccf91d41d8f9234)
2013-04-23 16:13:26 -04:00
Zev Benjamin a398eff891 Center the message condenser messages with respect to the message body
Previously it was centered with respect to its enclosing div, which
looked slightly off.

(imported from commit 3878f162d3eb50ce85cae7054102095069aa60c8)
2013-04-23 16:13:26 -04:00
Jeff Arnold 70e5bd4049 Automatically condense long messages
Addresses Trac #747.

(imported from commit 4ddf2d8773023805650f04924be62da2d0fe826e)
2013-04-23 16:13:26 -04:00