By splitting up all_msg_list and home_msg_list, we can properly add/remove
streams from the home view without having to jump through hoops.
(imported from commit 92767197759f7519197dfc58be951b60fa823fbb)
If we have removed a stream from the home view, and our pointer
that we load from the server refers to a message that is no longer
visible, we don't want to error out but rather select the nearest
message to our previously selected one.
(imported from commit d212f1fba7b06836d1d916b43042991625b6f41e)
Messages are now selected on a MessageList, which triggers a
message_selected event that other parts of the code can listen for.
(imported from commit 1da9e4121425c0ac4461b41b7aea169072e1512b)
Previously we would select the first message in the block. Now, we
only do that if a message that is selected will not be in the
resulting narrowed view. If the selected message will be in the
narrowed view, we select that message once narrowed.
(imported from commit 4da5a3a0b597b58c2e028f1b29ac20ae3808a4d1)
I kind of expect this to work, and hopefully this'll help with
people getting stuck on the "Settings" page in the tutorial.
(imported from commit 1159d884dcd331bcfb74864a0176fa293e8c3714)
Most of them are just typos or funny punctuation, but this also adds
the word "above" after "You and Humbug Tutorial Bot" since otherwise
the user might try to click the text in the message body and be
confused.
(imported from commit dccaf4acd24db713acab261440f0a9d03860e0f4)
Right now the MIT realm has pretty bad UI lag with 1,200 messages loaded. We
need to fix the root problem, but this commit at least makes reloading the page
a satisfactory remedy.
(imported from commit 93d47710891cfc4db9fa00beaa5ccd10113aa1c3)
Here I have a sketchy but functional framework for dealing with
all of the async stuff that a tutorial requires, and an early
draft of what such a tutorial might look like.
I could probably go and remove the first-run message, but I'll
keep it around for now in the unlikely event that something I
haven't anticipated goes wrong in starting up the tutorial.
(imported from commit de9779a66a1b3fe790082decb324c90ec180b39b)
The idea here is: part of the onboarding tutorial is going to
be you talking to the tutorial bot and it talking to you, from
our Javascript.
The reason it's driven by Javascript is that then in principle we can
do nice stuff like making popovers appear in places to point things
out to you, whereas if we were to do it strictly server-side, doing so
would be a lot harder.
The downside to doing it in Javascript is that you don't get any of
the Markdown rendering, since that happens on the server. So instead
we add this call where you give it a message, and it responds by
having the tutorial bot send you that message.
I don't think there are any security concerns here because
(1) The bot only messages you -- so you can't use it to make someone
else think that the system is telling them to do something
(2) If there were an issue associated with having the server parse
arbitrary Markdown, you could just trigger the issue by sending
a message yourself.
(imported from commit b34f594dab6be6bcb81899278ae1cbe447404468)
At Ksplice we used /usr/bin/python because we shipped dependencies as Debian /
Red Hat packages, which would be installed against the system Python. We were
also very careful to use only Python 2.3 features so that even old system
Python would still work.
None of that is true at Humbug. We expect users to install dependencies
themselves, so it's more likely that the Python in $PATH is correct. On OS X
in particular, it's common to have five broken Python installs and there's no
expectation that /usr/bin/python is the right one.
The files which aren't marked executable are not interesting to run as scripts,
so we just remove the line there. (In general it's common to have libraries
that can also be executed, to run test cases or whatever, but that's not the
case here.)
(imported from commit 437d4aee2c6e66601ad3334eefd50749cce2eca6)
This fixes the issue where toggling show/hide on a stream in the home
view scrolls the page, because the removing and adding of rows in the
still-visible message table causes the cursor and scroll position to jump
(imported from commit 7454284a44af26379a0d2e7fce3683d98e3e4c48)
We don't typically have these (see message type, etc.) and removing
them will allow simplifying the code.
(imported from commit fbefb08ee9b08c73f32c8150a6fa1060957aa8ad)
This keeps the name to the right of the bullet even if it overflows
onto two lines.
Fixes#909.
(imported from commit 60528bb30c2d9e29687b773abc76c18369a8a068)
Previously we were doing 100 queries for new messages being sent to
the main hacker school channels; they were faster than many similar
instances because they were all done within 1 transaction, but still,
send_message_backend would be spending up to 200ms (and 148 queries)
querying the database with the previous code on prod; this new version
should do a fixed number of database queries per message.
(imported from commit 3799e63aebb6f017932ddb0fe1f6209281c0ddcf)
This commit does the actual work of formatting recent dates as weekday
names and updating the format when the date is sufficiently far in the
past.
Also, silence a bogus jslint warning.
(imported from commit 503286fbd9c1e33a81cc7d78cf8d08d5e7f78c1a)
This commit just moves time rendering logic to its own file, and does
not make any functionality changes.
(imported from commit d111d03c6abc8d9550fcf65e4f89eab8056d1ed4)
That way it is visible more consistently when arrowing through
messages (arrowing causes scroll events).
(imported from commit ba629b907e4e593032a61a10b04f00e592fe8427)
This fixes Trac #898.
Prior to this commit, we only changed the tab to #home in
compose.start; i.e. when a compose was not already in progress. This
means that if the composebox was open and we were on the settings
page, clicking one of these buttons would not work.
(imported from commit aa88a605cdcb61d5b6a1ece6292001c5f5a19c66)
This fixes Trac #912.
My guess is that 'overflow: hidden hidden' was just being interpreted
as no setting on the overflow, which is why this was broken.
Regardless, this fix seems to work in Firefox and Chrome.
(imported from commit a08ddc5db97ee1e8a65a278c0d278e823afae65d)
We've been noticing a long delay between switching to a window with unread
messages and the time that those messages actually appear. This got much worse
around the time we added Notificon.
Our hypothesis (supported by some testing) is that the work done by Notificon
in creating a <canvas>, drawing into it, serializing it to PNG, etc. is using
up some quota of background operations that would be better spent rendering
messages.
Switching to precomputed images should mitigate this problem.
Resolves#896.
May resolve#882 to our satisfaction.
(imported from commit a2d98a163486bdd35fdfb5351f96c5529ba5c7e9)
This will discard punctuation symbols in the Postgres search and also prevent
syntax errors when users try to submit queries with symbols that to_tsquery
interprets as special syntax (such as '|' and '&').
Fixes#906
(imported from commit 3e3a0d6ae3d4a516beb8a5846f06065294ca9457)
To work around the issue we're having with queue draining between
parallel blocking connections, use the same rabbitmq queue for both
activity and presence events, keyed on a 'type' flag in the message
itself.
(imported from commit 188e8fda1695734e52c5740db2195072cfc81479)
I experimented briefly with window.getSelection to try to determine if
any text was highlighted (and then we could've done something like
"Don't popover if text is highlighted"), but I couldn't quickly get it
to work in a robust way, so for now I am removing the feature.
(imported from commit 76048b8fd070675b480a2d29e2c8d7d6018633ec)
This is accomplished by continuning to have per-conversation PM counts
but then summing them up into a global count.
We may split this off into per-conversation counts in the future.
(imported from commit 311e3b74715c3a01c0b75837e397a386ab65505c)
json_to_foo will raise a ValueError if the JSON passed to it is just a
string containing a number, e.g. "1".
Traceback (most recent call last):
File "/home/tabbott/humbug/zephyr/views.py", line 711, in extract_recipients
recipients = json_to_list(raw_recipients)
File "/home/tabbott/humbug/zephyr/decorator.py", line 289, in json_to_list
return json_to_foo(json, list)
File "/home/tabbott/humbug/zephyr/decorator.py", line 282, in json_to_foo
raise ValueError("argument is not a %s" % (type().__class__.__name__))
ValueError: argument is not a list
Fixes#776.
(imported from commit 0c123a610c009eda9004cf0b0b53d60695c4e8d5)
Fixes#792. I'm really curious what (if anything) this change does to
our Tornado load numbers.
(imported from commit daf81d2dea81f36ec4a3eb19993169b58681e717)
This cuts the MIT Zephyr home page load time and also #subscriptions
database-sourced load time for me from about a second to about 50ms on
my laptop. The home page still takes about 600ms to load due to
templating.
(imported from commit 1cfd8c750301abe6b6a881be4b2857532be947ec)
The purpose of the validator is to ensure the user isn't active, so
let's correctly test for that here.
(imported from commit 772ddb901098f78750efab274405a10f36c49232)
Previously we checked and bailed when there was a user registered with
an email address, regardless of active status.
This meant that MIT users who had inactive accounts autocreated had
issues where they would be confusingly told they were signed up even
though they had never taken any action on our site directly.
Now we instead check whether there are any current *active* user
accounts with that email address, and proceed with generating an
activation link if the user lacks a corresponding active account.
Security implications of this commit come into play if we start
implementing removing users ability to sign in as deactivation. Since we
lack a user removal story here, this isn't terribly concerning yet and
we'll revist this code when we decide to add such functionality in the
future.
This resolves trac #581 and #631.
(imported from commit c3fb93ce065e63e19b41f63c1f27891b93b75f86)
(And to let you know that it's OK that you have no messages.)
This fixes Trac #850 for the case where you first log in.
(imported from commit 47741856e34f67bfc2cc91bdc21def75ab6fe09d)
The previous select_related didn't properly get the User object,
containing the email address, and thus would make one query per user
with presernce information.
(imported from commit 3341bc5a65387030fa8737b03ca43f79089ef56b)
On my dev machine this cuts /activity load time with lots of users by more than
2/3. I expect the gains will be even greater in production due to the greater
relative cost of database queries.
(imported from commit 0391cb29f66b618b4b99902d9fb9ab0a6cff0cb3)
I don't like monkeypatching in general, but this will give us logging
of how much time is being spent in total on database queries on our
staging and prod instances for each of the views we have -- which
should help a lot with debugging performance problems.
The code being patched is identical in Django 1.5, so I don't expect
any issues upgrading in the near future.
I may work on a patch to upstream Django to support this as well.
(imported from commit 08624c7a30f1fd192a972973e0380dd6a131d84e)
The previous commit stopped the mousewheel event at .bottom_sidebar,
which means it was never getting to our individual scrolling lists.
(imported from commit 92d32c21bb596d0e14d887ff779a857223d45342)
This isn't perfect, because if, e.g. the stream list is super short,
it still takes up lots of space that the user list might want to use,
but it's an acceptable first pass solution, in my view.
(imported from commit 669737d769258c089b40ffea4abee3229902e857)
When DEBUG=False, Django doesn't log the data at all. We can and
should add tracking of that at some point, but it requires patching
Django; so for now just log the data when DEBUG=True.
(imported from commit 2b9e6a4c68009733373724f38debe4e77313fdeb)
This should make it much easier to debug issues where a particular
user is hosing our API, for example.
(imported from commit cbea49fd1e11805cadf564bd9160d3d6bf7e0eca)
This lets us clean up the HTML a little bit in preparation
for a later change which will cause the stream and people
lists to scroll independently of one another.
Also it feels a bit more fun.
(imported from commit b3b49149d7ec2960fd752fe50b41e55d363c1a98)
This is actually a tricky one, because:
* Later, probably if we display an unread count by the person's
name, the action on clicking them might very well be
"Narrow to PMs with that person"
* But for now, while we don't have that, everything about
historical precedent really does strongly suggest that
clicking that person's name is going to get you a PM with
that person.
So we implement that. For now anyway.
(imported from commit 4d461fd6edec122d542c4a97e23f2e400c31122e)
SimpleQueueClient was not thread-safe as the pika method
calls that we are using might disconnect and end up in a bad
state if two threads enqueue messages at the same time.
In order to avoid this, each user of the rabbitmq queue
gets its own instance of a SimpleQueueClient.
(imported from commit 694083b75cd58a60b8de282a8f40eb92a864c5ce)
Note: When deploying, restarting the process-user-activity-commandline script is needed
(imported from commit 63ee795c9c7a7db4a40170cff5636dc1dd0b46a8)
Adds a new db table for storing presences, and an API for setting
an individual user's idleness as well as fetching all idle status
for all users in a realm
(imported from commit 5aad3510d4c90c49470c130d6dfa80f0d36b0057)
Now if you click the timestamp, the popover appears near there, rather than in
the middle of the message.
(imported from commit 285a71fddd5b6d9ac86511e5fe41c811d2663342)
Now that this isn't the object controlling the actions popover, we can set its
title= back to anything we like.
(imported from commit 36d3ecb23fd22cdc9189537820b944442f7bb5c6)
This addresses another usability problem from #470.
I decided it was too confusing to have menu options for both stream and subject
narrowing, so I went with the latter.
(imported from commit 948fca954a265029ceb27c72ae178015d513785d)
Hopefully the text and link-like styling will make it more clear that this
feature exists.
The sender's name is still clickable, but loses the link-like styling since the
popover isn't about the sender. The space between the sender's name and this
link is now active as well.
We also remove the sender's email address that was previously visible on hover.
It's not hard for the name + email + actions link + timestamp to be too long
for one line. The email is still present in the popover.
(imported from commit 3a85f199c42102cac0d217543db0a539935e2914)
We now set a CSS class on the hovered message, which is used to
control email address visibility.
(imported from commit 787e24f71f20aa3a6452e57b94f5ca1a4c8bc32f)
The message timestamp is now always clickable, and the popover contains the
full long-form date and time. This addresses one problem from usability
testing (see #470).
(imported from commit ad502dff128ad1c934fc0d3faaf5e2931c91c37e)
Bootstrap ignored this and instead used the title= attribute, containing the
full timestamp, which seems like what we want.
(imported from commit 8442835d61f89bd0bce75c05e17aabe85e0f417c)
The interface is a little messy because so many of the inputs change
-- we need to better organize all these variables under centralized
'zfilt' and 'zhome' headings.
(imported from commit b247e86bf63ea2ea2c0d42ef23e8af0ce288d5dd)
So here's the reproduction recipe:
(1) Find a narrow that doesn't have any messages since 4 days ago
(2) Directly visit that narrow in your browser (or wait for someone to
do a deploy and thus auto-reload)
(3) Wait until load_old_messages has been called at least once
(4) Un-narrow
(5) Scroll up, and notice that the 400 most recent messages are above
sets of older messages.
The cause is that the code in add_messages assumed that
selected_message_id was within the range of message already in the
home view. This is true in most cases where add_message is being
called, but it is not true in the case that the user was in a narrowed
view containing only very old messages (And thus selected_message_id
would be older than everything in the home view).
We can fix this by tracking persistent_message_id separately and using
that for the relevant test in add_messages.
(imported from commit f0da2561ba68f729343b260adc398029fae6acf7)
Code like this is dangerous:
class SimpleQueue(object):
queues = set()
because all instances will share the same 'queues' set object.
We don't really need multiple instances, but neither is there a reason to break
them. So do the normal Python thing instead.
(imported from commit a56bb8414dd549cfd312c9565df198cf9d20f08a)
Prior to this commit, successfully creating a stream (or unsubscribing
from a stream) didn't clear earlier error messages.
Here's how you could reproduce it:
* Try to subscribe to ""
(You get an error)
* Now unsubscribe from something, or subscribe
to a new stream
(The error message does not go away)
(imported from commit c3c6fa6081df00378182ff0c3499e9d907577c04)
Prior to this commit, we also cleared out all of your checkboxes,
which is frustrating if you wanted to invite another person to the
exact same set of streams.
(imported from commit 389f27ec35dc9bf8c9058c5ffa867929ac90f674)
Waiting for text "Waseemio" to appear isn't reliable, because the "create
stream" modal already contains that text. Instead, do a more specific check
for a .subscription_name element.
(imported from commit 54689aeb9d243c3fb62d1765b2b4fb93d3bb40db)
Note that the tsvector cache column should be fully populated before
commit is deployed. Otherwise, full text search will be broken until
it's populated.
(imported from commit 23c36fb7d146c289148e8243c3d6a9a6494cfc62)
This needs to be done in three South migrations to not block users
from sending messages for a long time. Adding the column requires a
write lock on the zephyr_message table and populating the new column
takes a long time. Thus, we can't do them both in the same
transaction (which South forces on migrations). Additionally,
creating the index takes a lot of computation and needs to lock the
table when not done CONCURRENTLY, which can't be done inside of a
transaction.
To do this manual change, you need to run:
python manage.py migrate zephyr 0007
ssh postgres.humbughq.com 'echo "CREATE INDEX CONCURRENTLY zephyr_message_search_tsvector ON zephyr_message USING gin(search_tsvector);" | psql'
python manage.py migrate zephyr 0008
on staging. No action is required on prod since the database is
shared.
Note that this migration must be done completely before we switch to
using the tsvector cache column.
(imported from commit b6a27013a60c1fd196eabb095d2d11d20bba5aac)
Because a stream can never stop being invite-only, we don't
actually need any Javascript to manipulate this besides what's
in the Handlebars template.
(imported from commit 30dc3b0baf623d88d03a643f18cd411dbe3eacfb)
If the database is down and the Handler crashes, we get an email with an
obnoxiously long subject line and without proper sanitization performed.
By wrapping internal_send_message in a try-except, we eliminate the most
common (read: to date, only) cause of crashes in our Handlers.
(imported from commit fd59e931ad0a4fff574d3cafa47348eacacb858f)
We also remove META here since it rarely contains anything useful, and
often contains sensitive environment vars.
(imported from commit 2909613f9f52684bef9175600961801104644c75)
This allows us to remove fetch_colors() entirely, and should speed up page
load a bit.
We also JSONEncoderForHTML instead of dumps so that the result is safe
to embed.
(imported from commit 013630911960e2ac1d0bae6f5df31ad342750594)
This will give us flexibility in the future to add new properties to the
list.
In order to support that, we now do a list comprehension rather than just
returning the gather_subscriptions list in get_stream_colors.
(imported from commit a3c0f749a3320f647440f800105942434da08111)
Previously we checked if property was false after doing .strip(). Since
you can't call string methods on a NoneType, we were 500ing.
The code now does a normal dictionary get via [] and catches the
KeyError.
(imported from commit da7f28febf0865f44e92bcac1791f817c3d370f3)
Returning json_error inside an inner function call will result in the error
getting lost.
(imported from commit fd7754b15f7b62fd6e4197fd72ae03d6996a93da)
Previously we made calls to the JSON api, which means that the API key
was being ignored.
(imported from commit 46d8d0e5ac7926e824f300fd846ec42bc939e2c0)
Autogenerated schema migration:
+ Added field in_home_view on zephyr.Subscription
To do this manual change, you need to run:
python manage.py migrate zephyr 0005
on staging. No action is required on prod since there is a shared database.
(imported from commit d554f17b25631482ec2d5605a42ac0b9d6df421e)
Previously we only got the user ID for /json requests, not /api
requests, and also only got the user ID, not the email address.
(imported from commit c3625f9c1a48430e35183be6c90a7855f3714948)
Introduced in d19991c7. //-style comments aren't supported in CSS.
This was causing update-prod-static to fail with a stack overflow.
(imported from commit 4cb11a7025e8fef32c852d691fccf18a5ad3b955)
This was removed in 63788aa3. It was the only non-symlink in
zephyr/static-access-control/.
(imported from commit c9ee043b07c5fd4050aeaafa40394a6b69915bfb)
This test case was supposed to be making requests as 2 different
users, but was not logging in as each of them and thus used the wrong
credentials.
(imported from commit 0dfdc86fcb385d3d2a721181b2f8075bc39ad723)
This came up when I tried to subscribe myself to an existing stream
name but with leading whitespace added; the get_subscribers request
failed with a 400.
(imported from commit 03fe370d1e80fd7e8c282f260e63b7e5ada05960)
Before, it let you subscribe only yourself or one other person to
possibly many streams.
This is used by the subscriptions page to specify the subscribers when
you create a stream.
(imported from commit c1055e98b0bb27799ac9e6dad80b9c9fd87deca2)
In preparation for re-using the /json/subscriptions/exists request on
the subscriptions page.
(imported from commit 76eca95b952c4b60e583a050be711023ee5fedac)
Make sure they still stand out against the grey background.
It's less clear that we should do this for private messages as well, but it
seems fine for now.
Fixes#658.
(imported from commit 4c750208b5f2697e91c1c4fa7365d2744260c6f2)
Prior to this commit, if you have the composebox open, pressing 'c' or
'C' clears its contents. This change makes it work more analogously to
pressing the 'New stream message'/'New private message' buttons.
(imported from commit 3de5bf83754d8ab86b1967ce2ba15f5846090667)
As it currently stands, after the introduction of operators, narrowing
to messages that contained X would also trigger a find-in-page.
This stops that from happening, and then also makes the default action
of the search-bar-invoked-without-a-typeahead be 'narrow to messages
containing x' rather than 'find in page'.
(imported from commit 1beffce426c6b00449e7c1c803687a129747ed63)
This is a tricky one because it's kind of hard to see locally, but
there's a bit of a delay between when you click "Invite" and when we
get back to you. So we give the button a "loading" state so you know
not to click again.
(imported from commit 9c3389a3d06def777427c846d6106f6d9b30cc8b)
Leo points out that if you type a stream name, we probably want
to offer to narrow you to that stream more than we should offer
to find that stream name in the page.
(imported from commit 803ac681ec2f759f4dabb68a40722a07b86a0cab)
On my laptop, this cuts search time from several seconds down to
a few hundred milliseconds.
If we want even more speed, we could store the ts_vector as a column
on the message database. The Postgres documentation says this will
make "searches [...] faster, since it will not be necessary to redo
the to_tsvector calls to verify index matches." Going this route
requires creating a trigger to automatically insert the appropriate
column when new rows are inserted.
Note that the full text index must be fully created before this
commit is deployed. Full text search without an index is actually
significantly slower than using the LIKE operator.
(imported from commit ae74083da20d33aa2425d3e44fcdc19b160002ba)
This schema migration is only for use in automated migrations. To
deploy on the production database (the migration only needs to be
done once for both of staging and prod because they share a
database), you should instead execute the following SQL manually:
$ ssh postgres.humbughq.com
$ psql
humbug=> CREATE INDEX CONCURRENTLY zephyr_message_full_text_idx ON zephyr_message USING gin(to_tsvector('english', subject || ' ' || content));
Note the addition of the "CONCURRENTLY" keyword. The problem is that
creating the index takes non-trivial time and requires a write lock
on the table while the index is being created. This would mean that
users would be unable to send messages while we were generating the
index, which isn't acceptable. We can't create the index
concurrently in the South migration because concurrent index
creations can't happen inside of a transaction and South forces a
transaction on migration functions.
Also note that this index must be created before Postgres full text
search is deployed to the app because full text search without an index
is actually much slower than plain search using the LIKE operator.
(imported from commit 8b9445c27d0e427278de997b22342bffe6d855b7)
This is for allowing us to do things more complex than returning a Q
expression, needed for doing Postgres full text search.
(imported from commit 669ec71417c04baaf8ed1774bee147079b05b03d)
We will minify our code, rather than trying to restrict who can see the
un-minified code. Removing access control first simplifies things.
Manual deployment steps:
scp servers/puppet/files/nginx/humbug-include/app root@staging.humbughq.com:/etc/nginx/humbug-include/
ssh root@staging.humbughq.com service nginx reload
and then the same for app.humbughq.com once deployed to prod.
(imported from commit 63788aa3fa7ba5fd97fcf85b05760abb5e7cae4b)
Leo Franchi
Reid Barton
Keegan McAllister
Jessica McKellar
Zev Benjamin
Luke O'Malley
Waseem Daher
Tim Abbott
Luke Faraone
Jeff Arnold
Jacob Hurwitz