json_to_foo will raise a ValueError if the JSON passed to it is just a
string containing a number, e.g. "1".
Traceback (most recent call last):
File "/home/tabbott/humbug/zephyr/views.py", line 711, in extract_recipients
recipients = json_to_list(raw_recipients)
File "/home/tabbott/humbug/zephyr/decorator.py", line 289, in json_to_list
return json_to_foo(json, list)
File "/home/tabbott/humbug/zephyr/decorator.py", line 282, in json_to_foo
raise ValueError("argument is not a %s" % (type().__class__.__name__))
ValueError: argument is not a list
Fixes#776.
(imported from commit 0c123a610c009eda9004cf0b0b53d60695c4e8d5)
Fixes#792. I'm really curious what (if anything) this change does to
our Tornado load numbers.
(imported from commit daf81d2dea81f36ec4a3eb19993169b58681e717)
This cuts the MIT Zephyr home page load time and also #subscriptions
database-sourced load time for me from about a second to about 50ms on
my laptop. The home page still takes about 600ms to load due to
templating.
(imported from commit 1cfd8c750301abe6b6a881be4b2857532be947ec)
The purpose of the validator is to ensure the user isn't active, so
let's correctly test for that here.
(imported from commit 772ddb901098f78750efab274405a10f36c49232)
Previously we checked and bailed when there was a user registered with
an email address, regardless of active status.
This meant that MIT users who had inactive accounts autocreated had
issues where they would be confusingly told they were signed up even
though they had never taken any action on our site directly.
Now we instead check whether there are any current *active* user
accounts with that email address, and proceed with generating an
activation link if the user lacks a corresponding active account.
Security implications of this commit come into play if we start
implementing removing users ability to sign in as deactivation. Since we
lack a user removal story here, this isn't terribly concerning yet and
we'll revist this code when we decide to add such functionality in the
future.
This resolves trac #581 and #631.
(imported from commit c3fb93ce065e63e19b41f63c1f27891b93b75f86)
(And to let you know that it's OK that you have no messages.)
This fixes Trac #850 for the case where you first log in.
(imported from commit 47741856e34f67bfc2cc91bdc21def75ab6fe09d)
The previous select_related didn't properly get the User object,
containing the email address, and thus would make one query per user
with presernce information.
(imported from commit 3341bc5a65387030fa8737b03ca43f79089ef56b)
On my dev machine this cuts /activity load time with lots of users by more than
2/3. I expect the gains will be even greater in production due to the greater
relative cost of database queries.
(imported from commit 0391cb29f66b618b4b99902d9fb9ab0a6cff0cb3)
I don't like monkeypatching in general, but this will give us logging
of how much time is being spent in total on database queries on our
staging and prod instances for each of the views we have -- which
should help a lot with debugging performance problems.
The code being patched is identical in Django 1.5, so I don't expect
any issues upgrading in the near future.
I may work on a patch to upstream Django to support this as well.
(imported from commit 08624c7a30f1fd192a972973e0380dd6a131d84e)
The previous commit stopped the mousewheel event at .bottom_sidebar,
which means it was never getting to our individual scrolling lists.
(imported from commit 92d32c21bb596d0e14d887ff779a857223d45342)
This isn't perfect, because if, e.g. the stream list is super short,
it still takes up lots of space that the user list might want to use,
but it's an acceptable first pass solution, in my view.
(imported from commit 669737d769258c089b40ffea4abee3229902e857)
When DEBUG=False, Django doesn't log the data at all. We can and
should add tracking of that at some point, but it requires patching
Django; so for now just log the data when DEBUG=True.
(imported from commit 2b9e6a4c68009733373724f38debe4e77313fdeb)
This should make it much easier to debug issues where a particular
user is hosing our API, for example.
(imported from commit cbea49fd1e11805cadf564bd9160d3d6bf7e0eca)
This lets us clean up the HTML a little bit in preparation
for a later change which will cause the stream and people
lists to scroll independently of one another.
Also it feels a bit more fun.
(imported from commit b3b49149d7ec2960fd752fe50b41e55d363c1a98)
This is actually a tricky one, because:
* Later, probably if we display an unread count by the person's
name, the action on clicking them might very well be
"Narrow to PMs with that person"
* But for now, while we don't have that, everything about
historical precedent really does strongly suggest that
clicking that person's name is going to get you a PM with
that person.
So we implement that. For now anyway.
(imported from commit 4d461fd6edec122d542c4a97e23f2e400c31122e)
SimpleQueueClient was not thread-safe as the pika method
calls that we are using might disconnect and end up in a bad
state if two threads enqueue messages at the same time.
In order to avoid this, each user of the rabbitmq queue
gets its own instance of a SimpleQueueClient.
(imported from commit 694083b75cd58a60b8de282a8f40eb92a864c5ce)
Note: When deploying, restarting the process-user-activity-commandline script is needed
(imported from commit 63ee795c9c7a7db4a40170cff5636dc1dd0b46a8)
Adds a new db table for storing presences, and an API for setting
an individual user's idleness as well as fetching all idle status
for all users in a realm
(imported from commit 5aad3510d4c90c49470c130d6dfa80f0d36b0057)
Now if you click the timestamp, the popover appears near there, rather than in
the middle of the message.
(imported from commit 285a71fddd5b6d9ac86511e5fe41c811d2663342)
Now that this isn't the object controlling the actions popover, we can set its
title= back to anything we like.
(imported from commit 36d3ecb23fd22cdc9189537820b944442f7bb5c6)
This addresses another usability problem from #470.
I decided it was too confusing to have menu options for both stream and subject
narrowing, so I went with the latter.
(imported from commit 948fca954a265029ceb27c72ae178015d513785d)
Hopefully the text and link-like styling will make it more clear that this
feature exists.
The sender's name is still clickable, but loses the link-like styling since the
popover isn't about the sender. The space between the sender's name and this
link is now active as well.
We also remove the sender's email address that was previously visible on hover.
It's not hard for the name + email + actions link + timestamp to be too long
for one line. The email is still present in the popover.
(imported from commit 3a85f199c42102cac0d217543db0a539935e2914)
We now set a CSS class on the hovered message, which is used to
control email address visibility.
(imported from commit 787e24f71f20aa3a6452e57b94f5ca1a4c8bc32f)
The message timestamp is now always clickable, and the popover contains the
full long-form date and time. This addresses one problem from usability
testing (see #470).
(imported from commit ad502dff128ad1c934fc0d3faaf5e2931c91c37e)
Bootstrap ignored this and instead used the title= attribute, containing the
full timestamp, which seems like what we want.
(imported from commit 8442835d61f89bd0bce75c05e17aabe85e0f417c)
The interface is a little messy because so many of the inputs change
-- we need to better organize all these variables under centralized
'zfilt' and 'zhome' headings.
(imported from commit b247e86bf63ea2ea2c0d42ef23e8af0ce288d5dd)
So here's the reproduction recipe:
(1) Find a narrow that doesn't have any messages since 4 days ago
(2) Directly visit that narrow in your browser (or wait for someone to
do a deploy and thus auto-reload)
(3) Wait until load_old_messages has been called at least once
(4) Un-narrow
(5) Scroll up, and notice that the 400 most recent messages are above
sets of older messages.
The cause is that the code in add_messages assumed that
selected_message_id was within the range of message already in the
home view. This is true in most cases where add_message is being
called, but it is not true in the case that the user was in a narrowed
view containing only very old messages (And thus selected_message_id
would be older than everything in the home view).
We can fix this by tracking persistent_message_id separately and using
that for the relevant test in add_messages.
(imported from commit f0da2561ba68f729343b260adc398029fae6acf7)
Code like this is dangerous:
class SimpleQueue(object):
queues = set()
because all instances will share the same 'queues' set object.
We don't really need multiple instances, but neither is there a reason to break
them. So do the normal Python thing instead.
(imported from commit a56bb8414dd549cfd312c9565df198cf9d20f08a)
Prior to this commit, successfully creating a stream (or unsubscribing
from a stream) didn't clear earlier error messages.
Here's how you could reproduce it:
* Try to subscribe to ""
(You get an error)
* Now unsubscribe from something, or subscribe
to a new stream
(The error message does not go away)
(imported from commit c3c6fa6081df00378182ff0c3499e9d907577c04)
Prior to this commit, we also cleared out all of your checkboxes,
which is frustrating if you wanted to invite another person to the
exact same set of streams.
(imported from commit 389f27ec35dc9bf8c9058c5ffa867929ac90f674)
Waiting for text "Waseemio" to appear isn't reliable, because the "create
stream" modal already contains that text. Instead, do a more specific check
for a .subscription_name element.
(imported from commit 54689aeb9d243c3fb62d1765b2b4fb93d3bb40db)
Note that the tsvector cache column should be fully populated before
commit is deployed. Otherwise, full text search will be broken until
it's populated.
(imported from commit 23c36fb7d146c289148e8243c3d6a9a6494cfc62)
This needs to be done in three South migrations to not block users
from sending messages for a long time. Adding the column requires a
write lock on the zephyr_message table and populating the new column
takes a long time. Thus, we can't do them both in the same
transaction (which South forces on migrations). Additionally,
creating the index takes a lot of computation and needs to lock the
table when not done CONCURRENTLY, which can't be done inside of a
transaction.
To do this manual change, you need to run:
python manage.py migrate zephyr 0007
ssh postgres.humbughq.com 'echo "CREATE INDEX CONCURRENTLY zephyr_message_search_tsvector ON zephyr_message USING gin(search_tsvector);" | psql'
python manage.py migrate zephyr 0008
on staging. No action is required on prod since the database is
shared.
Note that this migration must be done completely before we switch to
using the tsvector cache column.
(imported from commit b6a27013a60c1fd196eabb095d2d11d20bba5aac)
Because a stream can never stop being invite-only, we don't
actually need any Javascript to manipulate this besides what's
in the Handlebars template.
(imported from commit 30dc3b0baf623d88d03a643f18cd411dbe3eacfb)
If the database is down and the Handler crashes, we get an email with an
obnoxiously long subject line and without proper sanitization performed.
By wrapping internal_send_message in a try-except, we eliminate the most
common (read: to date, only) cause of crashes in our Handlers.
(imported from commit fd59e931ad0a4fff574d3cafa47348eacacb858f)
We also remove META here since it rarely contains anything useful, and
often contains sensitive environment vars.
(imported from commit 2909613f9f52684bef9175600961801104644c75)
This allows us to remove fetch_colors() entirely, and should speed up page
load a bit.
We also JSONEncoderForHTML instead of dumps so that the result is safe
to embed.
(imported from commit 013630911960e2ac1d0bae6f5df31ad342750594)
This will give us flexibility in the future to add new properties to the
list.
In order to support that, we now do a list comprehension rather than just
returning the gather_subscriptions list in get_stream_colors.
(imported from commit a3c0f749a3320f647440f800105942434da08111)
Previously we checked if property was false after doing .strip(). Since
you can't call string methods on a NoneType, we were 500ing.
The code now does a normal dictionary get via [] and catches the
KeyError.
(imported from commit da7f28febf0865f44e92bcac1791f817c3d370f3)
Returning json_error inside an inner function call will result in the error
getting lost.
(imported from commit fd7754b15f7b62fd6e4197fd72ae03d6996a93da)
Previously we made calls to the JSON api, which means that the API key
was being ignored.
(imported from commit 46d8d0e5ac7926e824f300fd846ec42bc939e2c0)
Autogenerated schema migration:
+ Added field in_home_view on zephyr.Subscription
To do this manual change, you need to run:
python manage.py migrate zephyr 0005
on staging. No action is required on prod since there is a shared database.
(imported from commit d554f17b25631482ec2d5605a42ac0b9d6df421e)
Previously we only got the user ID for /json requests, not /api
requests, and also only got the user ID, not the email address.
(imported from commit c3625f9c1a48430e35183be6c90a7855f3714948)
Introduced in d19991c7. //-style comments aren't supported in CSS.
This was causing update-prod-static to fail with a stack overflow.
(imported from commit 4cb11a7025e8fef32c852d691fccf18a5ad3b955)
This was removed in 63788aa3. It was the only non-symlink in
zephyr/static-access-control/.
(imported from commit c9ee043b07c5fd4050aeaafa40394a6b69915bfb)
This test case was supposed to be making requests as 2 different
users, but was not logging in as each of them and thus used the wrong
credentials.
(imported from commit 0dfdc86fcb385d3d2a721181b2f8075bc39ad723)
This came up when I tried to subscribe myself to an existing stream
name but with leading whitespace added; the get_subscribers request
failed with a 400.
(imported from commit 03fe370d1e80fd7e8c282f260e63b7e5ada05960)
Before, it let you subscribe only yourself or one other person to
possibly many streams.
This is used by the subscriptions page to specify the subscribers when
you create a stream.
(imported from commit c1055e98b0bb27799ac9e6dad80b9c9fd87deca2)
In preparation for re-using the /json/subscriptions/exists request on
the subscriptions page.
(imported from commit 76eca95b952c4b60e583a050be711023ee5fedac)
Make sure they still stand out against the grey background.
It's less clear that we should do this for private messages as well, but it
seems fine for now.
Fixes#658.
(imported from commit 4c750208b5f2697e91c1c4fa7365d2744260c6f2)
Prior to this commit, if you have the composebox open, pressing 'c' or
'C' clears its contents. This change makes it work more analogously to
pressing the 'New stream message'/'New private message' buttons.
(imported from commit 3de5bf83754d8ab86b1967ce2ba15f5846090667)
As it currently stands, after the introduction of operators, narrowing
to messages that contained X would also trigger a find-in-page.
This stops that from happening, and then also makes the default action
of the search-bar-invoked-without-a-typeahead be 'narrow to messages
containing x' rather than 'find in page'.
(imported from commit 1beffce426c6b00449e7c1c803687a129747ed63)
This is a tricky one because it's kind of hard to see locally, but
there's a bit of a delay between when you click "Invite" and when we
get back to you. So we give the button a "loading" state so you know
not to click again.
(imported from commit 9c3389a3d06def777427c846d6106f6d9b30cc8b)
Leo points out that if you type a stream name, we probably want
to offer to narrow you to that stream more than we should offer
to find that stream name in the page.
(imported from commit 803ac681ec2f759f4dabb68a40722a07b86a0cab)
On my laptop, this cuts search time from several seconds down to
a few hundred milliseconds.
If we want even more speed, we could store the ts_vector as a column
on the message database. The Postgres documentation says this will
make "searches [...] faster, since it will not be necessary to redo
the to_tsvector calls to verify index matches." Going this route
requires creating a trigger to automatically insert the appropriate
column when new rows are inserted.
Note that the full text index must be fully created before this
commit is deployed. Full text search without an index is actually
significantly slower than using the LIKE operator.
(imported from commit ae74083da20d33aa2425d3e44fcdc19b160002ba)
This schema migration is only for use in automated migrations. To
deploy on the production database (the migration only needs to be
done once for both of staging and prod because they share a
database), you should instead execute the following SQL manually:
$ ssh postgres.humbughq.com
$ psql
humbug=> CREATE INDEX CONCURRENTLY zephyr_message_full_text_idx ON zephyr_message USING gin(to_tsvector('english', subject || ' ' || content));
Note the addition of the "CONCURRENTLY" keyword. The problem is that
creating the index takes non-trivial time and requires a write lock
on the table while the index is being created. This would mean that
users would be unable to send messages while we were generating the
index, which isn't acceptable. We can't create the index
concurrently in the South migration because concurrent index
creations can't happen inside of a transaction and South forces a
transaction on migration functions.
Also note that this index must be created before Postgres full text
search is deployed to the app because full text search without an index
is actually much slower than plain search using the LIKE operator.
(imported from commit 8b9445c27d0e427278de997b22342bffe6d855b7)
This is for allowing us to do things more complex than returning a Q
expression, needed for doing Postgres full text search.
(imported from commit 669ec71417c04baaf8ed1774bee147079b05b03d)
We will minify our code, rather than trying to restrict who can see the
un-minified code. Removing access control first simplifies things.
Manual deployment steps:
scp servers/puppet/files/nginx/humbug-include/app root@staging.humbughq.com:/etc/nginx/humbug-include/
ssh root@staging.humbughq.com service nginx reload
and then the same for app.humbughq.com once deployed to prod.
(imported from commit 63788aa3fa7ba5fd97fcf85b05760abb5e7cae4b)
1) Make the search hotkey (/) no longer clear the search box
2) Vertically center the "Search..." placeholder
(imported from commit 02dee6b72c4457e160f57d8604164e15e62f5f28)
This fixes Trac #723 - Message view scrolls to top after reloading on another tab (e.g. settings)
(imported from commit d9134cec6879625d577c43a08d258af3f6dacc5f)
Tim Abbott
Zev Benjamin
Waseem Daher
Keegan McAllister
Jeff Arnold
Luke Faraone
Reid Barton
Leo Franchi
Jessica McKellar
Jacob Hurwitz