This creates a new management command, subscribe_new_users, which should be
run as a daemon process. When new users are created, an event is passed to
RabbitMQ including the following data:
* Email
* Full name
* IP address of the person who confirmed registration
* Time of registration confirmation
MailChimp strongly encourages the collection of the last two to enable
responses to abuse requests, and providing more data lowers the chance that
we could get banned from their service if complaints do occur.
To use this commit, you need to install the "postmonkey" module from
PyPI.
(imported from commit 20c628c3fa8bb985aaead85a80ad3b38bf94b9dc)
Beanstalk integration uses webhooks that use http basic auth to authenticate
the sending user.
(imported from commit bd65f5b2d052a3c1eb04da64d055a3640a384892)
I think all that one needs to do to deploy this commit is on developer
laptops, run `generate-fixtures --force`.
(imported from commit 34916341435fef0875b5a2c7f53c2f5606cd16cd)
When this is deployed to staging, we need to run
./manage.py logout_all_users --realm=humbughq.com
When this is deployed to prod, we need to run
./manage.py logout_all_users
(imported from commit d6c6ea4b1c347f3d9122742db23c7b67767a7349)
Only a few of them took a User as an argument anyway.
This is preparatory work for merging the User and UserProfile models.
(imported from commit 65b2bd2453597531bcf135ccf24d2a4615cd0d2a)
Clients can now request to receive only certain kinds of events,
although they always receive restart events.
(imported from commit 1e72981f8fe763829ab2abde1e35f94cad5c34e4)
Previously user_profile was a kwarg, which was inconsistent with all other
_backend functions.
(imported from commit 6b857bcb2c3c978079af2f6edd367c1804d51988)
This includes a process_patch_as_post decorator which enables this view
to be invoked as a PATCH on an object.
Hopefully this decorator can go away once POST values are correctly parsed
in Django for PATCH verb invocations.
(imported from commit 6cf9d69cfb9dea5354ea37408566146757b5be54)
This slightly reduces code duplication and in the future the {api,json}_ methods
will hopefully go away, leaving only the _backend methods.
(imported from commit 82a6e4a2ff2ba5d272068e9ff043ea47a1a8d278)
Instead we now rely on the request._client value, which we were previously
passing along to s_m_b in all but one case.
For that one case, we just modify the Request object to include the value
beforehand.
(imported from commit 542f38f94bc447149cd4d2efaa5e8f48f756725b)
This can result in a significant performance benefit because we only
need to update the columns that changed..
(imported from commit 42bef1fcc58ad79bd864f89263fe82e90743ee5b)
This is a lot cleaner, and also cuts about 50-70 ms off of page load time in
local testing (with lots of users), presumably because there's less work to be
done by the slow Django template engine.
(imported from commit 257b700238ee5d9a4ae00a53011ed5bce018124c)
This will automatically fix bugs such as one in which
internal_send_message didn't properly strip() the subject argument
before sending a message.
We change the recipient_type argument to internal_send_message to take
the recipient type name (e.g. 'stream') both to better fit the API and
also because the previous code incorrectly handled huddles.
(imported from commit 78c2596d328f6bb1ce2eaa3eed9a9e48146e3b6a)
This is preparatory for using this new arguments checking function in
internal_send_message as well.
(imported from commit 578e09c50b8a700c019c7dd235b2d9527af34e39)
This fixes a nondeterministic test failure for me.
The first message sent in the test suite appears to get dropped. I don't know
why this is, and I'm pretty sure it was an existing bug. This message used to
be the one disabling the tutorial, which might explain why that didn't always
work.
Regardless, this commit at least makes the test suite usable, and we can work
on fixing that bug later.
(imported from commit 063e40871b9883e3a6dab93a4e0a51c5b2dae4b7)
This should substantially decrease the amount of server load generated
by the userpresence system.
I tested that this indeed was indeed saving one query on
/json/update_active_status requests on my laptop with 2 users from the
humbughq.com realm logged in.
(imported from commit 03e9d4eb95b9f664d489862684ae162db2076e08)
It's closer to a presence query than an update, and more importantly
this moves this out of Tornado -- previously Tornado was spending at
least 3ms per recipient on messages sent to the MIT realm fetching all
this data to return back to users. This should save around 100ms per
message sent to a popular stream the MIT realm -- but more
importantly, each such event is 100ms during which Tornado is not
processing other messages.
(imported from commit 134169f0fdcd9f6640fda957edc4a28b07783d8e)
We were previously having an issue where the tutorial could
be pre-empted if you got a few messages while you were first
logging in.
I have some reservations about this being slightly fragile, and a
better approach might be to just have a bit that we use to determine
whether or not you've already seen a tutorial. (Or potentially that
checks whether or not you've ever sent a message.)
(imported from commit f8858f64a36bcd25887b76314caff283929f340c)
When narrowing, we use the selected id in the home view as the anchor,
and if the user is new and hasn't selected any messages yet (or moved
the pointer since first using humbug), their home view selected message
will be -1. Rather than failing to get_old_messages for any narrows starting
from that point, return all matching messages.
(imported from commit 72cfe392d9ac01ed41abc8eadf0f47240e374665)
I find that I never use it, and I don't totally like our
experience in the app to be different from our users'.
Admittedly, this is a small way in which that's the case :)
Finally, since we do usability studies in @humbughq.com,
the link appears there too, and I'd like it not to.
(imported from commit 1225c4ae79de52fa98b21ce00a6542df76b667ea)
Prior to this change, any stream message sent by internal_send_message
could only be in the realm of the sender.
This was a problem most notably for... the tutorial bot, with the
hilarious consequence that the tutorial worked fine in humbughq.com,
but failed to start anywhere else.
(imported from commit 33a904a28e3a57e1a2cf9172c2e2a75b50967a50)
Require POST method for /accounts/logout. This has the side effect of
automatically enabling Django's CSRF protection.
(imported from commit 44b1b6ebaadc1c03006e21ae54ac768e31234801)
But only allow them to send to tutorial-<<your username>>.
The idea being that this helps reduce potential abuse from this JSON
call. (Because otherwise, anyone could call into this endpoint and
have the tutorial bot send random messages to random peoples's
streams.)
(imported from commit 471d4348d7ad43858b5df240e4f1dceba006aab6)
After c1d98239 the function works in CasperJS as well.
Reverts some of 90f4d6ac3ddb387e74051b9af2c230698fa94479.
(imported from commit 3579df33930bb34dc081908b84900905eee6d270)
The idea here is: part of the onboarding tutorial is going to
be you talking to the tutorial bot and it talking to you, from
our Javascript.
The reason it's driven by Javascript is that then in principle we can
do nice stuff like making popovers appear in places to point things
out to you, whereas if we were to do it strictly server-side, doing so
would be a lot harder.
The downside to doing it in Javascript is that you don't get any of
the Markdown rendering, since that happens on the server. So instead
we add this call where you give it a message, and it responds by
having the tutorial bot send you that message.
I don't think there are any security concerns here because
(1) The bot only messages you -- so you can't use it to make someone
else think that the system is telling them to do something
(2) If there were an issue associated with having the server parse
arbitrary Markdown, you could just trigger the issue by sending
a message yourself.
(imported from commit b34f594dab6be6bcb81899278ae1cbe447404468)
This will discard punctuation symbols in the Postgres search and also prevent
syntax errors when users try to submit queries with symbols that to_tsquery
interprets as special syntax (such as '|' and '&').
Fixes#906
(imported from commit 3e3a0d6ae3d4a516beb8a5846f06065294ca9457)
json_to_foo will raise a ValueError if the JSON passed to it is just a
string containing a number, e.g. "1".
Traceback (most recent call last):
File "/home/tabbott/humbug/zephyr/views.py", line 711, in extract_recipients
recipients = json_to_list(raw_recipients)
File "/home/tabbott/humbug/zephyr/decorator.py", line 289, in json_to_list
return json_to_foo(json, list)
File "/home/tabbott/humbug/zephyr/decorator.py", line 282, in json_to_foo
raise ValueError("argument is not a %s" % (type().__class__.__name__))
ValueError: argument is not a list
Fixes#776.
(imported from commit 0c123a610c009eda9004cf0b0b53d60695c4e8d5)
The purpose of the validator is to ensure the user isn't active, so
let's correctly test for that here.
(imported from commit 772ddb901098f78750efab274405a10f36c49232)
Previously we checked and bailed when there was a user registered with
an email address, regardless of active status.
This meant that MIT users who had inactive accounts autocreated had
issues where they would be confusingly told they were signed up even
though they had never taken any action on our site directly.
Now we instead check whether there are any current *active* user
accounts with that email address, and proceed with generating an
activation link if the user lacks a corresponding active account.
Security implications of this commit come into play if we start
implementing removing users ability to sign in as deactivation. Since we
lack a user removal story here, this isn't terribly concerning yet and
we'll revist this code when we decide to add such functionality in the
future.
This resolves trac #581 and #631.
(imported from commit c3fb93ce065e63e19b41f63c1f27891b93b75f86)
The previous select_related didn't properly get the User object,
containing the email address, and thus would make one query per user
with presernce information.
(imported from commit 3341bc5a65387030fa8737b03ca43f79089ef56b)
On my dev machine this cuts /activity load time with lots of users by more than
2/3. I expect the gains will be even greater in production due to the greater
relative cost of database queries.
(imported from commit 0391cb29f66b618b4b99902d9fb9ab0a6cff0cb3)
Note: When deploying, restarting the process-user-activity-commandline script is needed
(imported from commit 63ee795c9c7a7db4a40170cff5636dc1dd0b46a8)
Adds a new db table for storing presences, and an API for setting
an individual user's idleness as well as fetching all idle status
for all users in a realm
(imported from commit 5aad3510d4c90c49470c130d6dfa80f0d36b0057)
Note that the tsvector cache column should be fully populated before
commit is deployed. Otherwise, full text search will be broken until
it's populated.
(imported from commit 23c36fb7d146c289148e8243c3d6a9a6494cfc62)
This allows us to remove fetch_colors() entirely, and should speed up page
load a bit.
We also JSONEncoderForHTML instead of dumps so that the result is safe
to embed.
(imported from commit 013630911960e2ac1d0bae6f5df31ad342750594)
This will give us flexibility in the future to add new properties to the
list.
In order to support that, we now do a list comprehension rather than just
returning the gather_subscriptions list in get_stream_colors.
(imported from commit a3c0f749a3320f647440f800105942434da08111)
Previously we checked if property was false after doing .strip(). Since
you can't call string methods on a NoneType, we were 500ing.
The code now does a normal dictionary get via [] and catches the
KeyError.
(imported from commit da7f28febf0865f44e92bcac1791f817c3d370f3)
Returning json_error inside an inner function call will result in the error
getting lost.
(imported from commit fd7754b15f7b62fd6e4197fd72ae03d6996a93da)
Previously we made calls to the JSON api, which means that the API key
was being ignored.
(imported from commit 46d8d0e5ac7926e824f300fd846ec42bc939e2c0)
Before, it let you subscribe only yourself or one other person to
possibly many streams.
This is used by the subscriptions page to specify the subscribers when
you create a stream.
(imported from commit c1055e98b0bb27799ac9e6dad80b9c9fd87deca2)
On my laptop, this cuts search time from several seconds down to
a few hundred milliseconds.
If we want even more speed, we could store the ts_vector as a column
on the message database. The Postgres documentation says this will
make "searches [...] faster, since it will not be necessary to redo
the to_tsvector calls to verify index matches." Going this route
requires creating a trigger to automatically insert the appropriate
column when new rows are inserted.
Note that the full text index must be fully created before this
commit is deployed. Full text search without an index is actually
significantly slower than using the LIKE operator.
(imported from commit ae74083da20d33aa2425d3e44fcdc19b160002ba)
This is for allowing us to do things more complex than returning a Q
expression, needed for doing Postgres full text search.
(imported from commit 669ec71417c04baaf8ed1774bee147079b05b03d)
Trying to add a user to an invite-only stream that already
exists will result in in error
(imported from commit 910750580a122cee92096d7e83457cb0b8cce616)
Apparently short_name is not consistently the person's username
in our system -- for many MIT people, it's their full name.
(imported from commit d975300a59c7afe5ee20b4a0db1bd51110c3df33)
As of this writing, short_name is the part before the @-sign,
and we already display the realm, so no need to display it
twice.
(imported from commit f5f9b29bff99b36b0c23176348c2869193467c38)
Eventually there should be a database table or something where we can
configure this for each realm.
(imported from commit 5f3e6823f420e113da8ff8c521cc706175299fa4)
/?lurk=foo will show all messages to the stream "foo", regardless of
whether you're subscribed.
(imported from commit 049d98b3ee8df19ef0a9dc392ae941dd463f8dd5)
Right now we mitigate abuse by sending the principal user a Humbug
informing them that they've been subscribed. Longer term, we want
some sort of stream invitation system, instead.
(imported from commit bf1f1ee0c5d3803cead1ddd9a766ff02043b629e)
The schema change made this no longer a guarantee, and creating duplicate
PUs is not harmful.
(imported from commit ca05b4dad7d3cc12a35548817f98c0a072c14c8f)
Otherwise one gets:
AttributeError: 'module' object has no attribute 'time'
when trying to use the time module from inside zephyr.lib.
(imported from commit 645368672a3eff68320278dd480edeed56721fcc)
CasperJS can't handle them; window.webkitNotifications.requestPermission()
throws a type error. We can revisit this when we want to write tests for the
notification code.
(imported from commit 90f4d6ac3ddb387e74051b9af2c230698fa94479)
This view lives at /accounts/accept_terms, and (after getting an acceptance
from the user) sends an email to all@ documenting the acceptance.
(imported from commit 8f64286ab02887fd6544fa274b2967f6499b6dbc)
Inspection of the postgres slow queries log showed that the "narrow to
personals with a particular user" database queries were taking a long
time to run (0.5s+). Further investigation determined that the OR
gate construction used here was causing the entire zephyr_message
table to be scanned; primarily I think because we were using the
implicit constraint that the logged in user had received messages.
This change makes that query explicit (improving performance), while
cleaning up the code to avoid an unnecessary query and read a little
more clearly.
After this change, the relevant database query takes 10s of milliseconds.
(imported from commit 020f5af5846c958386615e37ea9318383bf99ca0)
This essentially reverts d900957e468815bcb99de67d570dfd7ce4413220.
This code was consuming up to 50-100ms per client recipient of a
message, so for any messages that would go to 50+ browser windows /
mobile devices, it would take several seconds to run, during which
time Tornado would be completely blocked.
In the future, we can re-fix #174 using a cache of recently delivered
messages, so that this code block doesn't go to the database and thus
can run instantaneously.
(imported from commit bdfa1664210429411737f70cde54ab5a56525341)
This addresses Greg Price's feedback in #527. We now distinguish
between normal pushes, force pushes, and branch deletions.
(imported from commit 0fab6055f63ffc7e6df283b8bb8ed9971000e6d5)
feedback-bot and zephyr_mirror will need to be updated and restarted
when this is deployed to prod.
(imported from commit fe2b524424c174bcb1b717a851a5d3815fda3f69)
Luke Faraone
Zev Benjamin
Leo Franchi
Keegan McAllister
Tim Abbott
Jessica McKellar
Waseem Daher
Reid Barton
Jeff Arnold
Luke O'Malley
Tyler Hallada