Commit Graph

265 Commits

Author SHA1 Message Date
Leo Franchi d7a33485ad Register a tornado atexit handler to disconnect from rabbitmq
(imported from commit b70650070f1df548794a9e3ff2948d134fd0c5de)
2013-04-16 11:49:03 -04:00
Jessica McKellar 7175dc534a Send invitation e-mails asynchronously through RabbitMQ.
This avoids 10s of seconds of delay when you invite several people at
once through the web UI.

(imported from commit 75acdbdb04caf62bbb08affc7796330246d8a00e)
2013-04-10 16:57:49 -04:00
Zev Benjamin a2010871e3 Make subscription properties less free-form
(imported from commit eda607c2abfa51d2dadddc7b9ecba3e2d0b5be4d)
2013-04-10 16:11:27 -04:00
Leo Franchi 57974787b3 Set messages to read in populate_db
(imported from commit 726d2015dbb446f56e5a640b44107a3713917ed5)
2013-04-09 09:59:31 -04:00
Tim Abbott 807b6396e0 realm_stats: Only count messages sent by humans in most stats.
For our primary measures of user engagement, messages sent by bots can
confuse the picture (e.g. a realm could be dead, but not appear to be,
because they didn't bother uninstalling their github and jenkins
hooks).  So it's best to leave those out of our main stats.

(imported from commit 4d0f0e6442093daab164d0ed016fff1d1aa906c7)
2013-04-05 17:28:04 -04:00
Tim Abbott e1173b08df Fix check for non-default Subscription colors.
(imported from commit b904220f7b1a013081b1d680d98b15a7532f0839)
2013-04-05 17:28:04 -04:00
Luke Faraone f4d00b6af9 [manual] Push new users' data to MailChimp.
This creates a new management command, subscribe_new_users, which should be
run as a daemon process. When new users are created, an event is passed to
RabbitMQ including the following data:
 * Email
 * Full name
 * IP address of the person who confirmed registration
 * Time of registration confirmation

MailChimp strongly encourages the collection of the last two to enable
responses to abuse requests, and providing more data lowers the chance that
we could get banned from their service if complaints do occur.

To use this commit, you need to install the "postmonkey" module from
PyPI.

(imported from commit 20c628c3fa8bb985aaead85a80ad3b38bf94b9dc)
2013-04-05 10:22:26 -07:00
Tim Abbott fdefa06190 Eliminate use of old StreamColor model.
(imported from commit c72a06bdc44f30fb6bca299463e259262367e8c2)
2013-04-04 17:48:51 -04:00
Tim Abbott 0ee684a4b5 [schema] [manual] Add colors to the subscription model.
This is preparatory for removing the StreamColor model, so we also set
things up so anything changing the StreamColor model changes the
Subscription model too.

The manual task is to run the copy_colors.py management command after
deployment to each of staging and prod.

(imported from commit 1be7523ca59f5266eb2c4dc2009e31209ed49635)
2013-04-04 14:17:01 -04:00
Tim Abbott 1a4047cb95 Remove User migration tools.
(imported from commit 928124d5a8056d5cb1736cfc4c7158c228437c1b)
2013-04-02 12:57:15 -04:00
Tim Abbott 1cec86eb2d [manual] Remove now-unused User model.
I think all that one needs to do to deploy this commit is on developer
laptops, run `generate-fixtures --force`.

(imported from commit 34916341435fef0875b5a2c7f53c2f5606cd16cd)
2013-04-02 12:57:10 -04:00
Tim Abbott 5dbe8b4c17 [manual] Authenticate using a user_profile as request.user.
When this is deployed to staging, we need to run

./manage.py logout_all_users --realm=humbughq.com

When this is deployed to prod, we need to run

./manage.py logout_all_users

(imported from commit d6c6ea4b1c347f3d9122742db23c7b67767a7349)
2013-04-02 12:07:08 -04:00
Tim Abbott 712d931350 Remove banish_broken management command.
The bug it addresses is long fixed, and it's buggy.

(imported from commit b198a30f56f950168e3662dcea2bab7d908631f9)
2013-04-02 12:07:08 -04:00
Tim Abbott 2fcbb31dcf Access the UserProfile's new is_active field rather than User's.
(imported from commit ed5bdaf3e3d6d20bfb741efdac48d30482ab9ef7)
2013-04-02 12:07:08 -04:00
Tim Abbott 1443edce00 Access the UserProfile's new email field rather than using User.
This is preparatory for stopping using the User model.

(imported from commit a1b0808c8cc2ddd19a25163f91c4f18620c9ce90)
2013-04-02 12:07:08 -04:00
Tim Abbott 3a35c5b00c Add management command to logout all users.
This is intended to be used logging out users during our deployment of
the UserProfile merge, but it could be useful for other things too.

(imported from commit bfe896d854f997f7a4d06e5bc0f19ec5b1aa5e69)
2013-04-02 12:07:05 -04:00
Tim Abbott 303e77c07d Fix clearing of user sessions.
Previously, we weren't clearing the users out of memcached (we just
killed them in the database), so in fact users were not logged out
when we deactivated them for an hour (when the memcached caches would
expire).

(imported from commit 0f0a2f70e003c184106c73b22b876f57c1ef3371)
2013-04-02 11:56:31 -04:00
Tim Abbott b82edb6fd6 [manual] Add User fields to the UserProfile model.
And keep the fields updated, by copying on UserProfile creation and
updating the UserProfile object whenever we're updating the User
object, and add management commands to (1) initially ensure that they
match and (2) check that they still match (aka that the updating code
is working).

The copy_user_to_userprofile migration needs to be run after this is
deployed to prod.

(imported from commit 0a598d2e10b1a7a2f5c67dd5140ea4bb8e1ec0b8)
2013-04-01 14:34:25 -04:00
Tim Abbott 7bae0bb75a populate_db: Fix User/UserProfile confusion in personals sampling.
We were incorrectly using User objects, rather than UserProfile
objects, for fetching Recipient objects for generated messages.

(imported from commit c3dfe52f4e0a68400e22ca49293b5bf2d6986402)
2013-03-29 16:18:38 -04:00
Tim Abbott 9317035fc7 Add hashed_password argument to do_change_password.
This way we're not directly manipulating user.password() in random
management commands.

(imported from commit e6e32ae422015ab55184d5d8111148793a8aca36)
2013-03-29 16:18:38 -04:00
Tim Abbott 80747567c0 Remove test_deactivate.py management command.
It is a less-featureful duplicate of deactivate_user.py.

(imported from commit d40fe8cc30284745aa267601b4855572bbe0e26a)
2013-03-29 16:18:38 -04:00
Tim Abbott 198480ef99 Use get_user_profile_by_email more consistently.
The previous situation was bad for two reasons:

(1) It had a lot of copies of the code, some of them missing pieces:
  UserProfile.objects.get(user__email__iexact=foo)

This was in particular going to be inconvenient since we are dropping
the __user part of that.

(2) It didn't take advantage of our memcached caching.

(imported from commit 2325795f288a7cf306cdae191f5d3080aac0651a)
2013-03-29 16:18:38 -04:00
Tim Abbott 2223b60dc0 Switch functions like do_activate_user to accept UserProfiles.
Only a few of them took a User as an argument anyway.

This is preparatory work for merging the User and UserProfile models.

(imported from commit 65b2bd2453597531bcf135ccf24d2a4615cd0d2a)
2013-03-29 16:18:37 -04:00
Zev Benjamin c6df5af84d Import tornado ioloop directly instead of passing it from runtornado
(imported from commit ce901d7daca81bb1682827175bdcd83d3e5c2344)
2013-03-28 16:57:48 -04:00
Zev Benjamin e4fba59538 Make event queues persistent across Tornado restarts using cPickle
(imported from commit 1434d1e9d394d725827b1740c0c07249d5e716ed)
2013-03-28 16:57:48 -04:00
Zev Benjamin 401fa6063e [manual] Add get_events URL routing
The new nginx configuration file needs to be copied to
/etc/nginx/humbug-include and nginx needs to be restarted when this
commit is deployed.

(imported from commit 6c43f3c2c7a6acee6a852c672c96a38bda01dd0d)
2013-03-28 16:48:52 -04:00
Zev Benjamin 8f4eaa63ad Initial event system implementation
This version has several limitations that are addressed in later
commits in this series.

(imported from commit 5d452b312d4204935059c4d602af0b9a8be1a009)
2013-03-28 16:48:14 -04:00
Keegan McAllister 1ecc063d0f Split out source map processing into a library
(imported from commit 345efcc703dc1049e31fd38a6a062bf39a589eb6)
2013-03-28 14:56:52 -04:00
Tim Abbott 7685e47f07 Fix Tornado idle time logging.
When we added rabbitmq usage within Tornado, we inadvertently caused
the Tornado ioloop to be initialized in runtornado.py's imports,
before we overwrote the _poll method.  The end result was that we
weren't running the our instrumented Tornado poll function.

Fix this by moving that code to its own file which we import at the
top of runtornado.py, and adding comments documenting the situation so
we don't break this in some future import reorganization.

(imported from commit 016717476f10566fef4ed2b656f29f865d2084db)
2013-03-28 14:40:49 -04:00
Keegan McAllister f42e841ffd Add a command to annotate a JavaScript traceback with source file locations
(imported from commit 1a4d656438990b1cbefbdecc5d403573d6c5dd06)
2013-03-28 12:11:23 -04:00
Luke Faraone 0d51e59fd5 Implement URLs for API redesign.
(imported from commit 2020491a737ec4c1e99a63f84eb6cfc594a2dd56)
2013-03-28 07:57:38 -07:00
Luke Faraone b63e5e3235 Refactor runtorando AsyncDjangoHandler URLs into a list.
(imported from commit f5c65f69dd2ccb5a0ba6339b28b8c486cde2c7ff)
2013-03-28 07:53:36 -07:00
Tim Abbott a4383f8264 Remove batch_bulk_create helper function.
The behavior we require is now standard in Django 1.5.

(imported from commit 509adb0bd371081a2dad7d4a04d24fcffbda7cdc)
2013-03-28 07:36:10 -04:00
Tim Abbott e2d010ef2d Fix most unnecessary database queries in huddle creation.
This saves 2 database queries per user in the huddle when sending the
first message to a particular huddle.

(imported from commit f71aa32df846fb4b82651a93ff9608087ffcaa5a)
2013-03-28 07:36:10 -04:00
Tim Abbott 36f09729e6 Run the memcached cache filling jobs in separate processes.
This will significantly improve the speed with which the memcached
caches are filled.

(imported from commit 445fec710a00b3d19b7962beddf89d4a75322eab)
2013-03-27 12:59:23 -04:00
Tim Abbott 144987e581 logs: Provide total time passed for longpolled requests.
Also improve display of times passed -- we now use display short times
in milliseconds for easier reading.

(imported from commit 08e1e7e6acbef48453080864946f7602a3395e7c)
2013-03-27 08:19:27 -04:00
Tim Abbott f3ad2d7a6b Centralize dispatch logic for which queue processor to use.
Previous we had around 4 copies of the logic for deciding whether we
should publish data via a SimpleQueueClient queue, a
TornadoQueueClient queue, or to directly handle the operation, which
resulted in their getting out of sync and buggy (see e.g. the previous
commit).

We need to add a lock around adding things to the queue to work around
a bug with pika's BlockingConnection.

I should note that the previous logic in some places had a bunch of
tests of the form "elif settings.TEST_SUITE" for doing the work that
would have been done by the queue processor directly; these should
have just been "else" clauses -- since we generally want that code to
run on development environments whether or not the test suite is
currently running.

(imported from commit 16bdbed4fff04b1bda6fde3b16bee7359917720b)
2013-03-27 08:19:27 -04:00
Tim Abbott fde2243dbc [django 1.5] Update read flags handling to use new Django 1.5 syntax.
(imported from commit af1e2608b6d6d3677bf6266980d8dc77739aa4ba)
2013-03-27 08:19:26 -04:00
Jessica McKellar 53ec385d0b create_user: ask for realm as part of user creation.
With open realms, you can't infer the realm from the e-mail address.

(imported from commit c05afa586f121a38673f9e62281cf26c8590570b)
2013-03-23 00:19:37 -04:00
Jessica McKellar f87ffd0703 create_user: fix missing spaces in options error message.
(imported from commit 25ea284b17dfb3160aca336e31203991f70994d8)
2013-03-23 00:19:37 -04:00
Jessica McKellar e5f1dad409 create_user: remove unused imports.
(imported from commit b2dd10f73fb0285c2c92ab56b1111cf9e859cedf)
2013-03-23 00:19:37 -04:00
Jessica McKellar ea11ce4ae6 create_user: Add Terms of Service warning.
(imported from commit bdb31aebad397e98db77d3ea95ef17ac9dd4737f)
2013-03-23 00:19:37 -04:00
Leo Franchi f64bc59dfb Automatically register consumer once asynchronous connection is completed
(imported from commit 3d860a9b79a42beffbd3b73f49aa2c25986dd3c6)
2013-03-22 15:55:49 -04:00
Leo Franchi c346abe6e2 Send Tornado callback notifications via RabbitMQ
(imported from commit 83814d8c6b78fe725aedea9d98fb588ed31123e7)
2013-03-22 15:55:40 -04:00
Keegan McAllister 23b8833ec5 Handle Tornado callback notify in a more generic, less HTTP-centric way
(imported from commit 85a74d1b40461236c4c95ad688e9796ab50f0bbf)
2013-03-22 15:51:23 -04:00
Keegan McAllister 591816d80e Use TornadoQueueClient in runtornado
(imported from commit cc4d5505987dc2826d177ff93d037b17f78072b8)
2013-03-22 15:39:07 -04:00
Leo Franchi 7837e36d0c Clean up some code by consolidating
(imported from commit 3978da70286174ad3666d08bcc8db2eb28446399)
2013-03-22 09:49:54 -04:00
Jessica McKellar 4f1bf1b2f2 realm_stats: restrict some usage stats to only active users.
They are more meaningful this way -- the fact that your bots that
never log in or inactive users don't have colored streams shouldn't
impact that statistic.

(imported from commit b39debda338cbbad06957bc969b42862a888026a)
2013-03-21 17:38:20 -04:00
Jessica McKellar 85086b6662 realm_stats: Also print enter-sends stats.
(imported from commit 34f9d9916e11ac2e2748ecbb8eb4f9a34a110340)
2013-03-21 17:38:20 -04:00
Leo Franchi 6d6c3364dc Rewrite message flag handling to fork out to a subprocess for batch handling
(imported from commit 1ef846f542950cabf32f8b176f5591cf5794a0ff)
2013-03-19 09:55:12 -04:00