Commit Graph

122 Commits

Author SHA1 Message Date
Luke Faraone f20f605376 [manual] Add file upload support.
This allows users to drag and drop content onto the compose box, storing
their data in Amazon S3.

New dependencies:
 - python-boto

(imported from commit 339874e483db5c36312c9ceae56db29da6ca0d99)
2013-04-05 13:07:13 -07:00
Luke Faraone f4d00b6af9 [manual] Push new users' data to MailChimp.
This creates a new management command, subscribe_new_users, which should be
run as a daemon process. When new users are created, an event is passed to
RabbitMQ including the following data:
 * Email
 * Full name
 * IP address of the person who confirmed registration
 * Time of registration confirmation

MailChimp strongly encourages the collection of the last two to enable
responses to abuse requests, and providing more data lowers the chance that
we could get banned from their service if complaints do occur.

To use this commit, you need to install the "postmonkey" module from
PyPI.

(imported from commit 20c628c3fa8bb985aaead85a80ad3b38bf94b9dc)
2013-04-05 10:22:26 -07:00
Tim Abbott 7ce6154464 Set UserProfile to be the AUTH_USER_MODEL.
(imported from commit 92bf66b3e6a8ed06054bd491bcc96ef422a1fbe1)
2013-04-04 15:45:44 -04:00
Tim Abbott 24a464cee1 Add IP addresses for app frontends to ALLOWED_HOSTS.
(imported from commit f5cc75f0ac3851bfe9a554d08875dcb5a19a9dc5)
2013-04-03 11:18:12 -04:00
Keegan McAllister 9dd987be7e Precompile Handlebars templates in production
Fixes #908.

(imported from commit 30458e9bea8117d1fa15dc962bb1a495f22672cc)
2013-04-02 14:43:59 -04:00
Keegan McAllister 17d5406b55 [manual] Fetch Handlebars templates using Ajax
...rather than embedding them into index.html.

This is only acceptable for dev, but the next commit adds an alternative
mechanism for prod.

There isn't actually a manual deployment step here.  However, this commit won't
work on staging / prod without the next one (since we don't serve
zephyr/static/templates in prod).

(imported from commit dce7ddfe89e07afc3a96699bb972fd124335aa05)
2013-04-02 14:43:58 -04:00
Zev Benjamin 68904bef00 Lower the client-side polling timeout for get_events now that we have a server-side timeout
(imported from commit b3b3a2d4083f22b0fdb876f2d4e97b2be4bda79a)
2013-03-29 11:11:34 -04:00
Zev Benjamin 6cc70d94f6 Add register() call to event system
(imported from commit 0c9fbfec1866591b2169ce2da2bc2af6003f8f31)
2013-03-28 16:57:47 -04:00
Keegan McAllister aa3c446665 Generate source maps from Closure Compiler
(imported from commit 0e4de860b1dba85aa43b60a2c819ac44403186c5)
2013-03-28 12:11:23 -04:00
Keegan McAllister f76984b4ef Switch to Closure Compiler for minifying JS
I've already confirmed that java is installed on both staging and prod.

(imported from commit a6354e60d529a72583e1e1bc14468d04697960b9)
2013-03-28 12:11:23 -04:00
Tim Abbott 3d39341fc2 Fix logging configuration for Django 1.5.
Apparently, something in Django 1.5's changes to their default logging
setup resulted in the logger 500 errors (logged in
django.core.handlers.base.handle_uncaught_exception) from reaching the
root logger -- they stopped at propagating at the 'django' logger.  We
deal with this by making our logging system handle those events in the
'django' logger ourselves (and making the related changes needed to
ensure that we still log to server.log and the console everything
logged by our own humbug.requests logger and anything that falls
through to the root logger).

This requires updating the mechanism we use in test_settings.py to
silence our request logging, since now the 'humbug.requests' logger is
being re-initialized by the Django logging setup, which runs after
test_settings.py.

While we're at it, set propagate=False in the commented-out
'django.db' logging configuration (previously, queries would be logged
twice).

(imported from commit 32af29084e52be1ba6f92a7952c3a3946925b46b)
2013-03-28 07:36:10 -04:00
Zev Benjamin d1b9ab9fd8 Move SERVER_GENERATION definition to settings.py
(imported from commit 4be2b912a63f49f525c66f46db0bc5c466ad364f)
2013-03-27 14:15:03 -04:00
Tim Abbott 2aae6190d2 [django 1.5] Set ALLOWED_HOSTS variable in settings.py.
This only does something if DEBUG=False, but it's now required that
you set this on Django 1.5 or the server will silently serve up 500s
for every request (not the best failure mode).

(imported from commit fa226c644770c468d73143c8a49d5d29d282df27)
2013-03-27 08:19:26 -04:00
Zev Benjamin f3f1205522 Use verify-full when connecting to the database
Now that we can use our servers' DNS names internally, using
verify-full gives us a little bit of extra security.

(imported from commit 3a3715fa8a59851d4543112a55b5c6b24981442e)
2013-03-19 12:15:24 -04:00
Tim Abbott 34021ffc29 debugging: Add commented-out code in settings.py to log all queries.
This is often useful when working on a local development system and so
seems worth putting in the code, but is so verbose that it probably
doesn't make sense to have on by default in development.

(imported from commit ddb7ae4c83136f96d69368a245ed64e7daf66f34)
2013-03-18 16:15:11 -04:00
Keegan McAllister ff745e46ae Use Postgres on all Linux dev machines
(imported from commit fe4dcc186debe726ffc146881dd8c6022c192c8b)
2013-03-18 15:14:40 -04:00
Keegan McAllister 45eb9bcf47 tests: Disable tutorial through the server side variable needs_tutorial
This fixes a nondeterministic test failure for me.

The first message sent in the test suite appears to get dropped.  I don't know
why this is, and I'm pretty sure it was an existing bug.  This message used to
be the one disabling the tutorial, which might explain why that didn't always
work.

Regardless, this commit at least makes the test suite usable, and we can work
on fixing that bug later.

(imported from commit 063e40871b9883e3a6dab93a4e0a51c5b2dae4b7)
2013-03-18 13:46:46 -04:00
Tim Abbott 07b72c4901 settings: Add docs on how to properly test our email handler.
(imported from commit e6ad4f517169ca47a32bb853a30aab7a634e7979)
2013-03-15 14:53:17 -04:00
Zev Benjamin 09cf339c2b blueslip: Handle exceptions from jQuery event handlers and $(document).ready functions
We treat these exceptions the same way we treat fatal errors: report
the error message to our server and then allow the exception to reach
the top level.

We could also override document.onerror, but don't.  There are a
couple of ramifications of this:
* Exceptions caused by event handlers directly attached to DOM
  elements aren't handled
* Exceptions caused by code at the top level that triggers an error
  (such as parse errors in our Javascript files) aren't handled

The reason we don't override document.onerror is because the
document.onerror handler has a limited interface and doesn't receive
the exception object.  It only gets the message, file, and line
number of the error.  Additionally, exceptions that we allow to
propogate out of blueslip trigger an onerror event when they're never
caught.  In order to avoid handling the error twice (once by blueslip
and once by the onerror handler), we'd have to encode the fact that
the error has already been handled in the error message, which is
pretty ugly.

(imported from commit 7f049ae519dc198a9f7cfd41fd5dd18e584bd061)
2013-03-13 10:55:34 -04:00
Tim Abbott 710358db8c Set the database cache's to essentially never timeout.
(imported from commit f6fdbfe52536c5458130db3a907b8b8f81163fa4)
2013-03-12 11:16:58 -04:00
Tim Abbott dd8759acc3 Increase the third_party_api_results cache size.
(imported from commit 2555dcb3913650d72e307017df721b7665ff2e4b)
2013-03-11 15:24:59 -04:00
Tim Abbott 967743ddab Fix database cache settings for !DEPLOYED systems.
(imported from commit 0805c17a6bc5d0f1438d6fd1b1ce739d1162d09e)
2013-03-11 15:24:59 -04:00
Zev Benjamin dc0913077a Add a new frontend error-reporting system
The new system, called blueslip, makes errors fatal when in debug
mode and only output a message when running in production.  In the
future, it could also send user errors back to us automatically.

(imported from commit 1232607c0311e885c8b5a5e8a45ffb28822426e0)
2013-03-11 13:22:12 -04:00
Tim Abbott d679a72952 [manual] Cache results of the Twitter API in the database.
This should substantially improve the repeat-rendering time for pages
with large numbers of tweets since we don't need to go all the way to
twitter.com, which can take like a second, to render tweets properly.

To deploy this commit properly, one needs to run

./manage.py createcachetable third_party_api_results

(imported from commit 01b528e61f9dde2ee718bdec0490088907b6017e)
2013-03-11 13:15:55 -04:00
Tim Abbott e63033f8b1 settings: Decrease duplicated code in database configuration.
(imported from commit cb89fd7f986e0cf2a5598eedf799da8fa99131da)
2013-03-06 11:36:15 -05:00
Luke Faraone 9c1b2665c0 Rotate database password for local testing.
This does not affect any deployment.

(imported from commit 77d722d19cfe64169055e32a9bf5cd565772f03f)
2013-02-28 15:40:21 -05:00
Luke Faraone 0fe0cf0ffb [manual] Implement backend support for authenticating a user via Google.
This code adds a dependency on python-django-auth-openid, installable as
django-openid-auth from PyPI.

On prod, one needs to run a syncdb in order to create the required
tables. A database *migration* is not required, as these are new tables
only.

(imported from commit c902a0df8d589d93743b27e480154a04402b2c41)
2013-02-27 10:16:54 -05:00
Keegan McAllister c5bab96587 Only use PipelineCachedStorage when not DEBUG
(imported from commit 6fea56b15122b9d54184f7c368f7e0113d581424)
2013-02-25 17:19:20 -05:00
Keegan McAllister 49e16b0ba6 Add portico and misc CSS and JS to Pipeline
Fixes #963.

(imported from commit 63ec313a41fd13350657c78356efc16422a5fff1)
2013-02-25 16:18:35 -05:00
Keegan McAllister d31eab9325 Include hash in minified filenames to avoid browsers using stale files
Fixes #853.

(imported from commit f85ebe52df754f488a29c2ad814d582b78aadd14)
2013-02-25 16:18:35 -05:00
Keegan McAllister 1975d7aa16 Set PIPELINE in settings.py so urls.py can read it
(imported from commit 641ff07e83ffa4b5271da311c0660d59e73b3f8f)
2013-02-25 16:18:35 -05:00
Zev Benjamin 814e3d6385 Implement a message list structure for storing messages and related information
(imported from commit 171de93636a215d9357c7fc4ee8fb71696d23fb5)
2013-02-21 13:27:45 -05:00
Waseem Daher b51dc36667 Add a tutorial.js file.
(imported from commit 58b72d08c0cd7815c3c54bd37c4a8033f32cc7ad)
2013-02-20 23:04:49 +00:00
Reid Barton 59dab21fcd Render recent dates as weekdays, part 1.
This commit just moves time rendering logic to its own file, and does
not make any functionality changes.

(imported from commit d111d03c6abc8d9550fcf65e4f89eab8056d1ed4)
2013-02-19 15:58:25 -05:00
Keegan McAllister bb5f59d310 Use white text for recipient labels on streams set to dark colors
Fixes #577.

(imported from commit 0518e33b96bc0028fc80d533f6b8ec35fd5cdc04)
2013-02-19 15:33:35 -05:00
Leo Franchi 83011f7f47 Show a user activity list in the sidebar
(imported from commit 95aaa55c7e4cc39f844518b5308866bedf2cd1c5)
2013-02-11 18:05:57 -05:00
Jacob Hurwitz 61acc58947 Add a notifications bar when there are additional messages out of view
(imported from commit 38fa78f63fd520d0b1f09921ba064cba010a6f99)
2013-02-02 01:16:25 -05:00
Keegan McAllister 638b8d29bc Enable testing minified files in dev
(imported from commit 257b8547849a85c447319d3d211f2c989616ce64)
2013-01-31 15:41:01 -05:00
Keegan McAllister 6990260b59 [manual] Minify JavaScript and CSS in production
Manual deployment steps: The same Nginx reload as for "Get rid of the
static-access-control mechanism".  If deploying both commits at once,
just do it once.

(imported from commit dd8dbbf14b95fce0a4b6f66f462fa0a6b50bfb8c)
2013-01-31 15:41:01 -05:00
Zev Benjamin 11d8cdef6f settings.py: Add 'schema' to database configuration
Django doesn't use this setting, but South consults it when
inspecting tables for their constraints.  The fact that we store our
tables in the 'humbug' schema was causing South to fail to find our
table constraints (it was looking in the 'public' schema) and
therefore throw an exception when we try to remove the unique
constraint in migration 0002.

(imported from commit 4230338a7b78329a759339b2f9fcd277137b7f32)
2013-01-29 12:16:32 -05:00
Zev Benjamin a3037ea91a Revert "Set a wildcard subdomain for the session and csrf cookie domains on staging"
This was to support get_updates sharding, which we never fully
implemented.  We can recommit this change later if we choose to bring
the feature back.

This reverts commit fda2d99d9e9a07951d11fcd9fc61cf229988f471.

(imported from commit aec8203c8d8a94dd6f30089aeee22814d1595fc5)
2013-01-28 13:11:58 -05:00
Zev Benjamin f6547dedb1 Set a wildcard subdomain for the session and csrf cookie domains on staging
(imported from commit fda2d99d9e9a07951d11fcd9fc61cf229988f471)
2013-01-17 21:40:46 -05:00
Jessica McKellar 8d1ccad29b Prominently display the user in Django 500 emails.
As a side-effect of customizing the e-mail, this also makes the host
on which the error happened a part of the subject line.

(imported from commit 7d5e9ad108b48fd34528512c5955567119935d4e)
2013-01-17 09:47:10 -05:00
Tim Abbott cb0de0fc60 Fix logging in with email addresses long than 30 characters.
(imported from commit 81d0c329d0c7403986e2379accbe2f1be3c731e3)
2013-01-16 17:02:06 -05:00
Keegan McAllister b5a0147e26 Log events to a file named after today's date
We need this so that we can safely expunge old events without interfering with
the running server.  See #414.

(imported from commit 4739e59e36ea69f877c158c13ee752bf6a2dacfe)
2013-01-15 14:37:36 -05:00
Tim Abbott e592e71515 [manual] Use rabbitmq queue to process UserActivity.
Before this is deployed, we need to install rabbitmq and pika on the
target server (see the puppet part of this commit for how).

When this is deployed, we need to start the new user activity bot:

./manage.py process_user_activity

in the screen session on the relevant server, or user_activity logs
won't be processed (which will eventually result in all users getting
notifications about how their mirrors are out of date).

(imported from commit 44d605aca0290bef2c94fb99267e15e26b21673b)
2013-01-14 13:28:23 -05:00
Tim Abbott 3b7d61e45f tornado: Get User and UserProfile objects from a memcached.
This commit has the effect of eliminating all of the non-UserActivity
database queries from the Tornado process -- at least in the uncached
case.

This is safe to do, if a bit fragile, since our Tornado code only
accesses these objects (as opposed to their IDs) in a few places that
are all fine with old data, and I don't expect us to add any new ones
soon:

* UserActivity logging, which I plan to move out of Tornado entirely

* Checking whether we're authenticated in our decorators (which could
  be simplified -- the actual security check is just whether the
  Django session object has a particular field)

* Checking the user realm for whether we should sync to the client
  notices about their Zephyr mirror being up to date, which is quite
  static and I think we can move out of this code path.

But implementation constraints around mapping the user_ids to
user_profile_ids mean that it makes sense to get the actual objects
for now.

This code is not what I want to do long-term.  I expect we'll be able
to clean up the dual User/UserProfile nonsense once we integrate the
upcoming Django 1.5 release, with its support for pluggable User
models, and after that I change, I expect it'll be fairly easy to make
the Tornado code only work with the user ID, not the actual objects.

(imported from commit 82e25b62fd0e3af7c86040600c63a4deec7bec06)
2013-01-11 16:11:07 -05:00
Tim Abbott e15c575977 Cache Django session objects using memcached.
(imported from commit 531ea34d5781b27401a7e2d90a0be99927d0bae5)
2013-01-10 16:59:36 -05:00
Jessica McKellar 2c6ac969ff Temporarily increase get_updates polling timeout to deal with increased load.
(imported from commit 8477c829f1f69299ddb9c22dee722825dd673ee7)
2013-01-09 21:07:43 -05:00
Luke Faraone 0646cd7e21 [manual] Convert Humbug to use Django South.
This was done using instructions provided by the South authors:
	<http://south.readthedocs.org/en/0.7.6/convertinganapp.html>

This adds a dependency on python-django-south >=0.7.5. Now when you are
reinitializing the database, you need to run "./manage.py migrate --all"
before running populate_db.

When deploying this commit onto existing servers, you need to run these
commands manually:

    ./manage.py syncdb
    ./manage.py migrate zephyr 0001 --fake
    ./manage.py migrate confirmation 0001 --fake

These do *not* need to be run on new databases, only on existing ones.

(imported from commit f24cff421a6be9ab9cf4c4342565c484ac336e2d)
2013-01-08 18:14:39 -05:00