Commit Graph

586 Commits

Author SHA1 Message Date
Tim Abbott 56e9ad230e [manual] Move our deployment scripts to scripts/.
This will require updating the post-receive code on git.zulip.net to
work.

(imported from commit 2e51fa2d7b891c1138d3f22ae534cfb8a6cf174c)
2013-10-28 10:54:48 -04:00
Zev Benjamin 7b6f6823a5 [manual] Change references to the humbug user, schema, and database to zulip
This commit must be simultaneously deployed on both staging and
prod0.  It also requires completely taking down the app.

To deploy these changes, do:
* check out this commit at /root/zulip on postgres0, postgres1, staging, and prod0

* stop the process_fts_updates job on postgres0 and postgres1
* stop the app on staging and prod0

* do a puppet apply on postgres0, postgres1, staging, and prod0
* move the new client certificates into place on staging and app
* move the new server certificates into place on postgres0 and postgres1
* reload the database config on postgres0 and postgres1 (this might
  actually require a restart)
* run tools/migrate-db on postgres0 as root

* do a deploy through this commit on staging and prod0
* start the process_fts_updates job on postgres0 and postgres1

* do a puppet apply on nagios

(imported from commit 819bdd14326c1425e2d3041a491a8ca3b9716506)
2013-10-26 04:16:27 -04:00
Steve Howell 13768495a6 Avoid long locks in process missedmessage_emails queue.
(imported from commit aed4a89244dce19085ed2ad89a7539ab297d8bc9)
2013-10-25 20:45:19 -04:00
Steve Howell 3cbb9f6b08 Avoid long locks in process digest_emails queue.
(imported from commit 49c7441463c5e63e52e5b099bd854bd4ae769f13)
2013-10-25 20:34:25 -04:00
Jessica McKellar 2be6c541cf digest emails: use a template filter to format new users, streams, etc.
(imported from commit e61a3dd6753db82da1b30c50e5bbf6273f302645)
2013-10-25 19:11:41 -04:00
Jessica McKellar d4b60e9304 Add a custom template filter for formatting displaying lists of items in emails.
We do this operation several times in the digest e-mails.

(imported from commit 7b906bad582c628d4461637dc7af374df349e09a)
2013-10-25 19:11:40 -04:00
Jessica McKellar a4a12cf407 digest: Add a queue processor that processes digest email requests.
(imported from commit 4f0abb70c92ae6bb47abb3792cca79d3f70e716a)
2013-10-25 19:09:45 -04:00
Jessica McKellar fac9d34d6c digest: Add a management command to enqueue digest email recipients.
(imported from commit 70ff2c7a4dae654f4077041c45e2154b3ac7afb7)
2013-10-25 19:08:47 -04:00
Jessica McKellar 772adc1969 digest: Add functions used to compute interesting traffic.
So far, we gather: "hot conversations", missed PMs, new streams, and
new users.

(imported from commit c3c723d0426cb55bd0e43917c67f93db9052f9ed)
2013-10-25 19:08:46 -04:00
Waseem Daher d44c0d5f5d Pass local_server into page_params.
(imported from commit 50dcee54644e0ee2ad8b13ec12e7134a9ab58d86)
2013-10-25 17:37:42 -04:00
Waseem Daher 66f48288b4 Rename LOCALSERVER -> LOCAL_SERVER.
(imported from commit b3abdd10d54d2ad7a9c463af9a291d2e2127707f)
2013-10-25 17:37:06 -04:00
Leo Franchi c12a97577e Rework local_settings_template.py to be clear about what the user needs to do
(imported from commit f69ea0d57e885509abc70dc4f2bee6e36169691f)
2013-10-25 16:35:27 -04:00
Luke Faraone a1b44986e0 [schema] Add a base_{api,site}_url property to our deployments.
This will allow us to redirect clients to the correct local site.

To apply this migration, just run:
     python manage.py migrate zilencer 0002

(imported from commit 7bd39b5f035145b6b52e1b2cb2ad5f6720d598ce)
2013-10-25 15:18:37 -04:00
Luke Faraone 238d19e1f1 Include a rest_dispatch in tornadoviews and call it from urls.py
(imported from commit aefe66b44be60b39d0f6bb3076502e964f1ed6bd)
2013-10-25 14:13:31 -04:00
Luke Faraone e552307511 Send feedback to a queue to be forwarded to staging.
(imported from commit 4a9a1bfc6c95763a816263a726cc61b3ca90bf15)
2013-10-25 14:13:30 -04:00
Luke Faraone 1fa65915f2 NACK on consumer failure
(imported from commit 1cad1096dd1a969b2551822575784c338a82f433)
2013-10-25 14:13:30 -04:00
Luke Faraone 9d4ef27b97 Make rest_dispatch accept a list of globals and move it into its own file.
We're going to be using this from other views, so make it more generic.

(imported from commit c2c4f64b2a212f8a254f75c21dc93d8fc3f8351a)
2013-10-25 14:13:30 -04:00
Luke Faraone 81d7dd1fda [schema] Support for authenticating Deployments via the API.
Here we introduce a new Django app, zilencer. The intent is to not have
this app enabled on LOCALSERVER instances, and for it to grow to include
all the functionality we want to have in our central server that isn't
relevant for local deployments.

Currently we have to modify functions in zerver/* to match; in the
future, it would be cool to have the relevant shared code broken out
into a separate library.

This commit inclues both the migration to create the models as well as a
data migration that (for non-LOCALSERVER) creates a single default
Deployment for zulip.com.

To apply this migration to your system, run:
   ./manage.py migrate zilencer

(imported from commit 86d5497ac120e03fa7f298a9cc08b192d5939b43)
2013-10-25 14:13:30 -04:00
Luke Faraone 52309b5789 Enable absolute imports for non-third-party code where it was omitted previously
(imported from commit 34856ac32c6b94b614273a6fe46c87a314058d9b)
2013-10-25 14:13:30 -04:00
Luke Faraone 70e161db01 Make settings.LOCALSERVER available as `localserver` inside templates
(imported from commit 4f0cfd1f1015061942f603cbc03a253a713a9f83)
2013-10-25 13:31:13 -04:00
Leo Franchi a9b9c0ec7b Add Email Mirror default settings and error out with readable warning if not configured
(imported from commit adac321d09561b51277ec1e314a1d480dc01e0c8)
2013-10-25 11:51:13 -04:00
Leo Franchi 2d276179d0 Refactor email-mirror to handle running on any machine
(imported from commit 2971449ceaacb564770e66874fc095f77e68d445)
2013-10-25 11:37:34 -04:00
Steve Howell 9979d884e5 Remove 06-unread.js.
This test has been broken for a couple months, and nobody has taken
ownership of fixing it.  It's always slow, sometimes it fails
randomly, sometimes it fails for things that aren't really problems,
and it's generally been way more trouble than it's worth.

(imported from commit 8080e81b226a372e763a2558f4e5668c3a4d087c)
2013-10-25 11:27:25 -04:00
Kevin Mehall 3bad5acc71 Fix upload auth redirect for user_activity.
Use rest_dispatch for upload auth redirect so it doesn't send the
long URL to user_activity.

(imported from commit ab327bbd529412e43eee6d109f8550180544dbbb)
2013-10-24 17:33:12 -04:00
Kevin Mehall 19b9b6e2ec Reduce the length of public upload URLs.
(imported from commit dc4d51da39b218068012ebad57e1f92cb50137c3)
2013-10-24 17:01:06 -04:00
Kevin Mehall 0a271f149b Enable authed file uploads for squarespace
(imported from commit c72e76d27f99cd7610eb6c93b57b8ff727b461a9)
2013-10-24 17:01:06 -04:00
Kevin Mehall 5e1caad88a Show upload filename in tooltip without other ugly parts of the URL.
This works for both public and authenticated uploads.

(imported from commit f9be340b53d0f72bf5b3c31a5d51a1fad80759be)
2013-10-24 17:01:06 -04:00
Kevin Mehall 0a3a22cb3d Support authenticated upload URLs.
Trac #1734

This is implemented by bouncing uploaded file links through a view
that checks authentication and redirects to an expiring S3 URL.

This makes file uploads return a domain-relative URI. The client converts
this to an absolute URI when it's in the composebox, then back to relative
when it's submitted to the server.

We need the relative URI because the same message may be viewed across
{staging,www,zephyr}.zulip.com, which have different cookies.

(imported from commit 33acb2abaa3002325f389d5198fb20ee1b30f5fa)
2013-10-24 17:01:06 -04:00
Kevin Mehall 473a98d0a4 Split out common code for existing and authenticated upload systems.
(imported from commit bf513eaff90ae7c4a0be825317690058f9f54621)
2013-10-24 17:01:06 -04:00
Kevin Mehall fb4eae9e2b Include realm ID in S3 metadata on uploaded files.
(imported from commit 7d41ec94991bd257485116791a0d1ef2c22d9b37)
2013-10-24 17:01:05 -04:00
Kevin Mehall e3b2e7ece2 Allow domain-relative and fragment links
(imported from commit 0a15eeb35ad709977f3b2e6c550c3b78f1b4b402)
2013-10-24 17:01:05 -04:00
Zev Benjamin ec21275b19 Move rabbitmq username into settings.py
(imported from commit f9a8f80f53b8774781b8558444e601567cb59835)
2013-10-24 16:40:38 -04:00
Zev Benjamin 98ec953e2e bitbucket: Include the branch name in the topic
(imported from commit b9074dc509818b4db0c4fa10f6200935c4738b64)
2013-10-24 16:25:23 -04:00
Leo Franchi ae104ace25 Only load APNS cert if it exists
(imported from commit 8505c9a32feb4c7470dfde6c4ceaaa0a8e8f33a1)
2013-10-24 15:55:42 -04:00
Tim Abbott 4b45c8bf96 Fix update_message_flags tests to test with real messages.
As it turns out, some of these tests used message IDs 1 and 2, which
Hamlet didn't even necessarily receive as the messages to update --
which meant that they previously updated 0 messages and returned
success.  So those tests started failing when I added a check for not
updating anything in the update_message_flags backend -- and this
commit fixes the tests to actually update a nonempty set of messages.

(imported from commit 9034b415d4862216a266416a8e509d987050ffd7)
2013-10-24 15:44:17 -04:00
Leo Franchi dc8bfde0b4 Add a check_apns_tokens manage.py command to flush old tokens
(imported from commit 15dcfaa1967bd749b582882c4a816a457ca6cc12)
2013-10-24 14:54:31 -04:00
Leo Franchi 410ee44eb6 Send users push notifications when they miss messages
(imported from commit 6c54fe44a82c5796268e56d3f5577bf4cfc8163a)
2013-10-24 14:54:31 -04:00
Leo Franchi 05564ec0e2 [schema] Add a config option for enabling/disabling push notifications
(imported from commit 5649188cd5e0e4edb6e71559ec7d7fc5e92efaf8)
2013-10-24 14:54:30 -04:00
Leo Franchi 7961d4f6b3 Add a push notification module to handle mobile client notifications
(imported from commit 3061a6e2d845226d3dce5bb262deb3a896e54f07)
2013-10-24 14:54:30 -04:00
Leo Franchi 507a3ecab3 Allow overwriting of APNS as iOS will send it on each startup
(imported from commit 4a22f9f37d550b6c2e06f543434466453e196458)
2013-10-24 14:54:30 -04:00
Leo Franchi 1d82704b8d [schema] Add a last_updated field to AppleDeviceToken
(imported from commit 2ea91e462773c50591585a610b0de63625ad2b7e)
2013-10-24 14:54:30 -04:00
Leo Franchi 13b452ddf8 Add a method=? parameter rest_dispatch to support PATCH-via-POST
(imported from commit 66f7011d6920639abc307ca85708bc958f3bb0e3)
2013-10-24 10:22:06 -04:00
Leo Franchi a2a1867f25 Only render tweets if we have credentials
(imported from commit 46c59f430aeae87a832f33d2ccb85209f9caec74)
2013-10-24 10:19:23 -04:00
Tim Abbott 1b381b343d Allow starring historical messages.
This has a small bug where we don't actually filter the message out of
the home view; fixing that requires adding an index on the "flags"
field of UserMessage.

(imported from commit 492c99d0a8e87b253e577be6564bec12099bd8e9)
2013-10-23 18:51:47 -04:00
Zev Benjamin 89bcabbb65 Disallow XHR streaming when running the test suite
There seems to be some sort of bug involving PhantomJS and XHR
streaming messages.  When successive pages are loaded that use XHR
streaming, PhantomJS seems to think the second one never finishes
loading and therefore hangs.

(imported from commit db93b4cab816f1fdc3f3f543c9394b1cba8abedb)
2013-10-22 18:45:11 -04:00
Zev Benjamin b1eebc0b84 Deliver messages inside Tornado when running tests
This saves us from having to run queue processors while testing.

(imported from commit 927bfa497a47b34603761a148c57d82b2f34f813)
2013-10-22 18:45:11 -04:00
Zev Benjamin a04ac35012 casper: Marginally improve page loaded detection
We really should be setting a variable in Javascript to indicate that
we've finished loading, but this hasn't bitten us yet.

(imported from commit ee1f7c76d9f3c482561cc5c44b81537c7e9636be)
2013-10-22 18:45:11 -04:00
Zev Benjamin 3c73468ba8 Disable unsupported SockJS transports
Because our authentication system reads cookies from the initial
connection attempt, several SockJS transports can't be used.

(imported from commit 34b9571225d39072985b8223fb12c43c7235841f)
2013-10-22 18:45:11 -04:00
Zev Benjamin 5979af3a45 [manual] Add asynchronous message sender via sockjs-tornado
New dependency: sockjs-tornado

One known limitation is that we don't clean up sessions for
non-websockets transports.  This is a bug in Tornado so I'm going to
look at upgrading us to the latest version:
https://github.com/mrjoes/sockjs-tornado/issues/47

(imported from commit 31cdb7596dd5ee094ab006c31757db17dca8899b)
2013-10-22 18:45:11 -04:00
Steve Howell 0662300e9e Remove /json/subscriptions/list endpoint.
(imported from commit db2c38f6b12ebee5250737a6cdd193863a4605af)
2013-10-22 15:51:19 -04:00