Commit Graph

7054 Commits

Author SHA1 Message Date
Steve Howell e6991c299d Add tools/migrate-db-osx.
(imported from commit b422f7d4e93655b77322170b2bdd4e2eb4493696)
2013-10-28 10:39:46 -04:00
Tim Abbott 5b2775724a Make tools/migrate-db optimized for the local dev instance case.
(imported from commit a4db99435646e337a09cbfe2f64d3d7ca6b0346a)
2013-10-28 10:39:46 -04:00
Steve Howell 09979252dd Add Steve's laptop to USING_RABBITMQ list.
(imported from commit 3527edd27e3f386eb8b668fe8ee7e3875282ead8)
2013-10-27 11:27:27 -04:00
Zev Benjamin cdcbe61cb0 [manual] Change references to the humbug_test user, schema, and database to zulip_test
This requires no changes in production, but is tagged as manual to
remind developers that they need to edit and run the tools/migrate-db
script to fix up their local database instances.

(imported from commit fbf764fb61592ef994d6d2ad56edad65ff01f14b)
2013-10-26 04:16:28 -04:00
Zev Benjamin 7b6f6823a5 [manual] Change references to the humbug user, schema, and database to zulip
This commit must be simultaneously deployed on both staging and
prod0.  It also requires completely taking down the app.

To deploy these changes, do:
* check out this commit at /root/zulip on postgres0, postgres1, staging, and prod0

* stop the process_fts_updates job on postgres0 and postgres1
* stop the app on staging and prod0

* do a puppet apply on postgres0, postgres1, staging, and prod0
* move the new client certificates into place on staging and app
* move the new server certificates into place on postgres0 and postgres1
* reload the database config on postgres0 and postgres1 (this might
  actually require a restart)
* run tools/migrate-db on postgres0 as root

* do a deploy through this commit on staging and prod0
* start the process_fts_updates job on postgres0 and postgres1

* do a puppet apply on nagios

(imported from commit 819bdd14326c1425e2d3041a491a8ca3b9716506)
2013-10-26 04:16:27 -04:00
Zev Benjamin 05b9c08cfa Add one-off tool for migrating our database user, schema, and db name from humbug to zulip
(imported from commit 4b5e27eb3b083f2e2f2cabe64901f3dbaecd4739)
2013-10-26 04:16:27 -04:00
Steve Howell 13768495a6 Avoid long locks in process missedmessage_emails queue.
(imported from commit aed4a89244dce19085ed2ad89a7539ab297d8bc9)
2013-10-25 20:45:19 -04:00
Steve Howell 3cbb9f6b08 Avoid long locks in process digest_emails queue.
(imported from commit 49c7441463c5e63e52e5b099bd854bd4ae769f13)
2013-10-25 20:34:25 -04:00
Jessica McKellar 2be6c541cf digest emails: use a template filter to format new users, streams, etc.
(imported from commit e61a3dd6753db82da1b30c50e5bbf6273f302645)
2013-10-25 19:11:41 -04:00
Jessica McKellar d4b60e9304 Add a custom template filter for formatting displaying lists of items in emails.
We do this operation several times in the digest e-mails.

(imported from commit 7b906bad582c628d4461637dc7af374df349e09a)
2013-10-25 19:11:40 -04:00
Jessica McKellar 62136d0c8e [manual] Add the new digest_emails queue processor to the supervisor config.
This commit needs a puppet apply to take effect.

(imported from commit 0b26c385bad7c928912411ba85b70512b84be643)
2013-10-25 19:11:38 -04:00
Jessica McKellar a4a12cf407 digest: Add a queue processor that processes digest email requests.
(imported from commit 4f0abb70c92ae6bb47abb3792cca79d3f70e716a)
2013-10-25 19:09:45 -04:00
Jessica McKellar fac9d34d6c digest: Add a management command to enqueue digest email recipients.
(imported from commit 70ff2c7a4dae654f4077041c45e2154b3ac7afb7)
2013-10-25 19:08:47 -04:00
Jessica McKellar 772adc1969 digest: Add functions used to compute interesting traffic.
So far, we gather: "hot conversations", missed PMs, new streams, and
new users.

(imported from commit c3c723d0426cb55bd0e43917c67f93db9052f9ed)
2013-10-25 19:08:46 -04:00
Jessica McKellar a29fc6ab16 digest: Add HTML and plaintext templates for digest emails.
(imported from commit 6c4dbdce1544b720fa45506a9175e8ace93bafe3)
2013-10-25 19:08:46 -04:00
Waseem Daher 1e6f3bb4cd Move 'unshare the love' flag into feature_flags.
All local_server JS stuff should ideally be controlled from
feature-flags so it's easy to see at a glance what's been done.

(imported from commit 45b1cdae382679d3fa3b5f02f67e8ab749a89a51)
2013-10-25 18:48:46 -04:00
Waseem Daher d97b09e164 local_server: Don't try to hire away our customers.
(imported from commit 968df69fc863598550a554c739f4b5cde1f93eab)
2013-10-25 18:34:10 -04:00
Waseem Daher e4db25a501 Remove link to our postcard when we're local server.
(imported from commit 67af411eeb545790ce0b85e1031ceaa99f6a72b9)
2013-10-25 18:34:10 -04:00
Waseem Daher a56fe810b2 Hide "Share the love" for local server.
We leave the HTML in because it's harmless. (We could do a more
thorough expunge, but I don't see the point).

(imported from commit d3d68b0b2df96a9f3de73903b043c18bc6f77563)
2013-10-25 18:08:47 -04:00
Waseem Daher d44c0d5f5d Pass local_server into page_params.
(imported from commit 50dcee54644e0ee2ad8b13ec12e7134a9ab58d86)
2013-10-25 17:37:42 -04:00
Waseem Daher 66f48288b4 Rename LOCALSERVER -> LOCAL_SERVER.
(imported from commit b3abdd10d54d2ad7a9c463af9a291d2e2127707f)
2013-10-25 17:37:06 -04:00
Leo Franchi 8e21d70afe Disable statsd on local server
(imported from commit cac2c95c2de18b613a0900fe1c04fd97963b9350)
2013-10-25 16:37:29 -04:00
Leo Franchi c12a97577e Rework local_settings_template.py to be clear about what the user needs to do
(imported from commit f69ea0d57e885509abc70dc4f2bee6e36169691f)
2013-10-25 16:35:27 -04:00
Leo Franchi 9e1173eb9e Add a generate_localserver_secrets.py script that populates local_settings.py
(imported from commit ced1397d225efe0cf2f0eef7d43b495e3ab9b845)
2013-10-25 16:29:18 -04:00
Luke Faraone 2d787e952b Expose an endpoint to identify the API/webclient bases for a particular user.
We fall back to guessing based on the realm if the user doesn't have a
profile in our system

(imported from commit 833885168c451074c885b4422d62986855a215f7)
2013-10-25 15:18:38 -04:00
Luke Faraone a1b44986e0 [schema] Add a base_{api,site}_url property to our deployments.
This will allow us to redirect clients to the correct local site.

To apply this migration, just run:
     python manage.py migrate zilencer 0002

(imported from commit 7bd39b5f035145b6b52e1b2cb2ad5f6720d598ce)
2013-10-25 15:18:37 -04:00
Luke Faraone 238d19e1f1 Include a rest_dispatch in tornadoviews and call it from urls.py
(imported from commit aefe66b44be60b39d0f6bb3076502e964f1ed6bd)
2013-10-25 14:13:31 -04:00
Luke Faraone d5d0ef809b [manual] Deconfigure standalone feedback bot, replacing it with a worker.
Steps needed:
 - puppet apply on staging/prod when deployed to respective sites
 - puppet apply on bots.z.n when deployed to prod
 - copy /var/tmp/.feedback-bot-ticket-number from bots to staging

(imported from commit 2c943dac8d871809b0997a4484f508ec5b078bcd)
2013-10-25 14:13:30 -04:00
Luke Faraone e552307511 Send feedback to a queue to be forwarded to staging.
(imported from commit 4a9a1bfc6c95763a816263a726cc61b3ca90bf15)
2013-10-25 14:13:30 -04:00
Luke Faraone 1fa65915f2 NACK on consumer failure
(imported from commit 1cad1096dd1a969b2551822575784c338a82f433)
2013-10-25 14:13:30 -04:00
Luke Faraone 9d4ef27b97 Make rest_dispatch accept a list of globals and move it into its own file.
We're going to be using this from other views, so make it more generic.

(imported from commit c2c4f64b2a212f8a254f75c21dc93d8fc3f8351a)
2013-10-25 14:13:30 -04:00
Luke Faraone 81d7dd1fda [schema] Support for authenticating Deployments via the API.
Here we introduce a new Django app, zilencer. The intent is to not have
this app enabled on LOCALSERVER instances, and for it to grow to include
all the functionality we want to have in our central server that isn't
relevant for local deployments.

Currently we have to modify functions in zerver/* to match; in the
future, it would be cool to have the relevant shared code broken out
into a separate library.

This commit inclues both the migration to create the models as well as a
data migration that (for non-LOCALSERVER) creates a single default
Deployment for zulip.com.

To apply this migration to your system, run:
   ./manage.py migrate zilencer

(imported from commit 86d5497ac120e03fa7f298a9cc08b192d5939b43)
2013-10-25 14:13:30 -04:00
Luke Faraone 52309b5789 Enable absolute imports for non-third-party code where it was omitted previously
(imported from commit 34856ac32c6b94b614273a6fe46c87a314058d9b)
2013-10-25 14:13:30 -04:00
Steve Howell 356b400500 Remove obsolete stream_data.all_subs().
(imported from commit 7542638ad5132ca382e5e461a76c58d141db532d)
2013-10-25 13:50:34 -04:00
Steve Howell 392197da28 Fix recent Streams page regression.
The Streams page should only show active public realms, even though
a user might have info about a "retired" stream in their browser.
I regressed this in 69b83d769 for "retired" streams.  A retired
stream is a stream that no longer has subscribers.  The bug
scenario here was that you could create a stream, regret it,
unsubscribe yourself, and then the stream never went away from
the Streams page.

This diff tries to be a little more explicit about building the
list of streams for the Streams page.  Basically you have two
sources:

    * Get only the subscribed streams from the internal
      data structures.

    * Get the unsubscribed streams by calling the back end
      for all public streams, and subtract out the subscribed
      streams.

I tested the following scenarios:

    normal stream with me: in Streams
    normal stream without me: in Streams
    my invite-only: in Streams
    their invite-only with me: in Streams
    their invite-only without me: not in Streams
    retired stream: not in Streams (but message colors are good)

See the email "custom query to get public stream names" for some
related discussion.

(imported from commit bc9224e68797b26b795399941117faa9d6858b39)
2013-10-25 13:50:34 -04:00
Steve Howell 20c18d905d Rename internal subscribed_streams() to exports.subscribed_subs().
I want to make subscribed_streams() external, but it conflicts with
a legacy name in the same module (stream_data.js), so I have to rename
it in the same commit.  The new name conforms better to the current
naming convention, which generally has functions returning objects
use "sub" in the name.

(imported from commit 9f1ed60772c649359a413257e0998857eab3603f)
2013-10-25 13:50:33 -04:00
Luke Faraone 70e161db01 Make settings.LOCALSERVER available as `localserver` inside templates
(imported from commit 4f0cfd1f1015061942f603cbc03a253a713a9f83)
2013-10-25 13:31:13 -04:00
Leo Franchi 0bc5b13648 Fix Zulip username of email gateway bot
(imported from commit f24c5fd3fc5888bf7dc4d9501723777ad8447704)
2013-10-25 11:51:13 -04:00
Leo Franchi a9b9c0ec7b Add Email Mirror default settings and error out with readable warning if not configured
(imported from commit adac321d09561b51277ec1e314a1d480dc01e0c8)
2013-10-25 11:51:13 -04:00
Tim Abbott f2f97dd335 [puppet] Increase maximum file descriptor count for zulip in limits.conf.
We ran into the 1024 file descriptor limit today for tornado.  The
limits.conf file descriptor limit isn't used for supervisor started as
we currently do it, but it's a good idea to have this in place so that
if we move supervisor to run as the zulip user, we don't experience a
nasty surprise.

(imported from commit 2eb4805b5c129bc2684d151b77f295c2eaa9fc3e)
2013-10-25 11:48:35 -04:00
Tim Abbott 836f313e69 [puppet] Increase maximum file descriptor count for supervisord.
We ran into the 1024 file descriptor limit today, causing some issues
for users.

(imported from commit 295e6e5cce46024a3668fb8cebc8669568844ab3)
2013-10-25 11:48:35 -04:00
Leo Franchi 2d276179d0 Refactor email-mirror to handle running on any machine
(imported from commit 2971449ceaacb564770e66874fc095f77e68d445)
2013-10-25 11:37:34 -04:00
Steve Howell 9979d884e5 Remove 06-unread.js.
This test has been broken for a couple months, and nobody has taken
ownership of fixing it.  It's always slow, sometimes it fails
randomly, sometimes it fails for things that aren't really problems,
and it's generally been way more trouble than it's worth.

(imported from commit 8080e81b226a372e763a2558f4e5668c3a4d087c)
2013-10-25 11:27:25 -04:00
Zev Benjamin 77a1ed967b Make the socket accessible externally for debuggging
(imported from commit 8f78d09edc05f8eed5f1a2d87643bdd7b7d9bec6)
2013-10-24 17:40:43 -04:00
Kevin Mehall 3bad5acc71 Fix upload auth redirect for user_activity.
Use rest_dispatch for upload auth redirect so it doesn't send the
long URL to user_activity.

(imported from commit ab327bbd529412e43eee6d109f8550180544dbbb)
2013-10-24 17:33:12 -04:00
Kevin Mehall 19b9b6e2ec Reduce the length of public upload URLs.
(imported from commit dc4d51da39b218068012ebad57e1f92cb50137c3)
2013-10-24 17:01:06 -04:00
Kevin Mehall 0a271f149b Enable authed file uploads for squarespace
(imported from commit c72e76d27f99cd7610eb6c93b57b8ff727b461a9)
2013-10-24 17:01:06 -04:00
Kevin Mehall 5e1caad88a Show upload filename in tooltip without other ugly parts of the URL.
This works for both public and authenticated uploads.

(imported from commit f9be340b53d0f72bf5b3c31a5d51a1fad80759be)
2013-10-24 17:01:06 -04:00
Kevin Mehall 0a3a22cb3d Support authenticated upload URLs.
Trac #1734

This is implemented by bouncing uploaded file links through a view
that checks authentication and redirects to an expiring S3 URL.

This makes file uploads return a domain-relative URI. The client converts
this to an absolute URI when it's in the composebox, then back to relative
when it's submitted to the server.

We need the relative URI because the same message may be viewed across
{staging,www,zephyr}.zulip.com, which have different cookies.

(imported from commit 33acb2abaa3002325f389d5198fb20ee1b30f5fa)
2013-10-24 17:01:06 -04:00
Kevin Mehall 473a98d0a4 Split out common code for existing and authenticated upload systems.
(imported from commit bf513eaff90ae7c4a0be825317690058f9f54621)
2013-10-24 17:01:06 -04:00