Commit Graph

336 Commits

Author SHA1 Message Date
Tim Abbott cd71840942 Use a better output filename for Django logs.
(imported from commit 97d37d9167ac8463ea54fb7a75a3f923fc78edee)
2013-07-17 17:50:19 -04:00
Tim Abbott 23beabb80c [manual] Rename manage.py subscribe_new_users to process_signups.
The old name was very confusing, and this fits the convention of "the
processor for the signups" queue a la "process_user_activity".

This requires doing a

supervisorctl stop humbug-workers:humbug-events-subscribe-new-users
puppet apply

to deploy the supervisord configuration changes and properly restart
the signups queue.

(imported from commit 0ee2dad837142afa64025446e22956709771a192)
2013-07-17 17:50:19 -04:00
Tim Abbott 27586559dc Remove Puppet configuration for zulip-staging.
The file was already deleted.

(imported from commit 04c68076ad59709c2848f81ca81db9612f9c65b8)
2013-07-17 17:50:19 -04:00
Tim Abbott f61b4ddb25 Remove second duplicate python-boto instance.
(imported from commit 11ef02de82d0f8cdd4c1cf3813e5d09619d4665b)
2013-07-17 17:50:19 -04:00
Zev Benjamin 642f347790 puppet: Add check_postgres.pl nagios plugin
(imported from commit 9d9d0ff74dc0fdf17d81b9a0b450df7cb37d04d5)
2013-07-17 14:34:00 -04:00
Zev Benjamin d16b336f29 puppet: Remove duplicate python-boto package declaration
(imported from commit ef3f8f73bc43363317be9b5875ed95e6fa13f4a3)
2013-07-17 14:34:00 -04:00
Zev Benjamin 81c05e02c2 nagios: Check for the expected number of autossh processes on munin.humbughq.com
(imported from commit 77d35b2aaacf303f6118d7794f481e393868da59)
2013-07-17 14:34:00 -04:00
Zev Benjamin 14e58ff6e4 Monitor postgres1
The fact that we weren't already was an oversight on my part.

(imported from commit 2082ae79ac2884f26e98b430bcb08c15938a26c0)
2013-07-17 14:34:00 -04:00
Zev Benjamin 431acdb186 munin: Rename postgres to postgres0
(imported from commit 07c324b5b7e0579e7b97b07c4fdf55f0e66f131c)
2013-07-17 14:34:00 -04:00
Zev Benjamin b4a208445b Run check_postgres.pl against the correct database
We were previously running it against the 'postgres' database, which
meant we weren't actually checking the non-clusterwide statistics.

(imported from commit a6be529b16d5f1927463e49a7f7f4cf0b5299213)
2013-07-17 14:34:00 -04:00
Luke Faraone e4c9abb021 Remove duplicate ipython entry from puppet
(imported from commit 2657ed1d707900ea4b1074fe59a0b662647805e4)
2013-07-15 18:07:56 -04:00
Luke Faraone 1f811133d1 Serve static /dist/ content on app servers when hostname zulip.com is used
(imported from commit cc78ffafdffe5df2baf08bdd70a219dbb694337d)
2013-07-15 16:49:55 -04:00
Luke Faraone bb0a7c8fc3 [manual] Switch various configuration files to refer to .zulip.net.
We only want to change cases where we're talking about the hostname; HTTP
requests should still go to staging.humbughq.com for now.

Before this commit is deployed the hostname of staging.humbughq.com should
be changed to staging.zulip.net on the VM.

(the same for prod)

(imported from commit 7412530773f720ac227f40061c9ddb1a851e19bb)
2013-07-15 16:49:55 -04:00
Luke Faraone 9bef61ad87 Interpret X-Forwarded-For on app servers' nginx.
See:
    http://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from

(imported from commit adc4ebf46aefd1c71bda187d84519d8c31f6c590)
2013-07-15 16:49:55 -04:00
Luke Faraone 44b49b3bf8 Puppet configuration and associated nginx files for lb0.zulip.net.
lb0.zulip.net will proxy connections to the relevant backend servers.

Depressingly, SSL certificate verification of the backend servers is not
performed at this time, see:
    <http://trac.nginx.org/nginx/ticket/13>

The above-mentioned bug has existed since 2011, but a CVE was not
allocated until January. The nginx developers don't seem to care. Sigh.

In any case, this is of somewhat limited impact at Humbug, since we can
have reasonable confidence that communications within AWS are not
subject to active MITMs. Passive MITM is not a concern, because the
traffic *is* in fact encrypted.

(imported from commit c96e1235fc17192c7452e0417a1309cfcda62de2)
2013-07-15 16:49:55 -04:00
Luke Faraone ebde5ab341 Switch to logging module instead of syslog.
(imported from commit 4c2c2f0f23e2688ce916d33d0cf513e386dca70c)
2013-07-15 16:49:54 -04:00
Luke Faraone 4843303267 Automatically configure iptables and routing for secondary interfaces.
This is a horrible hack.

(imported from commit 01dca4514f01f7ad419d735b8879a25a999b552e)
2013-07-15 16:49:54 -04:00
Luke Faraone 0696a3fbd7 Automatically configure all interfaces (including virtual!) at boot
On EC2-VPC we have the ability to attach multiple addresses to one
interface, and multiple interfaces to one machine.

We should configure those interfaces whenever our system boots, and
ideally whenever networking is restarted.

This commit adds a script that is executed once eth0 is brought up that
proceeds to configure all subsequent interfaces, real and virtual.

The script is configured to be installed (along with the helper script
that calls it) on all systems via Puppet.

(imported from commit fdc153ef649edbb8fedd40ff4d77262aae593c39)
2013-07-15 16:49:54 -04:00
Leo Franchi 6a61c8d237 [manual] Change Humbug to Zulip in Sparkle, and start with 0.3.4
This requires a puppet apply on prod

(imported from commit 6890146fd5330acd1c5cbac5609191f332ebca4a)
2013-07-15 13:31:15 -04:00
Luke Faraone bc7ee2bd45 Make python-django-guardian a dependency according to puppet.
(imported from commit 05c53f714b0ac3e5f86d946c8ab6fb2eaac8827b)
2013-07-12 11:29:11 -04:00
Luke Faraone f70e7fadea Miscellaneous updates to install-server for wheezy compatibility.
We switch to always specifying HostKeyAlgorithms=ssh-rsa because of a ECDSA
key bug in the Debian images which results in the fingerprint not being
printed to the console. Our config later forces RSA after we do a puppet
apply, so we might as well start using RSA from the beginning.

We start out sshing in as "admin", and delete the user (moving keys over to
"root") at the beginning.

We switch to the ops repo instead of backports, and drop the installation
of puppet from backports.

We no longer install humbug-self-signed.key on our servers; instead real
certificates must be installed manually.

(imported from commit cbabe65a4e0ef37df1fece6eaec053a2368f6ef5)
2013-07-12 11:29:11 -04:00
Luke Faraone 011690e727 install-server's hostname check should be for 'is zulip in the domain?'
(imported from commit c2004edb46b5d9503899c56f13ad597f1972c3e1)
2013-07-12 11:29:02 -04:00
Leo Franchi 2a5e53eaec [manual] Update desktop apps to 0.3.3
This requires a puppet apply on prod

(imported from commit aba8004684de70772d2ddd31a563b3650c4cbd9b)
2013-07-05 16:41:26 -04:00
Luke Faraone 6eb3d8b8b1 Syntax fix for staging_app_frontend.pp.
(imported from commit c363e6e9c18b08467555a1f0520680736dab2dd1)
2013-07-02 14:54:16 -04:00
Luke Faraone 7b30a8e7c0 Explicitly copy over the zulip-staging files in Puppet.
Unlike other directories, we explicitly enumerate the files we want to be
present in sites-available, so the previous commit series did not actually
instruct puppet to make the zulip-staging files accessible.

(imported from commit 22efc4d272eba8d6c869edbaa9114c50e1988288)
2013-07-02 12:14:41 -04:00
Luke Faraone 1be1cb121c nginx / Puppet configuration for staging.zulip.com
We create a new sites-available entry which is essentially a duplicate of
sites-available/humbug-staging with s/humbug/zulip, and add the associated
symlink directive in Puppet.

(imported from commit febcb585ce93c21c6849d96458cc2bd096b30538)
2013-07-02 12:04:56 -04:00
Leo Franchi 975e13a1b8 Update sparkle to our 0.3.0 release
(imported from commit bd02d67fbd13d709b579f93a69d625da5517eec7)
2013-07-02 10:40:12 -04:00
Leo Franchi 7036915933 Add windows sparkle files
(imported from commit b7c0770acd34f44e961014a00d2059dfc7bef701)
2013-07-01 16:25:35 -04:00
Tim Abbott 3bdd446651 puppet: Fix nginx configuration for api.humbughq.com.
(imported from commit d8b535b666a3b3d758a62812a118413c619c09a5)
2013-06-28 15:57:28 -04:00
Tim Abbott ea8a80603a [manual] Change API URLs to be based on api.humbughq.com/api.
This must be deployed after we update our running nginx configuration
to serve api.humbughq.com.

(imported from commit b5c34ebdd595f55eecd6dca6a18a37f105107bd5)
2013-06-28 15:57:27 -04:00
Scott Feeney 83cd963c49 Remove unused imports
(imported from commit 9e3050c72a2d1137b9096c6cfa1c3945341b9a56)
2013-06-27 16:22:39 -04:00
Zev Benjamin 6f874995ff [schema] Use custom stopwords file for full text search
This stop words file is just the default Postgres english stop file
with all the rest of the letters of the alphabet added.  Adding the
extra letters ensures that, e.g., "bed" doesn't get transformed into
"bed | b".

(imported from commit 0be3ef9a43eb524ed4f081d5081a786cf602c487)
2013-06-27 14:18:53 -04:00
Tim Abbott 400db86008 [manual] nginx: Pass post-rewrite URIs to FastCGI.
This requires us to do a puppet apply when it is deployed to each of
staging and prod.

(imported from commit eed631ce10340e7fe3252cd8a4f05fd59ef3c942)
2013-06-25 16:34:43 -04:00
Tim Abbott ae89b25d69 nginx: Add fastcgi_params to puppet.
(imported from commit 12e6b02cd2cb411ab83a29a486053df6dff9ebb8)
2013-06-25 16:34:43 -04:00
Tim Abbott 222ef672b5 [manual] Use ujson instead of simplejson.
This saves something like 15ms on our 1000 message get_old_messages
queries, and will save even more when we start sending JSON dumps into
our memcached system.

We need to install python-ujson on servers and dev instances before
pushing this to prod.

(imported from commit 373690b7c056d00d2299a7588a33f025104bfbca)
2013-06-24 10:17:10 -04:00
Zev Benjamin bf9ad1874e puppet: Run pg_backup_and_purge.py nightly
(imported from commit 98555b970533adc3c60377490074607da9f78efb)
2013-06-21 14:08:57 -04:00
Zev Benjamin 15d13f8f40 puppet: Add script for doing Postgres base backups and purging old backups
(imported from commit 93a92729b2e964e054aa1af7bcb8a0bae3fd1b33)
2013-06-21 14:08:57 -04:00
Zev Benjamin 33b3b1fa62 puppet: Switch which S3 bucket we backup Postgres to
The old bucket was versioned and didn't allow deletes.  This was
great for paranoia, but not so great for being able to delete old
backups.

(imported from commit be79b5c582ca5ee466cdfea6d3093b6d5ba0e23d)
2013-06-21 14:08:57 -04:00
Zev Benjamin 1b6514b89f puppet: Use the correct Postgres archive command
I hadn't changed it previously out of paranoia in the case we had a
faulty failover and had two masters both uploading to the same place.
However, I now don't think this can happen, as recovery completion
will cause Postgres to start a new timeline.

(imported from commit d58f1aa306eff4f6fd950664ff658539c1249bdf)
2013-06-21 14:08:57 -04:00
Zev Benjamin bf82fadc95 puppet: Move /tmp to local storage on Postgres master servers
(imported from commit eae0a31faad6d95c8e2b55c11481aa19d7e108f2)
2013-06-21 14:08:57 -04:00
Luke Faraone 6bd3886406 Don't pass along client locale settings when sshing in to our servers
(imported from commit d25f2a47b60c1ac7e4dcbd4a0133d0c0c9698b4e)
2013-06-18 17:20:48 -04:00
Leo Franchi 23322a791d puppet: Add sparkle configuration files
(imported from commit e36efd64584d946bb13fb5b44af817e85345e197)
2013-06-18 16:12:14 -04:00
Tim Abbott c3b4ab0f02 puppet: Move nagios homedir to /var/lib/nagios.
That's where it is supposed to be, and besides, that's what a Nagios
server is going to expect it to be.

(imported from commit c273f18533909fa8eac182246dbbe498a5381f6c)
2013-06-17 13:48:06 -04:00
Tim Abbott 261300d10e puppet: Add Nagios crontab to puppet.
(imported from commit 353b167b303b27ccbfc0cd0130665399faab80dc)
2013-06-17 13:48:06 -04:00
Tim Abbott 83016e501b puppet: Complete several more of our nagios TODOs.
(imported from commit 239cbe1606cc077119b65b3dc18b519b09b021a4)
2013-06-17 13:48:06 -04:00
Tim Abbott d3d5334a55 puppet: Import pagerduty_nagios.pl into puppet.
(imported from commit 1b91524498372d3e69f07468e4635c4d66c44d85)
2013-06-17 13:48:06 -04:00
Tim Abbott caafb405d7 puppet: Finish puppetizing git/wiki configuration.
(imported from commit 423083269cea7b23f8b5805561919b9c6b9a8f4d)
2013-06-17 13:48:06 -04:00
Tim Abbott 5c388ed28e puppet: Run our wiki out of supervisord.
(imported from commit a8f6d14ce55de0e7458496f9debb15529120deaf)
2013-06-17 13:48:06 -04:00
Tim Abbott 1bf4a36ac1 puppet: Clean up organization in base.pp a bit.
(imported from commit 2be403981f94b440f9b999d7b8a36a0ec0cedcb7)
2013-06-17 13:48:06 -04:00
Tim Abbott 91933a2004 puppet: Move configuring supervisor into its own module.
(imported from commit b2ad9bc5c5d6bde78380b0537696ed26cab8c264)
2013-06-17 13:48:06 -04:00