Commit Graph

359 Commits

Author SHA1 Message Date
Tim Abbott e111a2f9a5 [manual] Rename Django app from zephyr to zerver.
This needs to be deployed to both staging and prod at the same
off-peak time (and the schema migration run).

At the time it is deployed, we need to make a few changes directly in
the database:

(1) UPDATE django_content_type set app_label='zerver' where app_label='zephyr';
(2) UPDATE south_migrationhistory set app_name='zerver' where app_name='zephyr';

(imported from commit eb3fd719571740189514ef0b884738cb30df1320)
2013-08-06 07:39:36 -04:00
Luke Faraone b13361708f Warn users when sshing in to lb0.zulip.net
(imported from commit df30627fa6ee5eaf5589eb833879c7e03b204d5c)
2013-07-31 10:50:19 -04:00
Zev Benjamin 0008edc9d5 [manual] Add worker process for processing asynchronous FTS updates
Unlike the other workers, this process runs on the database server.
To prevent duplicate work, only the worker on the master actually
processes updates.

To deploy, we in theory just need to apply the new puppet config.  In
practice, the database servers aren't on wheezy yet, so the changes
should be applied by hand to postgres0 and postgres1.  This only
needs to be done on deploy to staging.

(imported from commit d679ffc0838f9fc8c7f0bede08a5568b339b7ddb)
2013-07-30 18:13:05 -04:00
Luke Faraone 7637574345 Install molly-guard on our servers
(imported from commit 8c065763e649a86e50754c738458f09e7d0cf71d)
2013-07-30 17:45:20 -04:00
Luke Faraone 942cc94b04 Install files to customise postgres for maximum performance.
Configuration provided by Zev.

(imported from commit 4d0cdbb6747689863208079e2c21864521e1cf02)
2013-07-26 17:11:54 -04:00
Leo Franchi 7b6579c74c s/humbughq/zulip/ in graphite setup
(imported from commit 5c87bbe3d1ae4f57c5f66899b66cbd4417e49887)
2013-07-26 16:23:03 -04:00
Luke Faraone 7c2de24213 Make trac.ini world-readable
This is bringing our puppet configuration in line with reality.

(imported from commit 48a10676a9f28566dd89fff1337a4b34c23c0b61)
2013-07-26 16:14:55 -04:00
Luke Faraone 5e2dda4d73 Configuration for puppetized MediaWiki on trac.
(imported from commit cff2becbe590e50414f74a31ea2a2914de49f4bd)
2013-07-26 16:14:55 -04:00
Luke Faraone 85988df75d [third] Import Auth_remoteuser.php into puppet.
This is a module we're using for REMOTE_USER support in MediaWiki. It is
not used in any app code nor is it distributed, nor is it incorporated
into any comapny works.

License: GPLv2+

(imported from commit da9a81db251cf850335987697ea8383623b58c53)
2013-07-26 16:14:55 -04:00
Luke Faraone 41504762a1 env-wal-e: Use a different bucket if not running on an app postgres server.
(imported from commit e06be77d544532885fbc4769fedb55eedac22ccd)
2013-07-26 16:14:55 -04:00
Luke Faraone 76a58fe800 We are no longer using gitit nor is wiki running off git.zulip.net.
(imported from commit fbe1bc2804a5ed6ccecb7b3090b55d7e9fa5b1bc)
2013-07-26 16:14:55 -04:00
Luke Faraone 639d963ff5 Split out app-specific postgres configuration.
(imported from commit 979811e69f436d02023a6c62e614a39c19bf64b8)
2013-07-26 16:14:55 -04:00
Tim Abbott 0d077e7f41 [manual] Use api.zulip.com to access the Humbug API.
This needs to be deployed after the nginx configuration is deployed.

(imported from commit 167a3d9d27595d40883bb400ebcc44c4ad9b6a2f)
2013-07-25 17:27:46 -04:00
Tim Abbott 4f8f46544c [manual] Add additional zulip hosts to nginx config.
We'll need to do a puppet apply on prod for this to take effect.

(imported from commit 764926317712ed8abebc429fb13d43c755e0f130)
2013-07-25 17:27:45 -04:00
Tim Abbott 9bef184eda Fix check_send_receive_time always checking prod.
The value it is passed is usually https://staging.humbughq.com, not
just staging.humbughq.com.

(imported from commit c3cd8fc5baa767377f506570aa8e7d2e1ed399ec)
2013-07-25 17:27:45 -04:00
Tim Abbott cdeaceea47 Fix URLs for wiki following mediawiki transition.
(imported from commit 140b4f7044f6e68c16d743c0e91dcf5a2557311e)
2013-07-25 17:27:45 -04:00
Tim Abbott 3fcd6eb9e6 Update humbughq.com => zulip.net in tools.
This shouldn't be pushed until we get graphite.zulip.net working.

(imported from commit c0dd208222385a9ee92ad1ca8fbf141894f7c7b6)
2013-07-25 17:27:45 -04:00
Tim Abbott 77b0ddef78 [manual] Rename shakespeare users to zulip.com domain.
We need to update our Nagios configuration when this is deployed.

(imported from commit 28cc3ecfe30c41562742733d6952c8e69b87d67e)
2013-07-25 17:19:17 -04:00
Tim Abbott 1f08f4e70f Rename nagios bot to zulip.com domain.
(imported from commit 9a2fba54295b4c473e030d3ff6ededbc3e2455af)
2013-07-25 17:16:53 -04:00
Leo Franchi e8a56a22e6 Use new key for git.hhq.com in install-server, and typo fixes
(imported from commit 23b69d23be24f2d995b70e03e1e7e6e604407877)
2013-07-24 16:06:34 -04:00
Leo Franchi bb2df13bfd Unset -e flag during humbug.pem login to preserve idempotency of install-server
(imported from commit d65b635a48ce33908b2888319a7c109d5d126b54)
2013-07-24 16:06:34 -04:00
Scott Feeney e539d6b2ac [manual] Add python-sourcemap dependency
(imported from commit 784cfa09ecea58f479adb0b17906a4d0ec10f618)
2013-07-23 15:57:07 -04:00
Tim Abbott 508ddca539 puppet: Specify commands for restarting supervisord.
(imported from commit e1a6322fd9ecf1655ec4325d0fcaf0ad160202ae)
2013-07-23 11:30:46 -04:00
Tim Abbott cd71840942 Use a better output filename for Django logs.
(imported from commit 97d37d9167ac8463ea54fb7a75a3f923fc78edee)
2013-07-17 17:50:19 -04:00
Tim Abbott 23beabb80c [manual] Rename manage.py subscribe_new_users to process_signups.
The old name was very confusing, and this fits the convention of "the
processor for the signups" queue a la "process_user_activity".

This requires doing a

supervisorctl stop humbug-workers:humbug-events-subscribe-new-users
puppet apply

to deploy the supervisord configuration changes and properly restart
the signups queue.

(imported from commit 0ee2dad837142afa64025446e22956709771a192)
2013-07-17 17:50:19 -04:00
Tim Abbott 27586559dc Remove Puppet configuration for zulip-staging.
The file was already deleted.

(imported from commit 04c68076ad59709c2848f81ca81db9612f9c65b8)
2013-07-17 17:50:19 -04:00
Tim Abbott f61b4ddb25 Remove second duplicate python-boto instance.
(imported from commit 11ef02de82d0f8cdd4c1cf3813e5d09619d4665b)
2013-07-17 17:50:19 -04:00
Zev Benjamin 642f347790 puppet: Add check_postgres.pl nagios plugin
(imported from commit 9d9d0ff74dc0fdf17d81b9a0b450df7cb37d04d5)
2013-07-17 14:34:00 -04:00
Zev Benjamin d16b336f29 puppet: Remove duplicate python-boto package declaration
(imported from commit ef3f8f73bc43363317be9b5875ed95e6fa13f4a3)
2013-07-17 14:34:00 -04:00
Zev Benjamin 81c05e02c2 nagios: Check for the expected number of autossh processes on munin.humbughq.com
(imported from commit 77d35b2aaacf303f6118d7794f481e393868da59)
2013-07-17 14:34:00 -04:00
Zev Benjamin 14e58ff6e4 Monitor postgres1
The fact that we weren't already was an oversight on my part.

(imported from commit 2082ae79ac2884f26e98b430bcb08c15938a26c0)
2013-07-17 14:34:00 -04:00
Zev Benjamin 431acdb186 munin: Rename postgres to postgres0
(imported from commit 07c324b5b7e0579e7b97b07c4fdf55f0e66f131c)
2013-07-17 14:34:00 -04:00
Zev Benjamin b4a208445b Run check_postgres.pl against the correct database
We were previously running it against the 'postgres' database, which
meant we weren't actually checking the non-clusterwide statistics.

(imported from commit a6be529b16d5f1927463e49a7f7f4cf0b5299213)
2013-07-17 14:34:00 -04:00
Luke Faraone e4c9abb021 Remove duplicate ipython entry from puppet
(imported from commit 2657ed1d707900ea4b1074fe59a0b662647805e4)
2013-07-15 18:07:56 -04:00
Luke Faraone 1f811133d1 Serve static /dist/ content on app servers when hostname zulip.com is used
(imported from commit cc78ffafdffe5df2baf08bdd70a219dbb694337d)
2013-07-15 16:49:55 -04:00
Luke Faraone bb0a7c8fc3 [manual] Switch various configuration files to refer to .zulip.net.
We only want to change cases where we're talking about the hostname; HTTP
requests should still go to staging.humbughq.com for now.

Before this commit is deployed the hostname of staging.humbughq.com should
be changed to staging.zulip.net on the VM.

(the same for prod)

(imported from commit 7412530773f720ac227f40061c9ddb1a851e19bb)
2013-07-15 16:49:55 -04:00
Luke Faraone 9bef61ad87 Interpret X-Forwarded-For on app servers' nginx.
See:
    http://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from

(imported from commit adc4ebf46aefd1c71bda187d84519d8c31f6c590)
2013-07-15 16:49:55 -04:00
Luke Faraone 44b49b3bf8 Puppet configuration and associated nginx files for lb0.zulip.net.
lb0.zulip.net will proxy connections to the relevant backend servers.

Depressingly, SSL certificate verification of the backend servers is not
performed at this time, see:
    <http://trac.nginx.org/nginx/ticket/13>

The above-mentioned bug has existed since 2011, but a CVE was not
allocated until January. The nginx developers don't seem to care. Sigh.

In any case, this is of somewhat limited impact at Humbug, since we can
have reasonable confidence that communications within AWS are not
subject to active MITMs. Passive MITM is not a concern, because the
traffic *is* in fact encrypted.

(imported from commit c96e1235fc17192c7452e0417a1309cfcda62de2)
2013-07-15 16:49:55 -04:00
Luke Faraone ebde5ab341 Switch to logging module instead of syslog.
(imported from commit 4c2c2f0f23e2688ce916d33d0cf513e386dca70c)
2013-07-15 16:49:54 -04:00
Luke Faraone 4843303267 Automatically configure iptables and routing for secondary interfaces.
This is a horrible hack.

(imported from commit 01dca4514f01f7ad419d735b8879a25a999b552e)
2013-07-15 16:49:54 -04:00
Luke Faraone 0696a3fbd7 Automatically configure all interfaces (including virtual!) at boot
On EC2-VPC we have the ability to attach multiple addresses to one
interface, and multiple interfaces to one machine.

We should configure those interfaces whenever our system boots, and
ideally whenever networking is restarted.

This commit adds a script that is executed once eth0 is brought up that
proceeds to configure all subsequent interfaces, real and virtual.

The script is configured to be installed (along with the helper script
that calls it) on all systems via Puppet.

(imported from commit fdc153ef649edbb8fedd40ff4d77262aae593c39)
2013-07-15 16:49:54 -04:00
Leo Franchi 6a61c8d237 [manual] Change Humbug to Zulip in Sparkle, and start with 0.3.4
This requires a puppet apply on prod

(imported from commit 6890146fd5330acd1c5cbac5609191f332ebca4a)
2013-07-15 13:31:15 -04:00
Luke Faraone bc7ee2bd45 Make python-django-guardian a dependency according to puppet.
(imported from commit 05c53f714b0ac3e5f86d946c8ab6fb2eaac8827b)
2013-07-12 11:29:11 -04:00
Luke Faraone f70e7fadea Miscellaneous updates to install-server for wheezy compatibility.
We switch to always specifying HostKeyAlgorithms=ssh-rsa because of a ECDSA
key bug in the Debian images which results in the fingerprint not being
printed to the console. Our config later forces RSA after we do a puppet
apply, so we might as well start using RSA from the beginning.

We start out sshing in as "admin", and delete the user (moving keys over to
"root") at the beginning.

We switch to the ops repo instead of backports, and drop the installation
of puppet from backports.

We no longer install humbug-self-signed.key on our servers; instead real
certificates must be installed manually.

(imported from commit cbabe65a4e0ef37df1fece6eaec053a2368f6ef5)
2013-07-12 11:29:11 -04:00
Luke Faraone 011690e727 install-server's hostname check should be for 'is zulip in the domain?'
(imported from commit c2004edb46b5d9503899c56f13ad597f1972c3e1)
2013-07-12 11:29:02 -04:00
Leo Franchi 2a5e53eaec [manual] Update desktop apps to 0.3.3
This requires a puppet apply on prod

(imported from commit aba8004684de70772d2ddd31a563b3650c4cbd9b)
2013-07-05 16:41:26 -04:00
Luke Faraone 6eb3d8b8b1 Syntax fix for staging_app_frontend.pp.
(imported from commit c363e6e9c18b08467555a1f0520680736dab2dd1)
2013-07-02 14:54:16 -04:00
Luke Faraone 7b30a8e7c0 Explicitly copy over the zulip-staging files in Puppet.
Unlike other directories, we explicitly enumerate the files we want to be
present in sites-available, so the previous commit series did not actually
instruct puppet to make the zulip-staging files accessible.

(imported from commit 22efc4d272eba8d6c869edbaa9114c50e1988288)
2013-07-02 12:14:41 -04:00
Luke Faraone 1be1cb121c nginx / Puppet configuration for staging.zulip.com
We create a new sites-available entry which is essentially a duplicate of
sites-available/humbug-staging with s/humbug/zulip, and add the associated
symlink directive in Puppet.

(imported from commit febcb585ce93c21c6849d96458cc2bd096b30538)
2013-07-02 12:04:56 -04:00
Leo Franchi 975e13a1b8 Update sparkle to our 0.3.0 release
(imported from commit bd02d67fbd13d709b579f93a69d625da5517eec7)
2013-07-02 10:40:12 -04:00