Commit Graph

372 Commits

Author SHA1 Message Date
Leo Franchi 9bb699f917 Add a nagios plugin for checking rabbitmq queue sizes
(imported from commit 32bd03bcfe4c4a4221ace17f83adb175f591c8ea)
2013-02-19 15:22:55 -05:00
Leo Franchi e819995454 Set better munin thresholds for rabbitmq
(imported from commit 032fec63873a05ea7c03fc738d0f82d74322ca3e)
2013-02-15 15:54:12 -05:00
Tim Abbott 63827c2301 Make the Nagios integration configurable, available, and documented.
(imported from commit 1208fc08ed366a892763c3b29b9aeafa90b29981)
2013-02-14 17:50:00 -05:00
Leo Franchi 67a41143f5 [manual] Add rabbitmq monitoring to munin.
When deploying to prod, this will require a manual install of the
rabbitmq_* files to /etc/munin/plugins/ and an edit of
/etc/munin/plugin-conf.d/munin-node.conf

(imported from commit 4c10e634b04200dda1c4f4989e37fe232143240f)
2013-02-12 10:55:01 -05:00
Leo Franchi 0a0c4bb9a0 [manual] Use rabbitmq for asynchronous presence updating
Note: When deploying, restarting the process-user-activity-commandline script is needed

(imported from commit 63ee795c9c7a7db4a40170cff5636dc1dd0b46a8)
2013-02-11 18:05:57 -05:00
Zev Benjamin c80a8a50f4 puppet: fix app_frontend pip command
(imported from commit 58e2e57892d6af68c386f4838d0de26c7d2e861e)
2013-02-11 13:19:42 -05:00
Zev Benjamin 98add6efa2 puppet: Store the machine's type in /etc and add script to reapply the appropriate puppet config
Fixes #815

(imported from commit 0c90b74b9b11e4abcc875fbd7a2f0c48356678b2)
2013-02-11 13:19:41 -05:00
Zev Benjamin 801d2c40e2 puppet: Restart iptables-persistent from puppet
(imported from commit d46271046b4a341ab5a5f636908583323e107c69)
2013-02-11 12:55:01 -05:00
Leo Franchi b1e04f8feb Fix path to send-receive script post api relayout
(imported from commit 69cf0334b7bd363a862f468ea9d5e73b777bdd73)
2013-02-08 16:56:52 -05:00
Zev Benjamin 8132cf2ea2 puppet: Have install-server use its 'type' argument to install the correct puppet manifest
This also removes the "run puppet twice" hack, which I believe is no
longer needed for Apache.

(imported from commit 20016c3e8f0d267e04aeef585f19f5910bf01ddc)
2013-02-08 16:06:34 -05:00
Zev Benjamin 0e66607910 puppet: Fix wiki dependencies
(imported from commit a16675d203363b4cd535c2714ad77fd4d0c239d3)
2013-02-08 16:06:34 -05:00
Zev Benjamin 61466c0637 puppet: Fix trac copy and paste error
(imported from commit e201ea56a3111423f65e7a386685a4d90bc1080b)
2013-02-08 16:06:34 -05:00
Zev Benjamin b91d510dac puppet: Fix apache dependencies
(imported from commit 340dde4d045da17dcc7dab492a6fc12e0df3539c)
2013-02-08 16:06:34 -05:00
Zev Benjamin a4f6a3e83f puppet: Fix iptables dependency
(imported from commit 8c3275f4b875b318931a6f6917e6293b111b247f)
2013-02-08 16:06:34 -05:00
Zev Benjamin 153777bd01 puppet: Fix pip dependencies
(imported from commit 34d2ef0fbf39734c66d7ed777c225045e1b8619d)
2013-02-08 16:06:34 -05:00
Zev Benjamin c76ae76d51 puppet: Ensure a particular 30-postgresql-shm.conf file instead of appending lines to its contents
(imported from commit 884e1f2663763d7286146583a097f4548253e347)
2013-02-08 16:06:34 -05:00
Zev Benjamin fb5e5519d9 puppet: Ensure a particular sshd_config instead of appending a line to its contents
(imported from commit 4e745e23afe0cf8e6dd117cdeb6d6ec3a14ef24b)
2013-02-08 16:06:34 -05:00
Zev Benjamin da95bb2988 puppet: Move all puppetized config files to the humbug module and reference them with puppet URLs
(imported from commit f0f325bbad381b87c12c6f7888f4dd5d6989f09f)
2013-02-08 16:06:34 -05:00
Zev Benjamin beb2ecf5c9 puppet: Ensure that sshd is restarted after turning off password auth
(imported from commit 46fef98df9ea1d9ee4038f400cc7c8689d80a0ec)
2013-02-08 16:05:51 -05:00
Zev Benjamin 10f3853abd puppet: Make common::line use absolute paths
(imported from commit c9a3b184433709361243d5c3af13d290cc710ecf)
2013-02-08 16:05:51 -05:00
Zev Benjamin 3e7a6619bd puppet: Use exec's 'creates' parameter instead of onlyif
(imported from commit 94b5e76ca5119443f143e4af5c86e3c16c99dc1e)
2013-02-08 16:05:51 -05:00
Zev Benjamin a7ca48e2ff puppet: Use common::line instead of our own common::append
(imported from commit 230efb3409eaa451fa28d1655b27a2f5e5f9d382)
2013-02-08 16:05:51 -05:00
Zev Benjamin eba1008b1c puppet: Use absolute paths in execs
(imported from commit 426c572a4653ad44aa315b43d49c0d6ce001a58d)
2013-02-08 16:05:50 -05:00
Zev Benjamin 5c6a3f3e66 puppet: move all our puppet classes into a 'humbug' module
(imported from commit 69b42598c003bbe85dfa4266c56dd019304ea7fb)
2013-02-08 16:05:50 -05:00
Keegan McAllister bcec450c49 [manual] Remove hunt server from Postgres config
Deployment steps: TBD.  Will do this when the commit hits master.

(imported from commit 1eebbe873d520cb44c6605845afc9421448c6fe4)
2013-02-08 13:33:28 -05:00
Tim Abbott a306c28aa2 puppet: Add documentation on setting up the API distribution site.
(imported from commit ea298e8123bb5fee079cf969802fcb8201ed3111)
2013-02-07 14:28:06 -05:00
Tim Abbott 4cd3fd234c puppet: Add supervisord configuration for feedback-bot.
(imported from commit c7deece3e48d59de856393a4a6b7929757bc1c7c)
2013-02-05 14:27:56 -05:00
Tim Abbott f5b44cf349 nagios: Add monitoring for zmirror subscriptions syncing.
(imported from commit 2e4ae2c35d589f14b57758cd68a58f8b49b7ecf3)
2013-02-05 14:27:56 -05:00
Tim Abbott a7281f7e5a Add notes to puppet config about manually deployed symlinks.
(imported from commit 219f3b407bd83e0728f049820ad06092d6eed12a)
2013-02-01 16:04:11 -05:00
Tim Abbott 3c6dc21b05 Add pagerduty_nagios.cfg to git.
(imported from commit 2f7110d5ab65893afcb83e6f38944bf065abedff)
2013-02-01 14:50:28 -05:00
Tim Abbott 26aece90b8 nagios: Enable the Nagios commands feature.
This allows us to in particular reschedule a Nagios check to run
immediately, which I've in the past found super useful when trying to
figure out whether we actually fixed a problem.

Unfortunately, Nagios config sucks and there's no easy way to create a
group containing all of us as people able to issue commands; you have
to list them in like 8 different places.

(imported from commit 2c1e53330eff1e47e09d0b1917136f101d64e86a)
2013-02-01 14:50:28 -05:00
Tim Abbott 1fe6045288 nagios: add check that process_user_activity is running.
This fixes trac #670, and also adds the "-u humbug" parameter on the
other check_procs run, since that is a good practice move to help
avoid the check counting its parent process as one of the matches.

(imported from commit 43ae9b4863ba67579a21c86a910b73019f85a538)
2013-02-01 14:50:28 -05:00
Tim Abbott 2dd2bc8759 nagios: Make default contact_groups not page.
This will help us avoid making things accidentally pageable.

Also, explicitly set contact_groups for all our services, to help
encourage making explicit decisions about which new items are
pageable.

(imported from commit 740c6550d4a7091e58681435eeb7aaabf98df75c)
2013-02-01 14:50:28 -05:00
Keegan McAllister 6990260b59 [manual] Minify JavaScript and CSS in production
Manual deployment steps: The same Nginx reload as for "Get rid of the
static-access-control mechanism".  If deploying both commits at once,
just do it once.

(imported from commit dd8dbbf14b95fce0a4b6f66f462fa0a6b50bfb8c)
2013-01-31 15:41:01 -05:00
Keegan McAllister ee6f668c4d puppet: Install django-pipeline on app servers
This is in Debian unstable but unfortunately not older versions.

(imported from commit b82654edef270ef06fcf5015dd5d20db1c5c92f5)
2013-01-31 15:34:13 -05:00
Keegan McAllister f57126d169 puppet: Install yui-compressor on app servers
We have lots of choices for JS / CSS minifier.  This one works fine
and is in Debian.

(imported from commit bb5a05a8e59d0821e746116af0ef7e3c8ad59aaa)
2013-01-31 15:34:12 -05:00
Keegan McAllister 5e9b0ba79d [manual] Get rid of the static-access-control mechanism
We will minify our code, rather than trying to restrict who can see the
un-minified code.  Removing access control first simplifies things.

Manual deployment steps:

    scp servers/puppet/files/nginx/humbug-include/app root@staging.humbughq.com:/etc/nginx/humbug-include/
    ssh root@staging.humbughq.com service nginx reload

and then the same for app.humbughq.com once deployed to prod.

(imported from commit 63788aa3fa7ba5fd97fcf85b05760abb5e7cae4b)
2013-01-31 15:34:12 -05:00
Leo Franchi 6e9b8d895c Add munin plugin for send-receive timing
(imported from commit e2ae0775379ce59ab43213e68ade4d3f88b578e6)
2013-01-31 13:02:57 -05:00
Jessica McKellar 14d0ec1096 nagios: add several postgres checks.
(imported from commit 5440b2b14d5db11fa9794fe4bcb86a1d6fe90b5d)
2013-01-30 10:55:35 -05:00
Jessica McKellar a5337033b7 nagios: add a send-receive delay check.
(imported from commit ed58f49440fc1e8175ea02eb5d1b0ae8b53472f0)
2013-01-30 10:55:35 -05:00
Zev Benjamin 726ba8dad9 Make Postgres have a log prefix more like what pgFouine requires
We'll still need a conversion script, but it should be easy.

pgFouine requires a log prefix of '%t [%p]: [%l-1] '.  We instead use
'%m [%c]: [%l-1] ' which contains strictly more data.  Specifically,
"%m" is "%t" (time) but with milliseconds and "%c" is "%p" (pid) but
with the process start time.

(imported from commit a0bb583b563bdea0ca19b8b21677df0b9a18092a)
2013-01-28 16:21:42 -05:00
Jessica McKellar 767bf16c1c Hack up paths to be able to import both the API and Django model.
(imported from commit ca89d6bf6208455db4b636198737698ffe575698)
2013-01-24 13:36:11 -05:00
Luke O'Malley 61843b8645 nagios: Add plugin to watch the latency for a message roundtrip.
(imported from commit 75888fa4f7ceedb4a95e9b6c4012c32e106ee1ad)
2013-01-24 13:36:11 -05:00
Tim Abbott 2be39640d3 Add postgres config for new frontend.
(imported from commit 0b67ec1cb2c4b06d85d875c14154dd3e453f05c2)
2013-01-17 22:08:39 -05:00
Tim Abbott 54748643d5 puppet: Install South on our servers.
(imported from commit 18b25c6329508a235229992da48a9888b2cc244c)
2013-01-17 22:08:39 -05:00
Keegan McAllister c9a555b605 Nginx: Drop caching directive for /static
This might fix problems where users were running old code even after reloading.

(imported from commit dedc4d513f884aa2bafa0c7cc7a817d6715b48a0)
2013-01-16 15:03:40 -05:00
Luke Faraone d0a5d7f7e2 Serve static content in /dist on app
(imported from commit b5850ee1f6c6663a27fee14f430f1fae7b690725)
2013-01-15 19:10:09 -05:00
Tim Abbott e592e71515 [manual] Use rabbitmq queue to process UserActivity.
Before this is deployed, we need to install rabbitmq and pika on the
target server (see the puppet part of this commit for how).

When this is deployed, we need to start the new user activity bot:

./manage.py process_user_activity

in the screen session on the relevant server, or user_activity logs
won't be processed (which will eventually result in all users getting
notifications about how their mirrors are out of date).

(imported from commit 44d605aca0290bef2c94fb99267e15e26b21673b)
2013-01-14 13:28:23 -05:00
Keegan McAllister 6d7ef69cda nginx: Add config for plant.humbughq.com
(imported from commit e90b8e350014b49de53bfd5640442060672e691d)
2013-01-11 17:41:11 -05:00
Keegan McAllister 56660f30f8 nginx: Factor out shared parts of app / staging config
(imported from commit e00d5eec1bc58754db6e97935bc803fe3a4fe291)
2013-01-11 17:39:51 -05:00
Keegan McAllister ef6a5220c8 nginx: Remove unused config humbug-dev
(imported from commit 178a320bf56076c61f4010bf6cb89ba04798b4a4)
2013-01-11 17:39:48 -05:00
Jessica McKellar 9730a65f59 nagios: revamp check_user_zephyr_mirror_liveness to monitor sudden drops in mirror use.
(imported from commit e92df66c40065584e84c049cfab8d82f71d6dddd)
2013-01-08 10:53:33 -05:00
Jessica McKellar 0655397536 Give the NTP check the default number of retries.
It had a max_check_attempts of 1, which makes it susceptible to
network blips.

(imported from commit 20e51878d75bef36d02c5afaab78b8cdd701077f)
2013-01-08 10:53:33 -05:00
Jessica McKellar 8d0a17cbc3 puppet: fix installed packages typo in humbug_apache_base.
(imported from commit f503c767cadd9ce5f501233859faafd652f2c4e8)
2013-01-08 10:53:33 -05:00
Jessica McKellar c186e8ad96 puppet: Add a humbug_bots class with supervisord dependencies.
(imported from commit f5bdf6bccf10c7c7f21cc96c415014a26d04c019)
2013-01-08 10:53:32 -05:00
Jessica McKellar 62284f39f4 nagios: monitor feedback bot liveness.
(imported from commit 64a97e74b8a44bf0a6faf97398f843d8209b8e36)
2013-01-08 10:53:32 -05:00
Jessica McKellar 5d7b64993b nagios: Add monitoring for clock skew.
(imported from commit 1db47e7c6b28c9dd119e4c50309867d52d3c294b)
2013-01-03 10:21:16 -05:00
Jessica McKellar ee0b01b8a3 puppet: munin: Document the manual SSH tunnel setup required.
The full documentation, referenced in the config file, is at
https://wiki.humbughq.com/Deployment%20process/components#munin.

(imported from commit b7f989accb2ee8c5f400e68bf7a7491115a7d0b3)
2013-01-02 17:41:50 -05:00
Jessica McKellar 7c7263ebfe puppet: Add munin packages.
(imported from commit 4cefc2505b03df7de42c8e7e2adcff9490753476)
2013-01-02 17:41:50 -05:00
Jessica McKellar 9083b0f184 puppet: Add munin and munin-node config files.
(imported from commit fa9d7b191fe89894f61f4fd15cb7382663e34837)
2013-01-02 17:41:50 -05:00
Jessica McKellar d8cd78ec85 nagios: Add and make the default contact a PagerDuty group.
(imported from commit 6ab1fd777f3ec7804e6b4f31eaa5efad51993f1a)
2013-01-02 17:41:50 -05:00
Jessica McKellar cfad014596 nagios: Do check_user_zephyr_mirror_liveness as user humbug.
That user has the necessary database certs.

(imported from commit 2f0778a1c5ca5259143b8e7ab25b557a6ddd76df)
2013-01-02 17:41:49 -05:00
Zev Benjamin a40b5da432 puppet: Use PostgreSQL's internal logging system
This also requires disabling logrotate for postgres log files.

(imported from commit eeedb87a4f488829c59eddecc041654e762d6d0e)
2013-01-02 16:56:57 -05:00
Zev Benjamin f7237ac2aa puppet: Install postgres config files
(imported from commit a01bd1d0b14b1436e50605fc14c6267fe77989fc)
2013-01-02 16:56:57 -05:00
Zev Benjamin 779191b30e puppet: Add postgres server configuration files
(imported from commit bbfe6e9246a9a172a48c4cf8257d32936de009f9)
2013-01-02 16:56:57 -05:00
Tim Abbott 45e6550270 Add starting point for zmirror puppet configuration.
I expect this will be fleshed out more later.

(imported from commit c05e994e737ca2ab4ca24d4a4ac03ce46c52336a)
2013-01-02 15:03:42 -05:00
Tim Abbott a2f26f1106 Nagios: Fix retry interval of zephyr_mirror_forwarding check.
(imported from commit eae984669dad0a2dd6779092e9759909fbbd1da7)
2012-12-19 11:21:47 -05:00
Zev Benjamin 1aa825e6d0 puppet: Add generic nagios monitoring for postgres.humbughq.com
(imported from commit 9e732b69580bc3da8507a5fe6fdd81f044fb4443)
2012-12-13 11:30:02 -05:00
Zev Benjamin cd73e13601 puppet: Add python postgres module (python-psycopg2) to humbug_app_frontend packages
(imported from commit 3f41629d6f1e2c26458e223bc2135a53ac3bdd14)
2012-12-13 11:30:02 -05:00
Zev Benjamin dc6d48611d puppet: Accept traffic on port 5432 (postgresql)
(imported from commit bf30d0af2377209f3d5c10add3a526a1fee28dd8)
2012-12-13 11:30:02 -05:00
Zev Benjamin 4d2899b5f8 puppet: Add postgres config
(imported from commit ca932a1a1af7e7236ff1f47785acf4b412b16650)
2012-12-13 11:30:02 -05:00
Zev Benjamin 155e2c4943 install-server: Allow users to use an alternate humbug root
(imported from commit 1b5e57c1ec8c175733c8fb15343b096c46e6b6b2)
2012-12-13 11:30:02 -05:00
Zev Benjamin ab373d6457 install-server: Use "apt-get -y" instead of "yes '' | apt-get"
(imported from commit 0157f179928d69a5f0ff574a2d003187f28c1772)
2012-12-13 11:21:25 -05:00
Zev Benjamin 11f8dc644d install-server: Quote filenames in existance checks
(imported from commit ed17b65d875b5321c57fff4e16263282cccf4dff)
2012-12-13 11:21:20 -05:00
Zev Benjamin 16a5af0b8d install-server: Accept apt-get prompts
(imported from commit 2f69d047488d3d82689a4fc71777e3c4667b36d5)
2012-12-13 11:21:16 -05:00
Zev Benjamin b6b0ab80cb install-server: Check for humbug-self-signed.key before running
(imported from commit 0c5ab50fbb278db740690522e2354f33f1958cc7)
2012-12-13 11:21:11 -05:00
Zev Benjamin d90fb5d00f install-server: Use named constants for file paths
(imported from commit 6178f8110c6f79c642dd3c8cde149be6e4d72e16)
2012-12-13 11:21:05 -05:00
Jessica McKellar 375f8e3540 nagios: disable flap detection.
This will ensure that we always get state change alerts, even when the
service is changing states frequently.

(imported from commit 57fa5a941dd1a6042eb782dbac2fed0e4cb934ba)
2012-12-11 10:22:52 -05:00
Keegan McAllister 5212a48d3b puppet: Only install ipython on the app servers
I'd like to have this everywhere, but it has a bunch of X dependencies.

(imported from commit c0c4089909ab7b3a5b6f9620c19eb0435b72762c)
2012-12-05 14:12:36 -05:00
Keegan McAllister 01b070a122 puppet: Install emacs without X support
(imported from commit b15e63613c6b6cf1815a8f5bb660bd8c8e80604c)
2012-12-05 14:12:36 -05:00
Keegan McAllister c34d39caf8 puppet: Separate out some packages needed only by the app servers
(imported from commit 447837f1d5f68d0bf160dec2a9a37fc1cb7e62d5)
2012-12-05 14:12:36 -05:00
Keegan McAllister d8b4cefccb nagios: Remove AllowOverride AuthConfig
We don't use it.

(imported from commit 875148e24e0de2815737b6bc03eeb7f1cb8d770d)
2012-12-03 17:54:16 -05:00
Keegan McAllister 2cf49c4ff2 nagios: Go straight to the service detail page
This bypasses the side navigation frame, but I think said frame currently
provides negative value.

(imported from commit b067d546e4a7fb95e7de2a35be7e7f947c7a0da1)
2012-12-03 17:54:16 -05:00
Keegan McAllister d435f29308 Add X-Frame-Options header on nagios, trac, wiki
Prevents clickjacking attacks.

(imported from commit 8b3872e607d8a4e714c280a3226465fde0d5a6ed)
2012-12-03 17:54:16 -05:00
Keegan McAllister 7c495d7232 Move the nagios Apache authentication directives to a <Location> block
Following the trac Apache config.

(imported from commit 01e773f2361d85f45f190f6ade2510b84a2f88ee)
2012-12-03 17:54:15 -05:00
Keegan McAllister 41319fe820 Rework the nagios Apache config as a proper vhost
This also adds HSTS.  Based on the trac Apache config.

Fixes #435.

Suggested viewing: git show -w

(imported from commit e7e9fe74687b88497ddb21f74febfc7fdf9b1979)
2012-12-03 17:54:15 -05:00
Keegan McAllister a9c16b38ce Fix up whitespace in Apache configs
(imported from commit 605253abf9b029e18774f80979d23c60ffca034b)
2012-12-03 17:54:15 -05:00
Keegan McAllister 922b44a1da Add iptables config for zmirror.humbughq.com
For now we allow all UDP traffic.  I'll look into doing something clever.

This isn't puppetized, either.

(imported from commit bdf53df87a5f6c8af6d950b25946b5ec8a4f910b)
2012-12-03 17:43:04 -05:00
Keegan McAllister ed0cb0a5f8 Puppetize nginx.conf
Fixes #201.

(imported from commit 0feaff372d94009fa51dabf2bda55062826e2ed5)
2012-12-03 15:58:16 -05:00
Keegan McAllister 4aa7615234 Nginx: Use $host instead of $server_name
The latter is just the first name in the 'server_name' directive.
The former uses the HTTP Host header, if provided.

This fixes the redirect

    from http://zephyr.humbughq.com
    to   https://zephyr.humbughq.com

(imported from commit be47b05f4f055bb2d1d82aebbe155579f49c538d)
2012-11-30 17:12:42 -05:00
Keegan McAllister 500a5e29c3 Nginx: Redirect unknown hostnames to https://humbughq.com
(imported from commit f6dd65c1db033d09f1df8f0a5972f067f3aeb80a)
2012-11-30 15:32:32 -05:00
Keegan McAllister ac18c533c8 Nginx: Serve the cert for zephyr.humbughq.com rather than app.humbughq.com
This will cause SSL errors for anyone still using the deprecated
app.humbughq.com name, which we concluded is (almost?) nobody.

(imported from commit 7f3c149a4064e7bdae8ec944f2bb8a482df6f90d)
2012-11-30 15:32:32 -05:00
Keegan McAllister 2fcb9cfd49 Nginx: Make zephyr.humbughq.com an alias for humbughq.com
(imported from commit d23ef5aeed990a04f294b7dffe322b8d174c1f07)
2012-11-30 15:32:32 -05:00
Keegan McAllister 0f20150a81 Nagios: move /var/lib/nagios/humbug-api to /usr/local/lib/humbug
(imported from commit ff3ff1e3cc54a4c556479e62e058002229143627)
2012-11-26 16:58:51 -05:00
Keegan McAllister d7b3afef6b Send Nagios alerts to Humbug
Fixes #385.

(imported from commit 7dac013debd6ccff031fc4da0dd7185e198b4498)
2012-11-26 14:42:55 -05:00
Keegan McAllister b609840e82 puppet: Install memcached on app servers
We use the default Debian configuration, which listens on localhost only.

(imported from commit efa8333c7fa423e71a99ec06b2b420cae36fddfb)
2012-11-26 11:59:48 -05:00
Jessica McKellar be27ec1ad4 nagios: Change zephyr mirror liveness check to only care about aggregate statistics.
Too many individual users occasionally don't update their mirrors,
causing us to be permanently alerting; we have sufficient user
notification at this point (plus Waseem keeping an eye on /activity)
that we don't need to alert on individual users.

We do, however, still care if something happens (say, Linerva going
down) that causes many users' mirrors to go down.

(imported from commit 392952c95739e183d4a711120e3a963671cec289)
2012-11-26 10:31:29 -05:00
Keegan McAllister 75526a2c67 nagios: Drop ssh -o StrictHostKeyChecking=no
This is bad for security.

I've checked that all currently known hosts for nagios@nagios.humbughq.com
match one of our existing servers.  When adding servers to nagios in the
future, it will be necessary to do an initial manual ssh from nagios@ and check
the host key fingerprint.

(imported from commit adfd1d29f03343d4be04e87c5e26a018f31e5194)
2012-11-26 00:25:15 -05:00
Keegan McAllister 043f0d8e15 nagios: Use lowercase host aliases
(imported from commit c653d5948894e651a5040339e8cd6af50af712b3)
2012-11-26 00:25:15 -05:00
Keegan McAllister 1939b55b5a nagios: Remove wiki.humbughq.com
This is the same machine as git.humbughq.com.

(imported from commit 8aa9306668d672052aa38a2f4453cb0127ab5cc5)
2012-11-26 00:25:15 -05:00
Keegan McAllister f761643724 nagios: Monitor bots.humbughq.com
(imported from commit 83cb5cc3c3c4bf54e1339d877bd60cd05586783b)
2012-11-26 00:25:15 -05:00
Keegan McAllister f8a065ed2c nagios: Remove dev.humbughq.com
This is an alias for staging.humbughq.com.

(imported from commit b2d2777e57773052dd59b3f5c067e23eafb60681)
2012-11-26 00:25:15 -05:00
Keegan McAllister b9c8f4a770 nagios: Enable ssh-based checks on all machines
(imported from commit 3905ad03cc2ed5dec6f8eb6b20d4b4f0896f164c)
2012-11-26 00:25:14 -05:00
Keegan McAllister 25916d9ce5 nagios: Check Debian update availability
(imported from commit 59da03c409281b6b2f5cf3612e5f7bd0caa76226)
2012-11-26 00:25:14 -05:00
Keegan McAllister 4b1a2e8a8a Install nagios-plugins-basic on every machine
We need this for check_debian_packages.

(imported from commit 588dfbe7d5b69acfd1db7fcf4060b64ec5151c2d)
2012-11-26 00:25:14 -05:00
Keegan McAllister 5836462cb4 Import check_debian_packages Nagios plugin
From http://exchange.nagios.org/directory/Plugins/Operating-Systems/Linux/check-debian-packages/details

(imported from commit 6304a2aa315a91fd48e9ad79fcdb584ba8a2ccb4)
2012-11-26 00:25:14 -05:00
Keegan McAllister 685deba16b Puppetize APT::Periodic config
(imported from commit 2ccdeb4f9c8173a83c7014987977304187651f67)
2012-11-26 00:25:14 -05:00
Keegan McAllister 8dd1f1efc0 Puppetize iptables config
(imported from commit aa58d06255aaf5a2979a7fcc4e0746c1ac2d91a7)
2012-11-19 11:06:33 -05:00
Jessica McKellar 905f2d3235 nagios: add monitoring for the liveness of our users' zephyr mirrors.
Using the check_user_zephyr_mirror_liveness plugin.

(imported from commit c17e112fe8696fab583a0dbc228ea9fb6e6988b0)
2012-11-16 11:36:33 -05:00
Jessica McKellar 5498557b4b nagios: add monitoring for Zephyr mirroring.
Using the check_zephyr_mirror plugin.

(imported from commit 8ef5d4870c3a2ec547729c191de838504cea1d3d)
2012-11-16 11:36:16 -05:00
Jessica McKellar b86ddf4ddc nagios: add a check_user_zephyr_mirror_liveness plugin.
It will alert when our users' mirrors don't appear to be running, as
assessed by having recently made a get_message API request.

(imported from commit 4b8c5f51b007568a90a92f7b095c51f3566d5117)
2012-11-16 11:28:37 -05:00
Jessica McKellar fcf5eb8f1f nagios: add a check_zephyr_mirror plugin.
It checks the output of api/bots/check-mirroring and alerts if we
aren't able to send and receive mirrored Zephyrs.

(imported from commit 6c9abc380fca955d00462f829fa7dcadfef24221)
2012-11-16 11:28:28 -05:00
Jessica McKellar a0aa1b31c8 nagios: Add remote disk and load checks.
(imported from commit 1f0a1f5540212357ac2ed0c8d50fb2291a1812ed)
2012-11-16 11:25:22 -05:00
Jessica McKellar 5ec66f467b nagios: add basic monitoring for new servers, and a hostgroup for SSH-based checks.
(imported from commit 7e5ad2bb024eb935bf6640a894cad762e45c0ab0)
2012-11-16 11:25:14 -05:00
Jessica McKellar 010f15c66e nagios: add a test contact and contact group for testing new alerts.
(imported from commit 9cc1ef2b7af6c84bfd87dc38c6a558ea3b36d267)
2012-11-16 11:24:46 -05:00
Jessica McKellar 609ff161a1 nagios: send the full multi-line alert data in Nagios e-mails.
(imported from commit c906bd2b6a2a1e0f009e4743a0f7b1968f371919)
2012-11-16 11:24:45 -05:00
Tim Abbott a4289e6553 Install the 'host' command on our servers.
(imported from commit 21171e553cf6974cd19170c47a79e3e7389b5534)
2012-11-14 16:57:21 -05:00
Keegan McAllister 35171b9d3e Tweak gitit config
The "signup code" was left over from a very early era.  We now use HTTP auth
and there's no way to register an account within Gitit at all.

(imported from commit 20f1e10de1fd978d0045c2fed2254e37ab6f7b6c)
2012-11-07 17:46:46 -05:00
Keegan McAllister b9452b5644 apache/ports.conf: Document which part is custom
(imported from commit b25c4ce8847509ce07d98e1caee402aa33369c4d)
2012-11-07 17:46:46 -05:00
Tim Abbott adab0c1880 Add staging/dev certs to puppet configuration.
(imported from commit 1415d909a4619adecc3b43ad0b7817f473bc2a73)
2012-11-06 16:59:28 -05:00
Tim Abbott 0c25a091d2 Install ntp on all our servers.
(imported from commit 59cb9ef4350a8ec9a528623fb3247e7ba6c15405)
2012-11-06 14:19:14 -05:00
Tim Abbott 4a3bf99fa0 Add staging server nginx configuration.
(imported from commit 560621e48098925d526c7a29681dc03c4508a878)
2012-11-06 14:12:18 -05:00
Tim Abbott 4aa91336b5 install-server: Set the hostname automatically.
(imported from commit c8081845eac0cf8d21711eb7c836f2ce39a6d9f6)
2012-11-02 10:50:59 -04:00
Tim Abbott 44bee33c41 Setup servers id_rsa for the humbug user as well.
(imported from commit 19a36fc829882eff7bfdab3a22765c5df6bdbd0d)
2012-11-02 10:50:59 -04:00
Tim Abbott 7d35c3135b Puppet: Update default classes for building new server.
(imported from commit 7283498779108992456c98d3d18b01751ccbb5b6)
2012-11-02 10:50:59 -04:00
Jessica McKellar c0b75ed93a Add Nagios config files to git.
(imported from commit 5d6ba166cf35afdd76ca4f2cfc8a13988cfdeaea)
2012-11-01 10:47:50 -04:00
Tim Abbott 2c577d70b3 Restart apache2, ssh, and nginx after running puppet.
(imported from commit 0e0b7fd10d1742efe86ff6bdab8d3ac4e4d291a9)
2012-10-30 14:21:31 -04:00
Keegan McAllister 545476c6a4 Puppet: Don't install sudo
We're no longer using it, and fewer setuid programs is better.

Fixes #225.

(imported from commit 68b06bb8afedc0854d96ad072b5de718832932ed)
2012-10-30 12:30:18 -04:00
Keegan McAllister f0c2421f00 Customize the 404 error page Nginx serves for missing static content
(imported from commit 70fc821f9ae29b8a902c48ce57e39273c90f57ff)
2012-10-30 11:00:10 -04:00
Keegan McAllister 233cba1380 Remove old copy of my.cnf
This is now found at servers/puppet/files/mysql/my.cnf .  It's identical to the
one we remove here.

(imported from commit fe70529a716842a57058386eaf697a23a6d6251c)
2012-10-30 00:03:31 -04:00
Keegan McAllister 9de0d3c7b8 install-server: Add note about hostname domain
(imported from commit 1fb9fd79cfc88d7ec191b1116b45e4b7b7bb9c2f)
2012-10-29 23:21:00 -04:00
Keegan McAllister e9a8d3e447 install-server: Restart sshd after configuring it
Needed for disabling password authentication to take effect.  Fixes #202.

(imported from commit f082b08076ecb1f0facb3f66d5b416ea545dfa4f)
2012-10-29 23:21:00 -04:00
Tim Abbott 85ead77956 Splitting SSL out didn't fix the dependency problems.
(imported from commit 14515ea1abecb6212842a4a5cba90eb705f65755)
2012-10-29 13:19:41 -04:00
Tim Abbott e6f196f372 Move certs .gitignore to where the certs are now.
(imported from commit c1a92978a7d835cb3d3eec5647ef7aa4f4f73b35)
2012-10-29 13:19:41 -04:00
Keegan McAllister 0f211673ac Puppet: Add missing packages python-flup
Needed for Django runfcgi.

(imported from commit cfd1f20a2f7a08c21e8ab3b321c2928a28319a54)
2012-10-27 12:11:22 -04:00
Keegan McAllister d71b9594fe Configure nginx for the new server
(imported from commit d073276912ea844e75fd710689f152fd7a2213c7)
2012-10-27 11:38:15 -04:00
Keegan McAllister 92b10e3bc2 settings.py: Change deployed check
This is security-critical so we have two checks.

(imported from commit adaa1cefe2d08526cdaac2fb0d8cc02773390224)
2012-10-27 11:18:51 -04:00
Keegan McAllister cbdbc12ab0 Remove config for Apache as app frontend
(imported from commit ae4072bdf59cdfccec76eeee7fd3b99a899eaa19)
2012-10-27 10:56:14 -04:00
Tim Abbott fdba3addc8 Update puppet configuration to be slightly more accurate.
This still needs a lot of work though.

(imported from commit 4472488c399f7c5b96bcf900b1a5e957625cb450)
2012-10-26 14:58:05 -04:00
Tim Abbott 8c20bafb98 Add Nagios configuration to puppet.
(imported from commit 34c09661e63b31bd177b9704b69a0d8d0d644de7)
2012-10-26 14:15:27 -04:00
Keegan McAllister f8540dcdae Wrap some other extremely long lines
(imported from commit e7d55f318c8865ca953bf4520d1b07f7e84a4aeb)
2012-10-25 15:22:18 -04:00
Keegan McAllister 2f5e7ba1ad Properly format wiki code highlighting CSS
(imported from commit 216ff55aa55847fa61a32a1d87737de273c21ae4)
2012-10-25 15:22:18 -04:00
Tim Abbott 811604021e puppet: Install requests module on servers.
(imported from commit 2aa2e5740065e9e46020cca0d22854e57e81a407)
2012-10-24 16:24:02 -04:00
Tim Abbott 7be626ff2f Run trac on top of Apache rather than the standalone server.
(imported from commit 2e9ee69a6f36b4c145d83abdf975bbe5d7ec1c7d)
2012-10-23 15:58:02 -04:00
Tim Abbott c5f262987f Add apache2 site configuration for trac.
(imported from commit 81d173070a449168d6d8e08a557134dbda66f2bb)
2012-10-23 14:52:04 -04:00
Tim Abbott 7079efba8d puppet: Make humbug's authorized_keys file owned by humbug.
(imported from commit 4237203722d7782cb5c479e408966494bc703149)
2012-10-23 14:52:04 -04:00
Tim Abbott 7a56448a35 Move humbug-self-signed.key to new certs directory.
(imported from commit ed8ff32739e21cca3d6173e19bb425da8a3a18ea)
2012-10-23 14:52:04 -04:00
Tim Abbott 9ed02e220c puppet: Disable creating new servers as wiki/frontend for now.
(imported from commit 20250365a3d82479d5dd33ecb0ee9b5c3db68029)
2012-10-23 13:32:34 -04:00
Keegan McAllister 9ff633ccfe Add nginx config
This is not yet hooked up to Puppet.

(imported from commit 1d0368285a5a9ef5a4af0651001db4930cf77578)
2012-10-22 18:00:37 -04:00
Keegan McAllister 0dcc7c2914 puppet: Remove humbug from sudoers
(imported from commit cdc8aafdcbfafe5fe97a18dfd32b5a7f15a77102)
2012-10-15 14:56:12 -04:00
Keegan McAllister 45273643ec Move humbug-self-signed.crt into a common directory
(imported from commit 0c914b87c06be7cd7b370d8d0f38efc9f1aaf57f)
2012-10-15 13:29:47 -04:00
Tim Abbott b040615cbe Move our server configuration into puppet.
(imported from commit fb1c096b46f23c56f2e08952cbbcc99b34ae0586)
2012-09-20 17:00:24 -04:00
Tim Abbott 1d36bbecef puppet-apt: Make priorities configurable.
(imported from commit 82ca93e510ec07251fed4c32bc0165c6e9187d6b)
2012-09-20 17:00:24 -04:00
Tim Abbott d757b630bf Import puppet-common from https://github.com/camptocamp/puppet-common.git
(imported from commit bb3ccac0dd0cc5688be0f1487092cbe34b107002)
2012-09-20 17:00:24 -04:00
Tim Abbott 5a4a5b0fc0 Import puppet-apt from https://github.com/camptocamp/puppet-apt.git.
(imported from commit 4940c1479b518971e1f3513315b046a571323604)
2012-09-20 17:00:24 -04:00
Keegan McAllister a8a8be2d8d Distinguish deploy server by hostname
(imported from commit 685001e49f8add7540ae807acb8d3a604a969d4f)
2012-09-17 18:15:42 -04:00
Keegan McAllister e6b80c890d setup.bash: More packages
(imported from commit a8a68fdb7a4d0e796ce9ab65318d5d3f297c1e72)
2012-09-17 18:11:56 -04:00
Tim Abbott 91d2f3d4b9 Changes suggested by Percona tool for our setup.
https://tools.percona.com/wizard

(imported from commit 350e18a68346aab0fda91eed6c517bc5e9f66b99)
2012-09-17 11:58:26 -04:00
Tim Abbott 9a7e3b2b5b Default Debian Squeeze my.cnf.
(imported from commit 5feac5683bbe67c73cf3828ebb03522f9fe63dde)
2012-09-17 11:56:14 -04:00
Tim Abbott 456a2536d1 setup.bash: Add python-simplejson.
(imported from commit 0e7f297ba5b5dc565693ed6e2a67eb12def329a4)
2012-09-17 11:42:07 -04:00
Keegan McAllister ffa0b74bca setup.bash: Add a note about wiki users
(imported from commit be00de3b777b0d9c779787dbcadb32ab29f248ef)
2012-09-13 11:30:14 -04:00
Keegan McAllister f25102df32 Add Apache ports.conf
(imported from commit c99ceef008176ca380d242e7b5fcd62d2723099d)
2012-09-13 11:30:11 -04:00
Tim Abbott af4ae41c1a setup.bash: Add humbug user to sudoers.
(imported from commit 1286b358f41317ab2672c6958110a7a942b18a34)
2012-09-13 10:58:30 -04:00
Tim Abbott 67a4a2af3e Add missing apache module.
(imported from commit 13ce6b93cb2e92716229b441e0ac32fd4d690233)
2012-09-13 10:56:20 -04:00
Tim Abbott af9caf4d59 setup.bash: more tweaks.
(imported from commit 459b5d6174831d4526c25de8ed66ad62a63775b8)
2012-09-13 10:56:20 -04:00
Tim Abbott d997eacc5e setup.bash: Some fixes to server installation process.
(imported from commit 13a675495cc561060c91958c5f217f706a1f4501)
2012-09-13 10:56:20 -04:00
Keegan McAllister 42fd6786f6 wiki: Indent non-highlighted code blocks
(imported from commit 2e85c5a84fdd8e20d82c341ba7b22b4cd6bbbef9)
2012-09-08 22:18:27 -04:00
Keegan McAllister 890420de49 Add wiki stylesheet
(imported from commit 8b9511cd548bcb3a131c14a2bcb2dfa4e5044733)
2012-09-06 21:31:43 +00:00
Keegan McAllister 9e3f588276 Install django-jstemplate on new servers
(imported from commit 650cc012030a3fe09ebae229ab3bff55e1cd35f0)
2012-09-06 15:48:15 -04:00
Keegan McAllister db858afe4a apache: Timeout talking to app after 20 minutes only
We need this to be longer than the client-side AJAX timeout (future commit)

(imported from commit a638c8392fd7ee1e5c1bee557012dbdd28f1547f)
2012-09-05 15:53:40 -04:00
Keegan McAllister 3078d9c0b6 Nuke tabs in Apache config
(imported from commit 0a858d7885294cef297bc4e0b3876af2fd1ab93e)
2012-09-05 10:36:01 -04:00
Keegan McAllister c7773ed2aa Script to automate setup of humbug-dev as currently configured (untested)
(imported from commit 8dcb85fad9f04dc393198f8ee2afcca23edf7b51)
2012-09-04 20:35:28 -04:00