Commit Graph

79 Commits

Author SHA1 Message Date
Keegan McAllister 0f20150a81 Nagios: move /var/lib/nagios/humbug-api to /usr/local/lib/humbug
(imported from commit ff3ff1e3cc54a4c556479e62e058002229143627)
2012-11-26 16:58:51 -05:00
Keegan McAllister d7b3afef6b Send Nagios alerts to Humbug
Fixes #385.

(imported from commit 7dac013debd6ccff031fc4da0dd7185e198b4498)
2012-11-26 14:42:55 -05:00
Keegan McAllister b609840e82 puppet: Install memcached on app servers
We use the default Debian configuration, which listens on localhost only.

(imported from commit efa8333c7fa423e71a99ec06b2b420cae36fddfb)
2012-11-26 11:59:48 -05:00
Jessica McKellar be27ec1ad4 nagios: Change zephyr mirror liveness check to only care about aggregate statistics.
Too many individual users occasionally don't update their mirrors,
causing us to be permanently alerting; we have sufficient user
notification at this point (plus Waseem keeping an eye on /activity)
that we don't need to alert on individual users.

We do, however, still care if something happens (say, Linerva going
down) that causes many users' mirrors to go down.

(imported from commit 392952c95739e183d4a711120e3a963671cec289)
2012-11-26 10:31:29 -05:00
Keegan McAllister 75526a2c67 nagios: Drop ssh -o StrictHostKeyChecking=no
This is bad for security.

I've checked that all currently known hosts for nagios@nagios.humbughq.com
match one of our existing servers.  When adding servers to nagios in the
future, it will be necessary to do an initial manual ssh from nagios@ and check
the host key fingerprint.

(imported from commit adfd1d29f03343d4be04e87c5e26a018f31e5194)
2012-11-26 00:25:15 -05:00
Keegan McAllister 043f0d8e15 nagios: Use lowercase host aliases
(imported from commit c653d5948894e651a5040339e8cd6af50af712b3)
2012-11-26 00:25:15 -05:00
Keegan McAllister 1939b55b5a nagios: Remove wiki.humbughq.com
This is the same machine as git.humbughq.com.

(imported from commit 8aa9306668d672052aa38a2f4453cb0127ab5cc5)
2012-11-26 00:25:15 -05:00
Keegan McAllister f761643724 nagios: Monitor bots.humbughq.com
(imported from commit 83cb5cc3c3c4bf54e1339d877bd60cd05586783b)
2012-11-26 00:25:15 -05:00
Keegan McAllister f8a065ed2c nagios: Remove dev.humbughq.com
This is an alias for staging.humbughq.com.

(imported from commit b2d2777e57773052dd59b3f5c067e23eafb60681)
2012-11-26 00:25:15 -05:00
Keegan McAllister b9c8f4a770 nagios: Enable ssh-based checks on all machines
(imported from commit 3905ad03cc2ed5dec6f8eb6b20d4b4f0896f164c)
2012-11-26 00:25:14 -05:00
Keegan McAllister 25916d9ce5 nagios: Check Debian update availability
(imported from commit 59da03c409281b6b2f5cf3612e5f7bd0caa76226)
2012-11-26 00:25:14 -05:00
Keegan McAllister 4b1a2e8a8a Install nagios-plugins-basic on every machine
We need this for check_debian_packages.

(imported from commit 588dfbe7d5b69acfd1db7fcf4060b64ec5151c2d)
2012-11-26 00:25:14 -05:00
Keegan McAllister 5836462cb4 Import check_debian_packages Nagios plugin
From http://exchange.nagios.org/directory/Plugins/Operating-Systems/Linux/check-debian-packages/details

(imported from commit 6304a2aa315a91fd48e9ad79fcdb584ba8a2ccb4)
2012-11-26 00:25:14 -05:00
Keegan McAllister 685deba16b Puppetize APT::Periodic config
(imported from commit 2ccdeb4f9c8173a83c7014987977304187651f67)
2012-11-26 00:25:14 -05:00
Keegan McAllister 8dd1f1efc0 Puppetize iptables config
(imported from commit aa58d06255aaf5a2979a7fcc4e0746c1ac2d91a7)
2012-11-19 11:06:33 -05:00
Jessica McKellar 905f2d3235 nagios: add monitoring for the liveness of our users' zephyr mirrors.
Using the check_user_zephyr_mirror_liveness plugin.

(imported from commit c17e112fe8696fab583a0dbc228ea9fb6e6988b0)
2012-11-16 11:36:33 -05:00
Jessica McKellar 5498557b4b nagios: add monitoring for Zephyr mirroring.
Using the check_zephyr_mirror plugin.

(imported from commit 8ef5d4870c3a2ec547729c191de838504cea1d3d)
2012-11-16 11:36:16 -05:00
Jessica McKellar b86ddf4ddc nagios: add a check_user_zephyr_mirror_liveness plugin.
It will alert when our users' mirrors don't appear to be running, as
assessed by having recently made a get_message API request.

(imported from commit 4b8c5f51b007568a90a92f7b095c51f3566d5117)
2012-11-16 11:28:37 -05:00
Jessica McKellar fcf5eb8f1f nagios: add a check_zephyr_mirror plugin.
It checks the output of api/bots/check-mirroring and alerts if we
aren't able to send and receive mirrored Zephyrs.

(imported from commit 6c9abc380fca955d00462f829fa7dcadfef24221)
2012-11-16 11:28:28 -05:00
Jessica McKellar a0aa1b31c8 nagios: Add remote disk and load checks.
(imported from commit 1f0a1f5540212357ac2ed0c8d50fb2291a1812ed)
2012-11-16 11:25:22 -05:00
Jessica McKellar 5ec66f467b nagios: add basic monitoring for new servers, and a hostgroup for SSH-based checks.
(imported from commit 7e5ad2bb024eb935bf6640a894cad762e45c0ab0)
2012-11-16 11:25:14 -05:00
Jessica McKellar 010f15c66e nagios: add a test contact and contact group for testing new alerts.
(imported from commit 9cc1ef2b7af6c84bfd87dc38c6a558ea3b36d267)
2012-11-16 11:24:46 -05:00
Jessica McKellar 609ff161a1 nagios: send the full multi-line alert data in Nagios e-mails.
(imported from commit c906bd2b6a2a1e0f009e4743a0f7b1968f371919)
2012-11-16 11:24:45 -05:00
Tim Abbott a4289e6553 Install the 'host' command on our servers.
(imported from commit 21171e553cf6974cd19170c47a79e3e7389b5534)
2012-11-14 16:57:21 -05:00
Keegan McAllister 35171b9d3e Tweak gitit config
The "signup code" was left over from a very early era.  We now use HTTP auth
and there's no way to register an account within Gitit at all.

(imported from commit 20f1e10de1fd978d0045c2fed2254e37ab6f7b6c)
2012-11-07 17:46:46 -05:00
Keegan McAllister b9452b5644 apache/ports.conf: Document which part is custom
(imported from commit b25c4ce8847509ce07d98e1caee402aa33369c4d)
2012-11-07 17:46:46 -05:00
Tim Abbott adab0c1880 Add staging/dev certs to puppet configuration.
(imported from commit 1415d909a4619adecc3b43ad0b7817f473bc2a73)
2012-11-06 16:59:28 -05:00
Tim Abbott 0c25a091d2 Install ntp on all our servers.
(imported from commit 59cb9ef4350a8ec9a528623fb3247e7ba6c15405)
2012-11-06 14:19:14 -05:00
Tim Abbott 4a3bf99fa0 Add staging server nginx configuration.
(imported from commit 560621e48098925d526c7a29681dc03c4508a878)
2012-11-06 14:12:18 -05:00
Tim Abbott 4aa91336b5 install-server: Set the hostname automatically.
(imported from commit c8081845eac0cf8d21711eb7c836f2ce39a6d9f6)
2012-11-02 10:50:59 -04:00
Tim Abbott 44bee33c41 Setup servers id_rsa for the humbug user as well.
(imported from commit 19a36fc829882eff7bfdab3a22765c5df6bdbd0d)
2012-11-02 10:50:59 -04:00
Tim Abbott 7d35c3135b Puppet: Update default classes for building new server.
(imported from commit 7283498779108992456c98d3d18b01751ccbb5b6)
2012-11-02 10:50:59 -04:00
Jessica McKellar c0b75ed93a Add Nagios config files to git.
(imported from commit 5d6ba166cf35afdd76ca4f2cfc8a13988cfdeaea)
2012-11-01 10:47:50 -04:00
Tim Abbott 2c577d70b3 Restart apache2, ssh, and nginx after running puppet.
(imported from commit 0e0b7fd10d1742efe86ff6bdab8d3ac4e4d291a9)
2012-10-30 14:21:31 -04:00
Keegan McAllister 545476c6a4 Puppet: Don't install sudo
We're no longer using it, and fewer setuid programs is better.

Fixes #225.

(imported from commit 68b06bb8afedc0854d96ad072b5de718832932ed)
2012-10-30 12:30:18 -04:00
Keegan McAllister f0c2421f00 Customize the 404 error page Nginx serves for missing static content
(imported from commit 70fc821f9ae29b8a902c48ce57e39273c90f57ff)
2012-10-30 11:00:10 -04:00
Keegan McAllister 233cba1380 Remove old copy of my.cnf
This is now found at servers/puppet/files/mysql/my.cnf .  It's identical to the
one we remove here.

(imported from commit fe70529a716842a57058386eaf697a23a6d6251c)
2012-10-30 00:03:31 -04:00
Keegan McAllister 9de0d3c7b8 install-server: Add note about hostname domain
(imported from commit 1fb9fd79cfc88d7ec191b1116b45e4b7b7bb9c2f)
2012-10-29 23:21:00 -04:00
Keegan McAllister e9a8d3e447 install-server: Restart sshd after configuring it
Needed for disabling password authentication to take effect.  Fixes #202.

(imported from commit f082b08076ecb1f0facb3f66d5b416ea545dfa4f)
2012-10-29 23:21:00 -04:00
Tim Abbott 85ead77956 Splitting SSL out didn't fix the dependency problems.
(imported from commit 14515ea1abecb6212842a4a5cba90eb705f65755)
2012-10-29 13:19:41 -04:00
Tim Abbott e6f196f372 Move certs .gitignore to where the certs are now.
(imported from commit c1a92978a7d835cb3d3eec5647ef7aa4f4f73b35)
2012-10-29 13:19:41 -04:00
Keegan McAllister 0f211673ac Puppet: Add missing packages python-flup
Needed for Django runfcgi.

(imported from commit cfd1f20a2f7a08c21e8ab3b321c2928a28319a54)
2012-10-27 12:11:22 -04:00
Keegan McAllister d71b9594fe Configure nginx for the new server
(imported from commit d073276912ea844e75fd710689f152fd7a2213c7)
2012-10-27 11:38:15 -04:00
Keegan McAllister 92b10e3bc2 settings.py: Change deployed check
This is security-critical so we have two checks.

(imported from commit adaa1cefe2d08526cdaac2fb0d8cc02773390224)
2012-10-27 11:18:51 -04:00
Keegan McAllister cbdbc12ab0 Remove config for Apache as app frontend
(imported from commit ae4072bdf59cdfccec76eeee7fd3b99a899eaa19)
2012-10-27 10:56:14 -04:00
Tim Abbott fdba3addc8 Update puppet configuration to be slightly more accurate.
This still needs a lot of work though.

(imported from commit 4472488c399f7c5b96bcf900b1a5e957625cb450)
2012-10-26 14:58:05 -04:00
Tim Abbott 8c20bafb98 Add Nagios configuration to puppet.
(imported from commit 34c09661e63b31bd177b9704b69a0d8d0d644de7)
2012-10-26 14:15:27 -04:00
Keegan McAllister f8540dcdae Wrap some other extremely long lines
(imported from commit e7d55f318c8865ca953bf4520d1b07f7e84a4aeb)
2012-10-25 15:22:18 -04:00
Keegan McAllister 2f5e7ba1ad Properly format wiki code highlighting CSS
(imported from commit 216ff55aa55847fa61a32a1d87737de273c21ae4)
2012-10-25 15:22:18 -04:00
Tim Abbott 811604021e puppet: Install requests module on servers.
(imported from commit 2aa2e5740065e9e46020cca0d22854e57e81a407)
2012-10-24 16:24:02 -04:00