Commit Graph

169 Commits

Author SHA1 Message Date
Leo Franchi 67a41143f5 [manual] Add rabbitmq monitoring to munin.
When deploying to prod, this will require a manual install of the
rabbitmq_* files to /etc/munin/plugins/ and an edit of
/etc/munin/plugin-conf.d/munin-node.conf

(imported from commit 4c10e634b04200dda1c4f4989e37fe232143240f)
2013-02-12 10:55:01 -05:00
Leo Franchi 0a0c4bb9a0 [manual] Use rabbitmq for asynchronous presence updating
Note: When deploying, restarting the process-user-activity-commandline script is needed

(imported from commit 63ee795c9c7a7db4a40170cff5636dc1dd0b46a8)
2013-02-11 18:05:57 -05:00
Zev Benjamin c80a8a50f4 puppet: fix app_frontend pip command
(imported from commit 58e2e57892d6af68c386f4838d0de26c7d2e861e)
2013-02-11 13:19:42 -05:00
Zev Benjamin 98add6efa2 puppet: Store the machine's type in /etc and add script to reapply the appropriate puppet config
Fixes #815

(imported from commit 0c90b74b9b11e4abcc875fbd7a2f0c48356678b2)
2013-02-11 13:19:41 -05:00
Zev Benjamin 801d2c40e2 puppet: Restart iptables-persistent from puppet
(imported from commit d46271046b4a341ab5a5f636908583323e107c69)
2013-02-11 12:55:01 -05:00
Leo Franchi b1e04f8feb Fix path to send-receive script post api relayout
(imported from commit 69cf0334b7bd363a862f468ea9d5e73b777bdd73)
2013-02-08 16:56:52 -05:00
Zev Benjamin 8132cf2ea2 puppet: Have install-server use its 'type' argument to install the correct puppet manifest
This also removes the "run puppet twice" hack, which I believe is no
longer needed for Apache.

(imported from commit 20016c3e8f0d267e04aeef585f19f5910bf01ddc)
2013-02-08 16:06:34 -05:00
Zev Benjamin 0e66607910 puppet: Fix wiki dependencies
(imported from commit a16675d203363b4cd535c2714ad77fd4d0c239d3)
2013-02-08 16:06:34 -05:00
Zev Benjamin 61466c0637 puppet: Fix trac copy and paste error
(imported from commit e201ea56a3111423f65e7a386685a4d90bc1080b)
2013-02-08 16:06:34 -05:00
Zev Benjamin b91d510dac puppet: Fix apache dependencies
(imported from commit 340dde4d045da17dcc7dab492a6fc12e0df3539c)
2013-02-08 16:06:34 -05:00
Zev Benjamin a4f6a3e83f puppet: Fix iptables dependency
(imported from commit 8c3275f4b875b318931a6f6917e6293b111b247f)
2013-02-08 16:06:34 -05:00
Zev Benjamin 153777bd01 puppet: Fix pip dependencies
(imported from commit 34d2ef0fbf39734c66d7ed777c225045e1b8619d)
2013-02-08 16:06:34 -05:00
Zev Benjamin c76ae76d51 puppet: Ensure a particular 30-postgresql-shm.conf file instead of appending lines to its contents
(imported from commit 884e1f2663763d7286146583a097f4548253e347)
2013-02-08 16:06:34 -05:00
Zev Benjamin fb5e5519d9 puppet: Ensure a particular sshd_config instead of appending a line to its contents
(imported from commit 4e745e23afe0cf8e6dd117cdeb6d6ec3a14ef24b)
2013-02-08 16:06:34 -05:00
Zev Benjamin da95bb2988 puppet: Move all puppetized config files to the humbug module and reference them with puppet URLs
(imported from commit f0f325bbad381b87c12c6f7888f4dd5d6989f09f)
2013-02-08 16:06:34 -05:00
Zev Benjamin beb2ecf5c9 puppet: Ensure that sshd is restarted after turning off password auth
(imported from commit 46fef98df9ea1d9ee4038f400cc7c8689d80a0ec)
2013-02-08 16:05:51 -05:00
Zev Benjamin 10f3853abd puppet: Make common::line use absolute paths
(imported from commit c9a3b184433709361243d5c3af13d290cc710ecf)
2013-02-08 16:05:51 -05:00
Zev Benjamin 3e7a6619bd puppet: Use exec's 'creates' parameter instead of onlyif
(imported from commit 94b5e76ca5119443f143e4af5c86e3c16c99dc1e)
2013-02-08 16:05:51 -05:00
Zev Benjamin a7ca48e2ff puppet: Use common::line instead of our own common::append
(imported from commit 230efb3409eaa451fa28d1655b27a2f5e5f9d382)
2013-02-08 16:05:51 -05:00
Zev Benjamin eba1008b1c puppet: Use absolute paths in execs
(imported from commit 426c572a4653ad44aa315b43d49c0d6ce001a58d)
2013-02-08 16:05:50 -05:00
Zev Benjamin 5c6a3f3e66 puppet: move all our puppet classes into a 'humbug' module
(imported from commit 69b42598c003bbe85dfa4266c56dd019304ea7fb)
2013-02-08 16:05:50 -05:00
Keegan McAllister bcec450c49 [manual] Remove hunt server from Postgres config
Deployment steps: TBD.  Will do this when the commit hits master.

(imported from commit 1eebbe873d520cb44c6605845afc9421448c6fe4)
2013-02-08 13:33:28 -05:00
Tim Abbott a306c28aa2 puppet: Add documentation on setting up the API distribution site.
(imported from commit ea298e8123bb5fee079cf969802fcb8201ed3111)
2013-02-07 14:28:06 -05:00
Tim Abbott 4cd3fd234c puppet: Add supervisord configuration for feedback-bot.
(imported from commit c7deece3e48d59de856393a4a6b7929757bc1c7c)
2013-02-05 14:27:56 -05:00
Tim Abbott f5b44cf349 nagios: Add monitoring for zmirror subscriptions syncing.
(imported from commit 2e4ae2c35d589f14b57758cd68a58f8b49b7ecf3)
2013-02-05 14:27:56 -05:00
Tim Abbott a7281f7e5a Add notes to puppet config about manually deployed symlinks.
(imported from commit 219f3b407bd83e0728f049820ad06092d6eed12a)
2013-02-01 16:04:11 -05:00
Tim Abbott 3c6dc21b05 Add pagerduty_nagios.cfg to git.
(imported from commit 2f7110d5ab65893afcb83e6f38944bf065abedff)
2013-02-01 14:50:28 -05:00
Tim Abbott 26aece90b8 nagios: Enable the Nagios commands feature.
This allows us to in particular reschedule a Nagios check to run
immediately, which I've in the past found super useful when trying to
figure out whether we actually fixed a problem.

Unfortunately, Nagios config sucks and there's no easy way to create a
group containing all of us as people able to issue commands; you have
to list them in like 8 different places.

(imported from commit 2c1e53330eff1e47e09d0b1917136f101d64e86a)
2013-02-01 14:50:28 -05:00
Tim Abbott 1fe6045288 nagios: add check that process_user_activity is running.
This fixes trac #670, and also adds the "-u humbug" parameter on the
other check_procs run, since that is a good practice move to help
avoid the check counting its parent process as one of the matches.

(imported from commit 43ae9b4863ba67579a21c86a910b73019f85a538)
2013-02-01 14:50:28 -05:00
Tim Abbott 2dd2bc8759 nagios: Make default contact_groups not page.
This will help us avoid making things accidentally pageable.

Also, explicitly set contact_groups for all our services, to help
encourage making explicit decisions about which new items are
pageable.

(imported from commit 740c6550d4a7091e58681435eeb7aaabf98df75c)
2013-02-01 14:50:28 -05:00
Keegan McAllister 6990260b59 [manual] Minify JavaScript and CSS in production
Manual deployment steps: The same Nginx reload as for "Get rid of the
static-access-control mechanism".  If deploying both commits at once,
just do it once.

(imported from commit dd8dbbf14b95fce0a4b6f66f462fa0a6b50bfb8c)
2013-01-31 15:41:01 -05:00
Keegan McAllister ee6f668c4d puppet: Install django-pipeline on app servers
This is in Debian unstable but unfortunately not older versions.

(imported from commit b82654edef270ef06fcf5015dd5d20db1c5c92f5)
2013-01-31 15:34:13 -05:00
Keegan McAllister f57126d169 puppet: Install yui-compressor on app servers
We have lots of choices for JS / CSS minifier.  This one works fine
and is in Debian.

(imported from commit bb5a05a8e59d0821e746116af0ef7e3c8ad59aaa)
2013-01-31 15:34:12 -05:00
Keegan McAllister 5e9b0ba79d [manual] Get rid of the static-access-control mechanism
We will minify our code, rather than trying to restrict who can see the
un-minified code.  Removing access control first simplifies things.

Manual deployment steps:

    scp servers/puppet/files/nginx/humbug-include/app root@staging.humbughq.com:/etc/nginx/humbug-include/
    ssh root@staging.humbughq.com service nginx reload

and then the same for app.humbughq.com once deployed to prod.

(imported from commit 63788aa3fa7ba5fd97fcf85b05760abb5e7cae4b)
2013-01-31 15:34:12 -05:00
Leo Franchi 6e9b8d895c Add munin plugin for send-receive timing
(imported from commit e2ae0775379ce59ab43213e68ade4d3f88b578e6)
2013-01-31 13:02:57 -05:00
Jessica McKellar 14d0ec1096 nagios: add several postgres checks.
(imported from commit 5440b2b14d5db11fa9794fe4bcb86a1d6fe90b5d)
2013-01-30 10:55:35 -05:00
Jessica McKellar a5337033b7 nagios: add a send-receive delay check.
(imported from commit ed58f49440fc1e8175ea02eb5d1b0ae8b53472f0)
2013-01-30 10:55:35 -05:00
Zev Benjamin 726ba8dad9 Make Postgres have a log prefix more like what pgFouine requires
We'll still need a conversion script, but it should be easy.

pgFouine requires a log prefix of '%t [%p]: [%l-1] '.  We instead use
'%m [%c]: [%l-1] ' which contains strictly more data.  Specifically,
"%m" is "%t" (time) but with milliseconds and "%c" is "%p" (pid) but
with the process start time.

(imported from commit a0bb583b563bdea0ca19b8b21677df0b9a18092a)
2013-01-28 16:21:42 -05:00
Jessica McKellar 767bf16c1c Hack up paths to be able to import both the API and Django model.
(imported from commit ca89d6bf6208455db4b636198737698ffe575698)
2013-01-24 13:36:11 -05:00
Luke O'Malley 61843b8645 nagios: Add plugin to watch the latency for a message roundtrip.
(imported from commit 75888fa4f7ceedb4a95e9b6c4012c32e106ee1ad)
2013-01-24 13:36:11 -05:00
Tim Abbott 2be39640d3 Add postgres config for new frontend.
(imported from commit 0b67ec1cb2c4b06d85d875c14154dd3e453f05c2)
2013-01-17 22:08:39 -05:00
Tim Abbott 54748643d5 puppet: Install South on our servers.
(imported from commit 18b25c6329508a235229992da48a9888b2cc244c)
2013-01-17 22:08:39 -05:00
Keegan McAllister c9a555b605 Nginx: Drop caching directive for /static
This might fix problems where users were running old code even after reloading.

(imported from commit dedc4d513f884aa2bafa0c7cc7a817d6715b48a0)
2013-01-16 15:03:40 -05:00
Luke Faraone d0a5d7f7e2 Serve static content in /dist on app
(imported from commit b5850ee1f6c6663a27fee14f430f1fae7b690725)
2013-01-15 19:10:09 -05:00
Tim Abbott e592e71515 [manual] Use rabbitmq queue to process UserActivity.
Before this is deployed, we need to install rabbitmq and pika on the
target server (see the puppet part of this commit for how).

When this is deployed, we need to start the new user activity bot:

./manage.py process_user_activity

in the screen session on the relevant server, or user_activity logs
won't be processed (which will eventually result in all users getting
notifications about how their mirrors are out of date).

(imported from commit 44d605aca0290bef2c94fb99267e15e26b21673b)
2013-01-14 13:28:23 -05:00
Keegan McAllister 6d7ef69cda nginx: Add config for plant.humbughq.com
(imported from commit e90b8e350014b49de53bfd5640442060672e691d)
2013-01-11 17:41:11 -05:00
Keegan McAllister 56660f30f8 nginx: Factor out shared parts of app / staging config
(imported from commit e00d5eec1bc58754db6e97935bc803fe3a4fe291)
2013-01-11 17:39:51 -05:00
Keegan McAllister ef6a5220c8 nginx: Remove unused config humbug-dev
(imported from commit 178a320bf56076c61f4010bf6cb89ba04798b4a4)
2013-01-11 17:39:48 -05:00
Jessica McKellar 9730a65f59 nagios: revamp check_user_zephyr_mirror_liveness to monitor sudden drops in mirror use.
(imported from commit e92df66c40065584e84c049cfab8d82f71d6dddd)
2013-01-08 10:53:33 -05:00
Jessica McKellar 0655397536 Give the NTP check the default number of retries.
It had a max_check_attempts of 1, which makes it susceptible to
network blips.

(imported from commit 20e51878d75bef36d02c5afaab78b8cdd701077f)
2013-01-08 10:53:33 -05:00