8edbd64bb8
Monitor the queue processors for the missedmessage_emails and slow_queries queues
...
(imported from commit 266b8f19b87a025ab35bd6dd4017bdf8a7694b49)
2013-10-04 17:58:44 -04:00
d188d829d7
Update UserActivity queries for monitoring Zephyr mirroring.
...
(imported from commit 04a9536da2891e905c6e14e0d452ca62d632641d)
2013-10-04 16:15:53 -04:00
2547e0768f
puppet: Remove rabbitmq consumer checks based on check_procs
...
These have been superceded by checks for the existance of consumers
of the relevant queues.
(imported from commit 68a0e79734366411e39e9e4346b5a61bdd34144b)
2013-10-04 14:19:16 -04:00
dc082cd96d
puppet: Add nagios notifications for the rest of our rabbitmq queues
...
(imported from commit 9d21a0ca3662396c436b482c574113d0cbc714a0)
2013-10-04 14:19:16 -04:00
61ca14b400
[manual] puppet: Consolidate check_rabbitmq_*_consumers commands
...
This temporarily breaks the rabbitmq consumer checks for
user_activity and notify_tornado on prod. This should be deployed in
such a way to minimize the time that the alert needs to be ignored.
(imported from commit 08fa2f0e7d78fca1346c62824573263e42339a45)
2013-10-04 14:19:16 -04:00
6e54ca3045
puppet: Factor out writing the rabbitmq consumer check state file into its own script
...
This temporarily breaks the rabbitmq consumer checks for the
user_activity and notify_tornado queues because their state files
were renamed to match their queue names. It will be fixed for
staging in the next commit.
(imported from commit a6aaa330a1134d8ddffe8f4959deb12b219f241a)
2013-10-04 14:19:16 -04:00
dd3281fea8
Rename some staging nginx config variables humbug => zulip.
...
(imported from commit 7937e0ee2b1ebbdf184be3ceec74afc206a56c83)
2013-10-04 11:45:40 -04:00
2614716fca
Log slow queries to zulip so we notice them
...
(imported from commit 23f311ad881edda4c4495089ea3b55213470a059)
2013-09-30 17:41:56 -04:00
a906890b4d
install-server: Run resize2fs
...
This allows us to have larger root filesystems than the AMI image.
(imported from commit 4e9698432b0c154a0bc635df07abd278c08a4905)
2013-09-30 11:09:26 -04:00
03fe84aa6a
nagios: use last Received date to determine message age.
...
If there are delays while routing the email, we don't want to get a
spurious alert.
(imported from commit 3a9e3abf0a4db2b026f797c929f1b46978f1e5e4)
2013-09-27 11:39:42 -04:00
4acddabe10
nagios: parse dates using timezone-aware functions in the email mirror check.
...
Why does email.utils.parsedate also exist? To put bugs in peoples' software.
(imported from commit a3dca741e5274027ef177388b49061b9b3c5d29e)
2013-09-27 11:39:42 -04:00
4be5d81af1
[manual] Write logs to /var/log/zulip rather than /var/log/humbug.
...
This requires a puppet apply to update the supervisor configuration.
(imported from commit f2836b6d9c53791af6f6ceb1650d0e0740df70ab)
2013-09-25 16:52:41 -04:00
0a4a53211c
[manual] Rename /var/log/humbug to /var/log/zulip.
...
This requires a "puppet apply" to be done to create /var/log/zulip
before we deploy anything using the new directory.
(imported from commit 2d7baedbf923df9f01b152cf0bda6494f0eac936)
2013-09-25 16:52:39 -04:00
0b9e54416d
[manual] puppet: Rename humbug=>zulip in nginx configuration filenames.
...
We need to manually remove the old humbug and humbug-staging sites-*
files when we deploy this via puppet.
(imported from commit d25e0172a14032c5acf1501668602d34b1b13b85)
2013-09-25 15:40:21 -04:00
32a333be42
puppet: Rename humbug=>zulip in nginx configuration.
...
(imported from commit c5880c7fe9f7d709d39e788cc2b6be940c279972)
2013-09-25 15:40:21 -04:00
4505df407b
puppet: Access apt repositories via apt.zulip.com hostname.
...
(imported from commit 2d4ecd7479ac9565d52bd85c324003e03495ebef)
2013-09-25 15:40:21 -04:00
351dd6dd5d
nagios: Update monitoring to use only the .zulip.net hostnames.
...
(imported from commit fc1a91546d39f4915ee9b9b63dad284b2cd54443)
2013-09-25 15:40:21 -04:00
17a4c39234
Rename humbug=>zulip in check_send_receive_time.
...
(imported from commit b734056dfb9d69d5dfc06a67e47c3579bb557f51)
2013-09-25 15:40:21 -04:00
dabb565179
puppet: Delete obsolete my.cnf file.
...
(imported from commit 7a61a367fe352f275f95cdceebd69fa46a7691ff)
2013-09-25 15:40:21 -04:00
bafe43f363
Release os x 0.3.8 via sparkle
...
(imported from commit 4d8b7ff23c48952f28a968c977c87951258c3333)
2013-09-25 11:18:41 -04:00
0d0655ebdf
puppet: Fix our supervisor worker configuration
...
* I forgot to add a worker for user_activity_interval
* user_activity_interval and user_presence weren't in the humbug-workers group
(imported from commit 61c29e54a0dff621a913b3e8491db41f4d0a3909)
2013-09-24 20:44:42 -04:00
90ee96d456
Split out presence and activity interval events from user_activity queue
...
(imported from commit 3c22448491867a9e72659629d5d0faaf12b85b2c)
2013-09-24 20:44:15 -04:00
e450fc80c9
Move existing queue processing workers to new framework
...
(imported from commit b448eeefcddb1840344bd337046be30714a7fdbc)
2013-09-24 20:44:15 -04:00
963b62873d
puppet: Show all nagios results on the same page
...
(imported from commit aff603faa93c5d24f372eb6afb541d870aaecf5b)
2013-09-24 16:19:52 -04:00
d3487aef15
Bump windows & mac apps to 0.3.7
...
(imported from commit 507332336538fae8321a200196b2778861967f6d)
2013-09-19 15:32:41 -04:00
4300d7441a
Add the new html2text requirement for the email mirror to Puppet.
...
(imported from commit 8d5e9eda4e554a3b48d4fa352e37125f31cafaac)
2013-09-18 15:41:27 -04:00
c81c418478
Aggregate stats_counts.app.cache.* as well as stats
...
(imported from commit 675d7b10f982bd180bf049d6552d3d0b75da8b80)
2013-09-17 14:23:46 -04:00
e13c94b904
Add aggregation for reading stats
...
(imported from commit c344b9c2897bf95287c0f0df281b0567261c87f7)
2013-09-10 10:24:15 -04:00
7aedfbba14
Include all configuration bits for stats in puppet
...
(imported from commit 439c062dafd1167f465b6a1a0d8e6767dc6babeb)
2013-09-10 10:24:15 -04:00
e5a39e5b4f
Fix apache2 nagios confing redirect
...
(imported from commit 88775e874e2852e0c8b7846bdde04d7d1d2a1afe)
2013-09-09 10:43:04 -04:00
f666e168e1
nagios: Fix argument expansion in check_postgres_alert_args command
...
This was apparently causing some quoting weirdness.
(imported from commit 680d3cb9f4b1cf47cc1652feeb9f19855e7aa3e0)
2013-09-05 20:02:02 -04:00
abe07b34d5
Up the thresholds for nagios alerts about Postgres locks
...
We can hopefully lower this again when we figure turn on autocommit
or figure out why we're suddenly taking more locks. For now, though,
the number of locks does not seem to be harmful to performance.
(imported from commit 0f8915d4a44aa28d1fa0969b93beea691d70d8d7)
2013-09-05 19:32:33 -04:00
11a503667d
Replace configuration references to trac.humbughq.com with trac.zulip.net
...
(imported from commit aa042a10ada3f04205fd8d3e94c2b13f12a5a1a7)
2013-09-05 15:43:08 -04:00
d95da86299
Apache etc configuration changes to stats to support stats1.zulip.net
...
(imported from commit 3a20e31b6a5bf897aa700b55eba1bb0e8346c36c)
2013-09-05 15:43:07 -04:00
1c48aa7b8c
Switch nagios to .zulip.net and add cert
...
We maintain a legacy hhq.c cert to avoid breaking links.
(imported from commit 4e8878185b02c0ed6c9193e21ff94ac96a2ef7b4)
2013-09-05 15:43:07 -04:00
4f6159049d
Switch trac to .zulip.net and add cert
...
We maintain a legacy hhq.c cert to avoid breaking links.
(imported from commit e9953a3cc93f057abd79d7ce652e0942051ff5dd)
2013-09-05 15:43:07 -04:00
a575ea997f
puppet: Run GitHub Camo on lb0 for external-content.zulipcdn.net.
...
(imported from commit c03f2018fee1d88b747f45fc7841949f61e5c06c)
2013-09-05 14:35:29 -04:00
ddc9c53e1b
nagios: Set max_check_attempts to 3 for rabbitmq consumers.
...
This works around the fact that we seem to have a mysterous extra
checkup 40s after an error first occurs with these checks, which
always fails because the data is updated by a cron job that runs every
minute.
(imported from commit e7fe9c85e8399115443269287e695b140b4443ff)
2013-09-05 14:35:29 -04:00
360cc754cc
Extract last_check_ts var in nagios script
...
(imported from commit b391ef2832100a8140789b0d1b553b028e0b7af6)
2013-09-03 22:30:00 -04:00
221f6e6199
puppet: Add cron job to clear expired Django sessions.
...
(imported from commit 1426430785f483d450f3d044147fe0fad50cb898)
2013-09-03 17:10:40 -04:00
4951ad2ce7
check_personal_zephyr_mirrors: Use newlines in display format.
...
(imported from commit d4114cde3575fafd0b37722681baf2e61517a6ad)
2013-09-03 11:40:42 -04:00
8176429328
Un-break PagerDuty and go back to using the e-mail integration.
...
(imported from commit 1b33336ad56d56e6d9e48a5b2497a5115fbd310f)
2013-08-30 11:32:12 -04:00
c125db4917
email-mirror: put Nagios configuration in puppet.
...
(imported from commit a6ecac877628cf7918e55d553cf509aa5c0635cb)
2013-08-28 16:43:40 -04:00
7737434dbe
email-mirror: run the management command as the humbug user.
...
We don't set up certs for root to be able to make database queries.
(imported from commit 0236745fb1d5aace3097acbe0ed4b5772a19b255)
2013-08-28 16:43:39 -04:00
ade55dc39c
Fix PATH for Postgres backup cron job
...
(imported from commit 9e4b868d444d73f9eab1507851b1b3ab47655977)
2013-08-28 15:16:25 -04:00
cfb623ae5c
Bump os x app to 0.3.6
...
(imported from commit 51ddf87797a94b97dec8ce998a08a83ed8837196)
2013-08-28 14:20:20 -04:00
1d13c091ef
pg_backup_and_purge.py: Set up logging
...
(imported from commit 0948f548b005505a7f0c38c3af245815ec9efd6c)
2013-08-28 13:30:51 -04:00
9d895e53d5
Add SHELL, PATH, and ROOT variables to crontab
...
(imported from commit 3e556b12bb1c5332bd08ba267d24e0dd524bff7d)
2013-08-27 17:28:07 -04:00
071415ae93
Update desktop apps to 0.3.5 in sparkle
...
(imported from commit cf7c1b9dbbc0cbda5c4776d7b4f523f9f4f885a8)
2013-08-27 15:42:06 -04:00
611955e9fa
zephyr_mirror: Use the new reasonable --log-path option.
...
(imported from commit 2208e4c47cf49e3005cc3b93314b8c2a0ffc23c6)
2013-08-27 14:21:26 -04:00
Zev Benjamin
Tim Abbott
Leo Franchi
Jessica McKellar
Leo Franchi
Luke Faraone
Steve Howell