Commit Graph

140 Commits

Author SHA1 Message Date
Luke Faraone 60cfd3cfb0 Accept SMTP connections on hosts.
(imported from commit 524ae3f4362ffea12ff96498ae554322f7fe8a3c)
2014-02-06 12:14:21 -05:00
Luke Faraone 374acb7f24 [puppet] Move email mirror cron to public module
This way we can reference it in the documentation.

(imported from commit 37d5cbfcfb745e2b44768674f53d7ba450518cd0)
2014-02-04 10:38:17 -05:00
Luke Faraone 3263d09939 Convert zmirror to use puppet apt module for debathena sources
(imported from commit 080d59d2ac750d03b55460752d7fe7d02e72611c)
2014-01-31 13:43:04 -05:00
Tim Abbott 532cd061fb [puppet] Raise maximum items per page for trac.
(imported from commit 2ffa5e04c220a87d51cba42ade89874cc43ba584)
2014-01-29 17:22:19 -05:00
Tim Abbott 5108253e97 nagios: Make Zephyr mirroring alerts not pageable.
(imported from commit ab98af762b1edf93703fc865496aedc59ce7bd2d)
2014-01-24 13:53:48 -05:00
Zev Benjamin 759d33fad1 puppet: Check all disks via nagios, not just /
(imported from commit 0bc9fc150e791ce3ccec99688f3593a8678a87c9)
2014-01-23 13:37:27 -05:00
Tim Abbott 57c7634a4e Increase Zulip worker memory limits.
(imported from commit 6969eb1d2db0ee47c7b115b7f9b55ded2c9265dd)
2014-01-22 17:19:19 -05:00
Zev Benjamin c4e1d9f02a puppet: check_postgres_backup: Connect to the 'postgres' database
This allows the utility to run on trac.zulip.net, which doesn't have a 'zulip'
database.

(imported from commit c8eabb89e5e161191d6f2c92ca2b1428b17a9aa0)
2014-01-22 12:07:57 -05:00
Zev Benjamin 49f2657c8d nagios: Add check_postgres checks for the trac and wiki databases
We don't do the sequence check because that requires read access to the database
itself, which the zulip user doesn't have.

(imported from commit fba7604826353b2974e9757f01dcb426297993b3)
2014-01-22 12:07:56 -05:00
Zev Benjamin 3840cf760f nagios: Move a few services from hostgroup postgres -> hostgroup postgres_appdb
(imported from commit 54a738f19f176d36526d40968c379f6357d56e6b)
2014-01-22 12:07:56 -05:00
Zev Benjamin 1ae040c7fb nagios: Specify the db and user for check_postgres via arguments
(imported from commit c3b1a7fe7c63094ed8956ed1bdf4861d747637bd)
2014-01-22 12:07:56 -05:00
Zev Benjamin a974301b8b nagios: Add trac to the postgres_other hostgroup
(imported from commit 7e531b982b8f8961f2201cdc8b88d90d5d238907)
2014-01-22 12:07:56 -05:00
Zev Benjamin 41e274a8e4 nagios: Split postgres hostgroup into more fine-grained groups
(imported from commit ab5fcc0893fb8635defecdf3045a3ffdd5e26f14)
2014-01-22 12:07:56 -05:00
Leo Franchi e734155a1c Mount and make graphite backup drive when creating stats1
(imported from commit f8af032fa314812610d0ec7eb6227ebb0b3c2f32)
2014-01-22 10:49:49 -05:00
Luke Faraone 92ae790130 [manual] Switch listen address to www.humbughq.com for humbughq.com domains
We cannot use SNI for these legacy domains because old plugins still
connect to them.

This commit (along with the three previous commits) requires a lb0 nginx
deployment to function.

(imported from commit f47f3d7b597666508b3817d965fe8ce19d50c2c0)
2014-01-21 11:15:08 -05:00
Luke Faraone e852580a0e Use correct key for humbughq SAN cert.
This is live right now.

(imported from commit 051a44e2962557f3fc293e3e2f2e169a5d6e658c)
2014-01-21 11:15:07 -05:00
Luke Faraone c9158dd3d9 [manual] Use SNI cert instead of wildcard for humbughq
To deploy, the certs need to manually be copied to lb0's /etc/ssl/certs
directory, the nginx config updated, and the server restarted

(imported from commit c70c7678cd010a1b2b0aba830ab3d862005bd627)
2014-01-17 15:03:29 -05:00
Tim Abbott 7ce692b3c3 Restore serving the app on humbughq.
Partially reverts b1a8de8763

(imported from commit ddd9443d527f1e46f78008178b2410374551b8a6)
2014-01-17 15:03:29 -05:00
Luke Faraone 846be23ce2 Load SNI-enabled www.hhq.c cert
This replaces the old www.humbughq.com cert.

Contains these hostnames:
 * www.humbughq.com
 * api.humbughq.com
 * humbughq.com

Generated per 9d674d6a0.

(imported from commit 0ef3f0ff2a02996246868466b5e634ebf45439a2)
2014-01-17 15:03:16 -05:00
Luke Faraone ce50478a1e Move humbughq.com hosts to www.zulip.com IP
These are redirect hosts, so they don't need their own IP.  Supporting
non-SNI clients isn't a priority for us.

(imported from commit b1a8de8763ab944885518c868e4e30307d84c11d)
2014-01-16 15:56:16 -05:00
Luke Faraone 2c86c5c8ee Redirect humbughq domains to www.zulip.com per Waseem.
(imported from commit d5b8e8f33787d2a590516219ca4043b304b80a21)
2014-01-16 15:54:53 -05:00
Luke Faraone b6a2208d84 nginx configuration for customer29 on lb0
(imported from commit 7b6712e3e68aca71e81a6224af7d3f876af6ab1e)
2014-01-16 15:54:53 -05:00
Luke Faraone 8ebf0a414c Remove expired and unused SSL certificates
(imported from commit 7b058878183edc6cca593df6cd4b8cfeb15bab70)
2014-01-16 15:54:53 -05:00
Zev Benjamin 20e4e31dcf puppet: Update env-wal-e to take the S3 bucket to use from /etc/zulip/zulip.conf
This will let us do normal puppet applies on our postgres hosts again.

Crudini is already installed and /etc/zulip/zulip.conf has already been edited
on the relevant hosts.

(imported from commit 8e2b88d2fe2f7b2367ecb73a50a299200fe381a0)
2014-01-16 15:23:21 -05:00
Zev Benjamin c045644097 puppet: Run check_ntp_time against an NTP pool instead of time.mit.edu
MIT implemented NTP rate-limiting to defend against on-going reflection attacks,
which was causing our nagios checks to fail intermittently.  When the attacks
die down or when external sites fix their NTP configurations, checking against
time.mit.edu will stop failing.  However, there also isn't much of a reason to
stick with checking against a single server.

(imported from commit 2c2a1a04646b880b010cbb4b6d94016b1eccd1a0)
2014-01-06 17:30:09 -05:00
Tim Abbott bdcc2e5c52 nagios: Set max_check_attempts to 3 for batched queue processors.
(imported from commit ec0ac86726cd6ff3d0fdfcfcb161d3329fca02ac)
2013-12-19 17:31:41 -05:00
Leo Franchi 9c82e869c2 [manual] Release OS X desktop app 0.4.2
This reqires a puppet apply as well as a manual move of the installed
files and symlink switch. Leo will do it when it hits master.

(imported from commit e58e52087ad38f1cb8e0e606b82266a93cf91e53)
2013-12-18 16:14:51 -05:00
Jessica McKellar 5e217a1079 Use correct time zone in digest email cron job.
(imported from commit fd470af4b44ffb9696ff3a97372aaf2524a4806b)
2013-12-18 14:31:03 -05:00
Tim Abbott ae6c17a87d puppet: Stop using /var/log/nginx/zulip.*.log.
It's confusing to have our log data on different files on different
systems (e.g. loadbalancer vs. app).

(imported from commit be701072ee05e2659f146b226a39f33cb4707180)
2013-12-17 16:22:08 -05:00
Tim Abbott 8dcf7d4cc3 [puppet] Add log2zulip tool for sending log files to Zulip.
This tool is a little crude; it runs out of a cron job and will
forward to staging a notice about any new lines in the declared log
files, truncating if there are more than 10 lines.

(imported from commit 6748ddff1def0907b061dc278a3a848bd2e933f1)
2013-12-17 11:02:55 -05:00
Jessica McKellar 8bb1caec8f [manual] digest: add the cron job that will trigger digest emails.
Manual deployment instructions:

On staging, do a puppet apply.

No action needs to be taken for the prod deploy.

(imported from commit 0f6e5ab22aaeacfcc69d57de12f2bb6fac6f0635)
2013-12-17 10:47:16 -05:00
Tim Abbott c872866289 puppet: Fix nginx upstreams for staging.
(imported from commit eb1e6e3b2d35533af4a24015a91201e2414f8e28)
2013-12-16 11:32:05 -05:00
Tim Abbott f8fe9d1dd4 Fix check_worker_memory process list computation.
(imported from commit 9ac58b894ecfd84da6ac8509c0dc2ceb60eedfce)
2013-12-16 10:09:59 -05:00
Luke Faraone 1370c014a5 Clean up logging and documentation in ec2 interfaces script
(imported from commit e55247931cdeb61563f2348ca09f3d7b9fc85f0c)
2013-12-13 18:07:08 -05:00
Luke Faraone 104c2a06ae Set iptables rules for each IP, not just each interface
(imported from commit c24d2123489dc384bf50e379d245807af3488ebf)
2013-12-13 18:07:08 -05:00
Kevin Mehall f929e51776 puppet: Make Camo Nagios check waste less bandwidth
Use http://www.google.com/favicon.ico instead of a 1.7MB animated gif from
imgur.

(imported from commit 94993af35bf87b0f22e6e743a9ba1cc1c5c9a78f)
2013-12-13 17:27:01 -05:00
Tim Abbott 950e4c800b puppet: Declare upstreams properly in app nginx config.
(imported from commit 859eeed0d5b92c1b5b2b0764aba06aebcde8e2e2)
2013-12-12 16:48:52 -05:00
Tim Abbott ae4d214c49 Fix longpolling treatment for api.zulip.com/v1/events.
(imported from commit 78029972938ad7c9aa862330e38965b4b032c935)
2013-12-12 16:03:45 -05:00
Tim Abbott 73f04b21e9 Add zulip.customer29.invalid host.
(imported from commit ea3e7bb465c920b8ec21b7471cd261868f5059e7)
2013-12-12 16:03:45 -05:00
Tim Abbott c21e85e569 Remove staging.humbughq.com loadbalancer config.
The DNS has been disabled for some time.

(imported from commit e054c0fb0b37077d8303eab4d4ffec6ff53e8990)
2013-12-12 16:03:45 -05:00
Luke Faraone 1b5c1ac021 Update style of client strings.
(imported from commit 1516461cf53b2715de68e01f16bb8a8cc33c48ad)
2013-12-09 11:47:52 -05:00
Leo Franchi e39cc5324b [puppet] Aggregate narrow timing stats
(imported from commit 4eff25635a3cb7687e995ad1127cff68da51329a)
2013-12-07 10:44:54 -05:00
Leo Franchi f70878e6c5 Fix aggregation rules for endtoend time
(imported from commit 29165b09e2d8904ee502cc04610a951d87ef896f)
2013-12-07 10:44:54 -05:00
Tim Abbott abeb29c226 Fix incorrect proxy_pass location for staging longpolling.
(imported from commit a4ac2c5c3416a8d8f748237411df6235f237e893)
2013-12-07 08:02:55 -05:00
Tim Abbott 09a61e8128 nginx: Enable keepalive for communication between lbs and frontends.
(imported from commit a7c8d9dfefbb6e5d01c8050688d831787b31bbd4)
2013-12-07 07:41:45 -05:00
Tim Abbott 1843262672 puppet: Mark all Nagios plugins as executable.
They were being installed as executable anyway, but this will make
running them manually a bit easier.

(imported from commit a1181d2c90770af5aa44b0f65a47a460efdcf2d7)
2013-12-05 15:25:25 -05:00
Tim Abbott 676e9d90ff nginx: Get rid of trailing / in loadbalancer proxy_pass directives.
The trailing "/" actually means "replace the location with /", which
is either useless or actively harmful, depending on the location.

(imported from commit 58b9c4c9e55e3a162ffce49c954bc2182ec57dde)
2013-12-05 15:25:25 -05:00
Tim Abbott cc00ed6d7e nginx: Clean up now-empty 'loadbalancer' include file.
(imported from commit d13b5d91f6b85ba3e0bef7728985d0eba1cae084)
2013-12-05 15:25:25 -05:00
Tim Abbott afaff0c2cf nginx: Set X-Forwarded-For in common proxy configuration.
Previously we sometimes set it to $proxy_add_x_forwarded_for and other
times to $remote_addr, but according to

http://wiki.nginx.org/HttpProxyModule#.24proxy_add_x_forwarded_for

$proxy_add_x_forwarded_for handles this for us -- it will be
$remote_addr if there was no X-Forwarded-For header anyway.

(imported from commit 67dc52250e3e7751b1bf375d1a71d0272475435c)
2013-12-05 15:25:25 -05:00
Tim Abbott afe167ea58 nginx: Use the longpolling proxy configuration on load balancers.
(imported from commit f590e6b1eec2856b5128e310797f8ba58846417a)
2013-12-05 15:25:25 -05:00
Tim Abbott 9e24558092 nginx: Move common proxy configuration into an include file.
(imported from commit 2ee5afc74fe146f8ee98f18f846342351c61c7f0)
2013-12-05 15:25:24 -05:00
Tim Abbott 3760609f3f Enable /sockjs handling on api.zulip.com (not used yet).
(imported from commit c2581e3243b2129c980fd3dd318eb3d99f3eb593)
2013-12-05 15:25:24 -05:00
Tim Abbott 79910fa2b3 Disable proxy_next_upstream for sockjs in remaining proxy_pass lines.
(imported from commit f14c7962253b34040ed9ab077a58c8b200df5d9d)
2013-12-05 15:25:24 -05:00
Tim Abbott e5be713103 Clean up EXTERNAL_API_HOST usage and defaults.
We now have 2 variablse:
EXTERNAL_API_PATH: e.g. staging.zulip.com/api
EXTERNAL_API_URI: e.g. https://staging.zulip.com/api

The former is primarily needed for certain integrations.

(imported from commit 3878b99a4d835c5fcc2a2c6001bc7eeeaf4c9363)
2013-12-04 15:10:54 -05:00
Tim Abbott b8a151ca4e Revert "[puppet] Add cron job to restart our workers daily."
This reverts commit 0b0180b0751f6c618d877b9c9ffc2b8287254e4d.

(imported from commit a81c552100345d369ffcaf69f28a86dea0893128)
2013-12-04 10:27:45 -05:00
Tim Abbott 606d8a4f9b Add Nagios check for queue worker memory usage.
This is detect future memory leaks.

(imported from commit 75fd4c2ad41ea71e87a53fb33e2106c5773909d5)
2013-12-04 10:27:44 -05:00
Tim Abbott 850eae3e8e puppet: Disable proxy_next_upstream feature in nginx config.
(imported from commit 84cad76701f9ee40fa9601ae06b3f804948b96d4)
2013-12-03 15:20:45 -05:00
Tim Abbott 5007d4d87a [puppet] Update set_real_ip_from to use lb0's internal IP address.
This is something we forgot to do in the VPC migration, so our IPs
have all been the lb0 IP in our logs :(.

(imported from commit 9d3fc69cf72a84f7bd7c54e50fb1e776a67d971f)
2013-12-03 14:29:34 -05:00
Leo Franchi 42e23dc82e [manual] Release desktop app 0.4.1 for OS X
This requires a puppet apply on prod0, and an update of the
Zulip-latest.dmg and Humbug-latest.dmg symlinks in
/src/www/dist/apps/mac and /srv/www/dist/apps/sso/mac

(imported from commit e83170a19ac2de6458a0fd43140068fab4135483)
2013-12-02 15:24:32 -05:00
Leo Franchi d36510e4c3 [manual] Release Zulip Desktop 0.4.0
This requires a puppet apply, and also a manual update of
the Zulip-latest.* symlinks in /srv/www/dist/apps

(imported from commit 991dd6924ba33d81f486e914bcbadfec5b350660)
2013-11-26 17:41:25 -05:00
Zev Benjamin 7af4b92b98 puppet: Rename app to prod0 in nagios
(imported from commit c2d1c2c06276a816ef33e057d3f859c755490cb3)
2013-11-25 11:43:16 -05:00
Zev Benjamin 9f2af6fd0d puppet: Fix postgres_primary alias
(imported from commit 1cd199224e45700fac03e68c99f9d4f7d9212b45)
2013-11-25 11:43:16 -05:00
Zev Benjamin 847d4dfbca puppet: Specify hosts for the postgres autovac_freeze check via a hostgroup
(imported from commit d0afc1b78015740fa9638563a5672d3400dd5002)
2013-11-23 12:08:49 -05:00
Zev Benjamin 139518ccbe puppet: Remove postgres0 from nagios and munin configs
(imported from commit 6a4eb208b2a344d65d684cf904ba882a5400056d)
2013-11-23 12:06:27 -05:00
Zev Benjamin dacf97db48 puppet: Use peer authentication for Postgres nagios checks
(imported from commit d8f02d5320d6f8b97fd82cd3f0ca65f6e5c42b03)
2013-11-23 10:01:15 -05:00
Zev Benjamin 3454680e4b puppet: Add VPC subnets to pg_hba.conf
(imported from commit 633bf08bfe2f3695bd6c9ed8584b78971ebe065f)
2013-11-23 08:23:49 -05:00
Zev Benjamin bf8fb3c0df puppet: Add postgres2 to nagios monitoring
(imported from commit 799b1304eebe49cf6d8153fb2bfd0b11a3bcab00)
2013-11-23 08:10:44 -05:00
Zev Benjamin 658972dda3 [manual] puppet: Add postgres2 to munin monitoring
You must run
autossh -2 -fN -M 20018 -L 5009:localhost:4949 nagios@postgres2.zulip.net
as nagios on nagios.zulip.net after deploying this commit.

(imported from commit bd8a61f99555ccf0a0010d79dbd89017aaafbb8f)
2013-11-23 08:10:44 -05:00
Zev Benjamin c3f4ab6c94 puppet: Add replicator access from postgres2 to pg_hba.conf
(imported from commit 2a4f150c67d3136a5e97cb673cc7f14256ffae01)
2013-11-22 17:38:52 -05:00
Tim Abbott 8919ebe6b2 puppet: Make sure prod0's future external IP has access to postgres.
(imported from commit 91523dc92fd15dc0cf19b7bca70513250c4da983)
2013-11-22 16:43:10 -05:00
Zev Benjamin 18fc8c2059 puppet: Do peer authentication for user zulip on the DB servers
(imported from commit dceed53990db64b3c345726b02bf0c25815c2b25)
2013-11-22 15:58:09 -05:00
Tim Abbott c31dbba9cc [puppet] Update pg_hba.conf to include staging's public IP.
This is for the interval while staging is running in VPC and postgres
is not; we can clean up these changes once that's no longer the case.

This also updates test1's IP, which apparently someone forgot to
commit previously.

We're currently running this on prod.

(imported from commit 3feced750f643bb218d4240e9a3d5cd7116963ee)
2013-11-21 11:27:16 -05:00
Tim Abbott f0c6b63526 [puppet] Add cron job to restart our workers daily.
This is to ensure that if we have an interval where we're not doing
prod deploys, we don't have to worry about worker memory leaks killing
us.

(imported from commit 0b0180b0751f6c618d877b9c9ffc2b8287254e4d)
2013-11-20 18:34:16 -05:00
Tim Abbott 630cfd72f1 Deregister event queues when done in our Nagios scripts.
(imported from commit a1f73403163323e1dd9eda2f5269e94c60abdd1a)
2013-11-20 18:34:16 -05:00
Tim Abbott b50db26a18 puppet: Add monitoring for camo.
(imported from commit b3cf29b02de285cf860fc173183cb6f4f3a17c74)
2013-11-19 15:25:14 -05:00
Tim Abbott 1b009c47fc Clean up our nginx configuraiton to make better use of app.d.
Now app.d is something that any app frontend will read, and we just
have secondary manifests add additional files to the app.d directory
for custom stuff.

This fixes the issue that we were incorrectly including the
lb0-related app configuration in the enterprise version.

(imported from commit dec8dcdf2506b82e51186ff936c26dc1cd6cf61b)
2013-11-15 15:04:13 -05:00
Leo Franchi be3257de62 Add sso sparkle paths and files
(imported from commit f8a953f4262b170931792ed223f1ddc29c5fe5ed)
2013-11-14 10:22:35 -05:00
acrefoot 0175440afc [manual] fixup nagios postmaster configuration
(imported from commit e3c00b31bbb0ced38e62d31ae80b58e8c6374c7f)
2013-11-13 17:37:54 -05:00
acrefoot 6d38285a2e fixup supervisor oops related to postmaster config
(imported from commit 8b5c39f0d13abb5e1def9f88a2ab82cfa67b42f6)
2013-11-13 17:15:55 -05:00
acrefoot eab6a1d190 [manual] add nagios checks for email_deliver
manual step: puppet apply, make sure that these nagios checks are working properly

(imported from commit abc75b8a5b153510243c14035b820fbc864b7776)
2013-11-13 16:41:36 -05:00
Jessica McKellar 1cb339a23d Expose password-protected /enterprise/download through nginx.
(imported from commit 8fba915bf0d0c718138ac62bd4333aef4e845d55)
2013-11-13 16:20:43 -05:00
Tim Abbott bbf8424b3e puppet: Move our ops repository apt configuration to internal module.
(imported from commit 2aca8e8c2edbd87a77fd5f00b3ae250484721fb4)
2013-11-12 09:34:25 -05:00
Tim Abbott 71e1f00fc6 puppet: Move our iptables config to zulip_internal.
(imported from commit f177b3989092f4fa7f00ae5bfb833ea23fe35489)
2013-11-09 07:28:18 -05:00
Tim Abbott e957c8adb2 puppet: Move our SSHD config to zulip_internal.
(imported from commit 0c6314963ebe9246d8136dc6db3176d226dc2049)
2013-11-09 07:27:19 -05:00
Leo Franchi 312e3013af Release desktop app v. 0.3.10
(imported from commit 0114d0b631ad027d6e57a5a264a8c8efd3438fa7)
2013-11-06 14:13:57 -05:00
Leo Franchi 82ef17207a Aggregate receive and displayed times as well
(imported from commit e7789a627e4f1396d2c11c5a4b448135197324ab)
2013-11-06 13:37:15 -05:00
Leo Franchi c9c26a9ecb Remove 0.3.9 update while we prepare 0.3.10
(imported from commit f8f469dc93601d69f6974ed11efff259040d8ef2)
2013-11-06 12:58:09 -05:00
Leo Franchi 8dd4bf8f00 [puppet] Log endtoend send time on a per-realm basis as well as aggregate
(imported from commit 07226b20081d203af1f52776475228d9b6783869)
2013-11-06 11:25:00 -05:00
Tim Abbott b2ea2dca81 puppet: Add the stats1 cert (was missing from git).
(imported from commit 909c70089f95a08cf62656432e09df170b322aaa)
2013-11-05 17:06:33 -05:00
Tim Abbott b5979a3fed [manual]: Rename zulip-internal puppet module to zulip_internal.
(imported from commit 64ac7ec0f3495b1fe7810da3d4d41263c52b9b3b)
2013-11-05 17:06:32 -05:00