20f3b3af8f
Fix zulip->zulip_internal puppet path change for apns checker
...
(imported from commit 1fd43a4f4907c24fcbbda73bbaf3cf092a6cace1)
2014-01-10 21:38:59 -05:00
91c54754fb
[puppet] Add the apns-token crontab file to puppet
...
(imported from commit f12001453c9ca924c801a6000927e3ee2696a392)
2014-01-10 21:38:57 -05:00
c045644097
puppet: Run check_ntp_time against an NTP pool instead of time.mit.edu
...
MIT implemented NTP rate-limiting to defend against on-going reflection attacks,
which was causing our nagios checks to fail intermittently. When the attacks
die down or when external sites fix their NTP configurations, checking against
time.mit.edu will stop failing. However, there also isn't much of a reason to
stick with checking against a single server.
(imported from commit 2c2a1a04646b880b010cbb4b6d94016b1eccd1a0)
2014-01-06 17:30:09 -05:00
61d660f9f3
[manual] digest: move cron job from staging to all app frontends.
...
Manual instructions:
This commit requires a puppet apply after deployment on both staging
and prod.
(imported from commit 2d10e33c6db2f5e9cc1204cdd5f2c91833da2a8e)
2013-12-20 12:50:23 -05:00
bdcc2e5c52
nagios: Set max_check_attempts to 3 for batched queue processors.
...
(imported from commit ec0ac86726cd6ff3d0fdfcfcb161d3329fca02ac)
2013-12-19 17:31:41 -05:00
9c82e869c2
[manual] Release OS X desktop app 0.4.2
...
This reqires a puppet apply as well as a manual move of the installed
files and symlink switch. Leo will do it when it hits master.
(imported from commit e58e52087ad38f1cb8e0e606b82266a93cf91e53)
2013-12-18 16:14:51 -05:00
5e217a1079
Use correct time zone in digest email cron job.
...
(imported from commit fd470af4b44ffb9696ff3a97372aaf2524a4806b)
2013-12-18 14:31:03 -05:00
d62ca820db
puppet: Fix permissions on /etc/cron.d/log2zulip.
...
(imported from commit 33ee5ae97b09b3925849940262ecd0bcbce38a3f)
2013-12-17 16:22:14 -05:00
ae6c17a87d
puppet: Stop using /var/log/nginx/zulip.*.log.
...
It's confusing to have our log data on different files on different
systems (e.g. loadbalancer vs. app).
(imported from commit be701072ee05e2659f146b226a39f33cb4707180)
2013-12-17 16:22:08 -05:00
6ccf19bed6
Run log2zulip on load balancers too.
...
(imported from commit 74c8be20d2d03aa524f05b7681febe9a9be9cdff)
2013-12-17 13:46:00 -05:00
8dcf7d4cc3
[puppet] Add log2zulip tool for sending log files to Zulip.
...
This tool is a little crude; it runs out of a cron job and will
forward to staging a notice about any new lines in the declared log
files, truncating if there are more than 10 lines.
(imported from commit 6748ddff1def0907b061dc278a3a848bd2e933f1)
2013-12-17 11:02:55 -05:00
8bb1caec8f
[manual] digest: add the cron job that will trigger digest emails.
...
Manual deployment instructions:
On staging, do a puppet apply.
No action needs to be taken for the prod deploy.
(imported from commit 0f6e5ab22aaeacfcc69d57de12f2bb6fac6f0635)
2013-12-17 10:47:16 -05:00
b6acbe040c
Fix missing nginx service notifications on configuration changes.
...
(imported from commit 0bfce276bab3704e508f6c8a58c9434e9fc224cd)
2013-12-16 13:44:50 -05:00
c872866289
puppet: Fix nginx upstreams for staging.
...
(imported from commit eb1e6e3b2d35533af4a24015a91201e2414f8e28)
2013-12-16 11:32:05 -05:00
f8fe9d1dd4
Fix check_worker_memory process list computation.
...
(imported from commit 9ac58b894ecfd84da6ac8509c0dc2ceb60eedfce)
2013-12-16 10:09:59 -05:00
1370c014a5
Clean up logging and documentation in ec2 interfaces script
...
(imported from commit e55247931cdeb61563f2348ca09f3d7b9fc85f0c)
2013-12-13 18:07:08 -05:00
104c2a06ae
Set iptables rules for each IP, not just each interface
...
(imported from commit c24d2123489dc384bf50e379d245807af3488ebf)
2013-12-13 18:07:08 -05:00
f929e51776
puppet: Make Camo Nagios check waste less bandwidth
...
Use http://www.google.com/favicon.ico instead of a 1.7MB animated gif from
imgur.
(imported from commit 94993af35bf87b0f22e6e743a9ba1cc1c5c9a78f)
2013-12-13 17:27:01 -05:00
950e4c800b
puppet: Declare upstreams properly in app nginx config.
...
(imported from commit 859eeed0d5b92c1b5b2b0764aba06aebcde8e2e2)
2013-12-12 16:48:52 -05:00
ae4d214c49
Fix longpolling treatment for api.zulip.com/v1/events.
...
(imported from commit 78029972938ad7c9aa862330e38965b4b032c935)
2013-12-12 16:03:45 -05:00
73f04b21e9
Add zulip.customer29.invalid host.
...
(imported from commit ea3e7bb465c920b8ec21b7471cd261868f5059e7)
2013-12-12 16:03:45 -05:00
c21e85e569
Remove staging.humbughq.com loadbalancer config.
...
The DNS has been disabled for some time.
(imported from commit e054c0fb0b37077d8303eab4d4ffec6ff53e8990)
2013-12-12 16:03:45 -05:00
b0a0853bd2
Specify full fingerprint rather than short key ID
...
(imported from commit fc4e9d51c440000e469f8e3882739215a3bcb022)
2013-12-11 10:54:30 -05:00
510b3349a7
Switch to downloading keys via SSL in puppet
...
(imported from commit 05d2b0626338f09370614e916050cfcee7f14829)
2013-12-11 10:54:30 -05:00
1b5c1ac021
Update style of client strings.
...
(imported from commit 1516461cf53b2715de68e01f16bb8a8cc33c48ad)
2013-12-09 11:47:52 -05:00
e39cc5324b
[puppet] Aggregate narrow timing stats
...
(imported from commit 4eff25635a3cb7687e995ad1127cff68da51329a)
2013-12-07 10:44:54 -05:00
f70878e6c5
Fix aggregation rules for endtoend time
...
(imported from commit 29165b09e2d8904ee502cc04610a951d87ef896f)
2013-12-07 10:44:54 -05:00
abeb29c226
Fix incorrect proxy_pass location for staging longpolling.
...
(imported from commit a4ac2c5c3416a8d8f748237411df6235f237e893)
2013-12-07 08:02:55 -05:00
09a61e8128
nginx: Enable keepalive for communication between lbs and frontends.
...
(imported from commit a7c8d9dfefbb6e5d01c8050688d831787b31bbd4)
2013-12-07 07:41:45 -05:00
1843262672
puppet: Mark all Nagios plugins as executable.
...
They were being installed as executable anyway, but this will make
running them manually a bit easier.
(imported from commit a1181d2c90770af5aa44b0f65a47a460efdcf2d7)
2013-12-05 15:25:25 -05:00
64807c0628
nginx: Ensure zulip-include files are distributed to the right systems.
...
There were a few recently introduced bugs, and this also cuts down on
our having to review diffs that don't actually affect the relevant
server when doing updates.
(imported from commit 43f3cff9a414bc1632f45a8222012846353e8501)
2013-12-05 15:25:25 -05:00
676e9d90ff
nginx: Get rid of trailing / in loadbalancer proxy_pass directives.
...
The trailing "/" actually means "replace the location with /", which
is either useless or actively harmful, depending on the location.
(imported from commit 58b9c4c9e55e3a162ffce49c954bc2182ec57dde)
2013-12-05 15:25:25 -05:00
cc00ed6d7e
nginx: Clean up now-empty 'loadbalancer' include file.
...
(imported from commit d13b5d91f6b85ba3e0bef7728985d0eba1cae084)
2013-12-05 15:25:25 -05:00
afaff0c2cf
nginx: Set X-Forwarded-For in common proxy configuration.
...
Previously we sometimes set it to $proxy_add_x_forwarded_for and other
times to $remote_addr, but according to
http://wiki.nginx.org/HttpProxyModule#.24proxy_add_x_forwarded_for
$proxy_add_x_forwarded_for handles this for us -- it will be
$remote_addr if there was no X-Forwarded-For header anyway.
(imported from commit 67dc52250e3e7751b1bf375d1a71d0272475435c)
2013-12-05 15:25:25 -05:00
afe167ea58
nginx: Use the longpolling proxy configuration on load balancers.
...
(imported from commit f590e6b1eec2856b5128e310797f8ba58846417a)
2013-12-05 15:25:25 -05:00
9e24558092
nginx: Move common proxy configuration into an include file.
...
(imported from commit 2ee5afc74fe146f8ee98f18f846342351c61c7f0)
2013-12-05 15:25:24 -05:00
3760609f3f
Enable /sockjs handling on api.zulip.com (not used yet).
...
(imported from commit c2581e3243b2129c980fd3dd318eb3d99f3eb593)
2013-12-05 15:25:24 -05:00
79910fa2b3
Disable proxy_next_upstream for sockjs in remaining proxy_pass lines.
...
(imported from commit f14c7962253b34040ed9ab077a58c8b200df5d9d)
2013-12-05 15:25:24 -05:00
e5be713103
Clean up EXTERNAL_API_HOST usage and defaults.
...
We now have 2 variablse:
EXTERNAL_API_PATH: e.g. staging.zulip.com/api
EXTERNAL_API_URI: e.g. https://staging.zulip.com/api
The former is primarily needed for certain integrations.
(imported from commit 3878b99a4d835c5fcc2a2c6001bc7eeeaf4c9363)
2013-12-04 15:10:54 -05:00
b8a151ca4e
Revert "[puppet] Add cron job to restart our workers daily."
...
This reverts commit 0b0180b0751f6c618d877b9c9ffc2b8287254e4d.
(imported from commit a81c552100345d369ffcaf69f28a86dea0893128)
2013-12-04 10:27:45 -05:00
606d8a4f9b
Add Nagios check for queue worker memory usage.
...
This is detect future memory leaks.
(imported from commit 75fd4c2ad41ea71e87a53fb33e2106c5773909d5)
2013-12-04 10:27:44 -05:00
850eae3e8e
puppet: Disable proxy_next_upstream feature in nginx config.
...
(imported from commit 84cad76701f9ee40fa9601ae06b3f804948b96d4)
2013-12-03 15:20:45 -05:00
5007d4d87a
[puppet] Update set_real_ip_from to use lb0's internal IP address.
...
This is something we forgot to do in the VPC migration, so our IPs
have all been the lb0 IP in our logs :(.
(imported from commit 9d3fc69cf72a84f7bd7c54e50fb1e776a67d971f)
2013-12-03 14:29:34 -05:00
42e23dc82e
[manual] Release desktop app 0.4.1 for OS X
...
This requires a puppet apply on prod0, and an update of the
Zulip-latest.dmg and Humbug-latest.dmg symlinks in
/src/www/dist/apps/mac and /srv/www/dist/apps/sso/mac
(imported from commit e83170a19ac2de6458a0fd43140068fab4135483)
2013-12-02 15:24:32 -05:00
d36510e4c3
[manual] Release Zulip Desktop 0.4.0
...
This requires a puppet apply, and also a manual update of
the Zulip-latest.* symlinks in /srv/www/dist/apps
(imported from commit 991dd6924ba33d81f486e914bcbadfec5b350660)
2013-11-26 17:41:25 -05:00
3971f18de8
loadbalancer: Fix missing location-sockjs config.
...
(imported from commit 27b168e73014d7b7c71fb00ce5b75271393fc491)
2013-11-26 12:22:17 -05:00
7af4b92b98
puppet: Rename app to prod0 in nagios
...
(imported from commit c2d1c2c06276a816ef33e057d3f859c755490cb3)
2013-11-25 11:43:16 -05:00
9f2af6fd0d
puppet: Fix postgres_primary alias
...
(imported from commit 1cd199224e45700fac03e68c99f9d4f7d9212b45)
2013-11-25 11:43:16 -05:00
847d4dfbca
puppet: Specify hosts for the postgres autovac_freeze check via a hostgroup
...
(imported from commit d0afc1b78015740fa9638563a5672d3400dd5002)
2013-11-23 12:08:49 -05:00
139518ccbe
puppet: Remove postgres0 from nagios and munin configs
...
(imported from commit 6a4eb208b2a344d65d684cf904ba882a5400056d)
2013-11-23 12:06:27 -05:00
dacf97db48
puppet: Use peer authentication for Postgres nagios checks
...
(imported from commit d8f02d5320d6f8b97fd82cd3f0ca65f6e5c42b03)
2013-11-23 10:01:15 -05:00
3454680e4b
puppet: Add VPC subnets to pg_hba.conf
...
(imported from commit 633bf08bfe2f3695bd6c9ed8584b78971ebe065f)
2013-11-23 08:23:49 -05:00
bf8fb3c0df
puppet: Add postgres2 to nagios monitoring
...
(imported from commit 799b1304eebe49cf6d8153fb2bfd0b11a3bcab00)
2013-11-23 08:10:44 -05:00
658972dda3
[manual] puppet: Add postgres2 to munin monitoring
...
You must run
autossh -2 -fN -M 20018 -L 5009:localhost:4949 nagios@postgres2.zulip.net
as nagios on nagios.zulip.net after deploying this commit.
(imported from commit bd8a61f99555ccf0a0010d79dbd89017aaafbb8f)
2013-11-23 08:10:44 -05:00
d7d98aaacc
puppet: Move /etc/iptables/rules to /etc/iptables/rules.v4
...
The /etc/init.d/iptables-persistent initfile changed to expect there to be two
files in /etc/iptables (rules.v4 and rules.v6) instead of a single rules file.
Several of our machines are currently running without iptables rules as a
result.
(imported from commit 266c2ff26b77f7c9ae793690b0d544ee4cfa5020)
2013-11-23 08:10:44 -05:00
c3f4ab6c94
puppet: Add replicator access from postgres2 to pg_hba.conf
...
(imported from commit 2a4f150c67d3136a5e97cb673cc7f14256ffae01)
2013-11-22 17:38:52 -05:00
8919ebe6b2
puppet: Make sure prod0's future external IP has access to postgres.
...
(imported from commit 91523dc92fd15dc0cf19b7bca70513250c4da983)
2013-11-22 16:43:10 -05:00
18fc8c2059
puppet: Do peer authentication for user zulip on the DB servers
...
(imported from commit dceed53990db64b3c345726b02bf0c25815c2b25)
2013-11-22 15:58:09 -05:00
c31dbba9cc
[puppet] Update pg_hba.conf to include staging's public IP.
...
This is for the interval while staging is running in VPC and postgres
is not; we can clean up these changes once that's no longer the case.
This also updates test1's IP, which apparently someone forgot to
commit previously.
We're currently running this on prod.
(imported from commit 3feced750f643bb218d4240e9a3d5cd7116963ee)
2013-11-21 11:27:16 -05:00
8bfbaab1d5
Fix typo in puppet directories for zulip_internal.
...
(imported from commit 52627a9e71dfc28bedd6c955069da46d3ef56e83)
2013-11-21 10:06:40 -05:00
c6b862d26d
Add desktop sso puppet folder dependencies
...
(imported from commit 63d8cad722cf015a5925a28ab860431be31175be)
2013-11-20 19:12:53 -05:00
f0c6b63526
[puppet] Add cron job to restart our workers daily.
...
This is to ensure that if we have an interval where we're not doing
prod deploys, we don't have to worry about worker memory leaks killing
us.
(imported from commit 0b0180b0751f6c618d877b9c9ffc2b8287254e4d)
2013-11-20 18:34:16 -05:00
630cfd72f1
Deregister event queues when done in our Nagios scripts.
...
(imported from commit a1f73403163323e1dd9eda2f5269e94c60abdd1a)
2013-11-20 18:34:16 -05:00
b50db26a18
puppet: Add monitoring for camo.
...
(imported from commit b3cf29b02de285cf860fc173183cb6f4f3a17c74)
2013-11-19 15:25:14 -05:00
1b009c47fc
Clean up our nginx configuraiton to make better use of app.d.
...
Now app.d is something that any app frontend will read, and we just
have secondary manifests add additional files to the app.d directory
for custom stuff.
This fixes the issue that we were incorrectly including the
lb0-related app configuration in the enterprise version.
(imported from commit dec8dcdf2506b82e51186ff936c26dc1cd6cf61b)
2013-11-15 15:04:13 -05:00
6826ef4e9a
puppet: Switch from nginx to nginx-full.
...
(imported from commit 38dd5966d75946842b39e4e619d82ebbb0fb041c)
2013-11-15 15:04:13 -05:00
be3257de62
Add sso sparkle paths and files
...
(imported from commit f8a953f4262b170931792ed223f1ddc29c5fe5ed)
2013-11-14 10:22:35 -05:00
0175440afc
[manual] fixup nagios postmaster configuration
...
(imported from commit e3c00b31bbb0ced38e62d31ae80b58e8c6374c7f)
2013-11-13 17:37:54 -05:00
6d38285a2e
fixup supervisor oops related to postmaster config
...
(imported from commit 8b5c39f0d13abb5e1def9f88a2ab82cfa67b42f6)
2013-11-13 17:15:55 -05:00
eab6a1d190
[manual] add nagios checks for email_deliver
...
manual step: puppet apply, make sure that these nagios checks are working properly
(imported from commit abc75b8a5b153510243c14035b820fbc864b7776)
2013-11-13 16:41:36 -05:00
1cb339a23d
Expose password-protected /enterprise/download through nginx.
...
(imported from commit 8fba915bf0d0c718138ac62bd4333aef4e845d55)
2013-11-13 16:20:43 -05:00
a16a7a028c
Fix internal postgres_common.pp.
...
We didn't remove python-argparse from the requirements when that was
removed, and we also still need python-pip to install wal-e :(.
(imported from commit b82d3b429cffe0a3993819358511e11268ee2fef)
2013-11-13 15:35:45 -05:00
47682d47c9
Move our apache2site library into the public manifest.
...
(imported from commit 8cb6d0a01fc286ad1a98a7e2d27a80293667e9b8)
2013-11-13 12:07:15 -05:00
c1bdb0c7a8
puppet: Move the removal of 30-postgresql-shm.conf into zulip::postgres_appdb
...
(imported from commit 6a4d089fd47a8ba6ea92eeac321e3077fa0d8cc4)
2013-11-13 12:02:01 -05:00
17a98b3afd
puppet: Move embedly to the internal puppet configuration.
...
(imported from commit 9b5c2f2726f2cac5bba5619ee9b7371dada0ea35)
2013-11-12 09:34:25 -05:00
5de6b879b7
puppet: Move minify-js and statsd dependencies to internal manifest.
...
(imported from commit 3363fb962a4fde575591b42db888b92bb6edd0f5)
2013-11-12 09:34:25 -05:00
bbf8424b3e
puppet: Move our ops repository apt configuration to internal module.
...
(imported from commit 2aca8e8c2edbd87a77fd5f00b3ae250484721fb4)
2013-11-12 09:34:25 -05:00
138d7053b7
puppet: Move the wal-e dependencies to the internal postgres config.
...
(imported from commit 67251263ec98e5b141f4c7587042b4db7aed36f2)
2013-11-09 07:28:19 -05:00
43e1c5e47d
puppet: Move our camo installation setup to its own manifest.
...
(imported from commit 401018c3ff49fea485c3c8b4adb42574bb0b54b0)
2013-11-09 07:28:19 -05:00
b902cc6eb1
puppet: Fix loadbalancer dependencies to include nodejs.
...
(imported from commit 9962e8ee774b13abdb95028d44cd687cee63aa13)
2013-11-09 07:28:19 -05:00
a3b813e7af
puppet: Comment the dependency lists for some service manifests.
...
(imported from commit 49817eb7daeab7cb83799d116ac829c7cd8e84e7)
2013-11-09 07:28:19 -05:00
f552149772
puppet: Organize the puppet configuration for various internal services.
...
(imported from commit ad3525f608dbc5ceb04e6829fb1da0b3baba3258)
2013-11-09 07:28:19 -05:00
709850ab9e
puppet: Organize the puppet configuration for zmirror.
...
(imported from commit 4e9d1771fd5198b1f47f8fac187b915287568510)
2013-11-09 07:28:18 -05:00
399d28777a
puppet: Move python-html2text to the app frontend dependencies.
...
Otherwise, the email mirror won't work on local server.
(imported from commit ddd388eeb8943f1ce84cf3d113525c1fc7b7b826)
2013-11-09 07:28:18 -05:00
180107588f
puppet: Organize the zulip_internal package list.
...
(imported from commit 53b4d28316e41d59f75bf070acfd776267682141)
2013-11-09 07:28:18 -05:00
657756b7e1
puppet: Move python-netifaces to zulip_internal (only used there).
...
(imported from commit 303f32f20b5d2f6962a04134eb569c69ab216c7c)
2013-11-09 07:28:18 -05:00
17db94e728
puppet: Move various dependencies to zulip_internal.
...
These are things that don't make sense to require on our local server
appliance systems.
(imported from commit 66a3ab750b0d27fa011b55c8f7ef9b22511de56c)
2013-11-09 07:28:18 -05:00
71e1f00fc6
puppet: Move our iptables config to zulip_internal.
...
(imported from commit f177b3989092f4fa7f00ae5bfb833ea23fe35489)
2013-11-09 07:28:18 -05:00
e957c8adb2
puppet: Move our SSHD config to zulip_internal.
...
(imported from commit 0c6314963ebe9246d8136dc6db3176d226dc2049)
2013-11-09 07:27:19 -05:00
312e3013af
Release desktop app v. 0.3.10
...
(imported from commit 0114d0b631ad027d6e57a5a264a8c8efd3438fa7)
2013-11-06 14:13:57 -05:00
82ef17207a
Aggregate receive and displayed times as well
...
(imported from commit e7789a627e4f1396d2c11c5a4b448135197324ab)
2013-11-06 13:37:15 -05:00
c9c26a9ecb
Remove 0.3.9 update while we prepare 0.3.10
...
(imported from commit f8f469dc93601d69f6974ed11efff259040d8ef2)
2013-11-06 12:58:09 -05:00
8dd4bf8f00
[puppet] Log endtoend send time on a per-realm basis as well as aggregate
...
(imported from commit 07226b20081d203af1f52776475228d9b6783869)
2013-11-06 11:25:00 -05:00
b2ea2dca81
puppet: Add the stats1 cert (was missing from git).
...
(imported from commit 909c70089f95a08cf62656432e09df170b322aaa)
2013-11-05 17:06:33 -05:00
b5979a3fed
[manual]: Rename zulip-internal puppet module to zulip_internal.
...
(imported from commit 64ac7ec0f3495b1fe7810da3d4d41263c52b9b3b)
2013-11-05 17:06:32 -05:00
Leo Franchi
Zev Benjamin
Jessica McKellar
Tim Abbott
Luke Faraone
Kevin Mehall
acrefoot