Commit Graph

53 Commits

Author SHA1 Message Date
Zev Benjamin 41e3a89398 [manual] puppet: Puppetize Munin
To deploy this, the zulip_internal::base and zulip_internal::munin classes must
be added to nagios.zulip.net.

(imported from commit 50d6a4ed19fcc9c62c7104977d69043bf5b9bbf9)
2014-02-13 13:26:40 -05:00
Zev Benjamin 631783f3cd [manual] Use dedicated Redis server for staging
Before we deploy this commit, we must migrate the data from the staging redis
server to the new, dedicated redis server.  The steps for doing so are the
following:

* Remove the zulip::redis puppet class from staging's zulip.conf
* ssh once from staging to redis-staging.zulip.net so that the host key is known
* Create a tunnel from redis0.zulip.net to staging.zulip.net
  * zulip@redis0:~$ ssh -N -L 127.0.0.1:6380:127.0.0.1:6379 -o ServerAliveInterval=30 -o ServerAliveCountMax=3 staging.zulip.net
* Set the redis instance on redis0.zulip.net to replicate the one on staging.zulip.net
  * redis 127.0.0.1:6379> slaveof 127.0.0.1 6380
* Stop the app on staging
* Stop redis-server on staging
* Promote the redis server on redis0.zulip.net to a master
  * redis 127.0.0.1:6379> slaveof no one
* Do a puppet apply at this commit on staging (this will bring up the tunnel to redis0)
* Deploy this commit to staging (start the app on staging)
* Kill the tunnel from redis0.zulip.net to staging.zulip.net
* Uninstall redis-server on staging

The steps for migrating prod will be the same modulo s/staging/prod0/.

(imported from commit 546d258883ac299d65e896710edd0974b6bd60f8)
2014-02-10 13:23:28 -05:00
Zev Benjamin 1d7976d332 puppet: Add manifest for dedicated Redis server
(imported from commit 894ad5ca005de0fb9a64bfb58da374f72734eb8d)
2014-02-10 13:23:28 -05:00
Luke Faraone 602f7f96e5 Move postfix inclusion from public app_frontend to internal manifest
Otherwise, we will enable the postfix config on all frontends,
regardless of whether Enterprise deployments requested it.

(imported from commit 9592be3706adcee7547f6795f32fe7b8d85e71ee)
2014-02-07 01:01:33 -05:00
Luke Faraone 24f8492236 [manual] Enable local email mirror on all frontends.
This removed the cronjob from all app_frontend servers and enables the
local Postfix mail server on the same.

This is a no-op on staging if the parent commit has already been
applied.

To deploy this commit, run a puppet-apply on prod.

(imported from commit 6d3977fd12088abcd33418279e9fa28f9b2a2006)
2014-02-06 10:26:56 -05:00
Luke Faraone 30a6fd3bd7 [manual] Enable postfix email mirror on staging
This will cause us to recieve messages sent to streams.staging.zulip.com
via the local Postfix daemon running on staging.

This commit does not impact prod. To deploy, a puppet-apply is needed on
staging.

(imported from commit 9eaedc28359f55a65b672a2e078c57362897c0de)
2014-02-04 10:38:17 -05:00
Luke Faraone 882047515c [manual] Move polling email mirror to prod from staging
This will allow us to roll out the Postfix-based mirror on staging in
the future without impacting production mirroring.

This branch should be puppet-deployed first on prod, then staging.

(imported from commit eceaa6c02a06f7074cacc19c6439e5928eef3ae4)
2014-02-04 10:38:17 -05:00
Luke Faraone 374acb7f24 [puppet] Move email mirror cron to public module
This way we can reference it in the documentation.

(imported from commit 37d5cbfcfb745e2b44768674f53d7ba450518cd0)
2014-02-04 10:38:17 -05:00
Luke Faraone 3263d09939 Convert zmirror to use puppet apt module for debathena sources
(imported from commit 080d59d2ac750d03b55460752d7fe7d02e72611c)
2014-01-31 13:43:04 -05:00
Luke Faraone aa52475e96 Switch to puppetlabs/apt
(imported from commit b2f581280dc7877051ef79d86eac671bfd455ace)
2014-01-31 13:43:04 -05:00
Zev Benjamin 20e4e31dcf puppet: Update env-wal-e to take the S3 bucket to use from /etc/zulip/zulip.conf
This will let us do normal puppet applies on our postgres hosts again.

Crudini is already installed and /etc/zulip/zulip.conf has already been edited
on the relevant hosts.

(imported from commit 8e2b88d2fe2f7b2367ecb73a50a299200fe381a0)
2014-01-16 15:23:21 -05:00
Luke Faraone 16ae70948f Move python-googleapi dep to public Zulip manifest
(imported from commit 20298f82fbd674b3cf6b67b7741bf800b9733f36)
2014-01-13 16:24:21 -05:00
Luke Faraone 3948e1673d [manual] Accept OAuth2 tokens for API login via Google Apps
This is used by the Android app to authenticate without prompting for a
password.

To do so, we implement a custom authentication backend that validates
the ID token provided by Google and then tries to see if we have a
corresponding UserProfile on file for them.

If the attestation is valid but the user is unregistered, we return that
fact by modifying a dictionary passed in as a parameter. We then return
the appropriate error message via the API.

This commit adds a dependency on the "googleapi" module. On Debian-based
systems with the Zulip APT repository:
    sudo apt-get install python-googleapi

For OS X and other platforms:
    pip install googleapi

(imported from commit dbda4e657e5228f081c39af95f956bd32dd20139)
2014-01-13 13:30:55 -05:00
Leo Franchi 20f3b3af8f Fix zulip->zulip_internal puppet path change for apns checker
(imported from commit 1fd43a4f4907c24fcbbda73bbaf3cf092a6cace1)
2014-01-10 21:38:59 -05:00
Leo Franchi 91c54754fb [puppet] Add the apns-token crontab file to puppet
(imported from commit f12001453c9ca924c801a6000927e3ee2696a392)
2014-01-10 21:38:57 -05:00
Jessica McKellar 61d660f9f3 [manual] digest: move cron job from staging to all app frontends.
Manual instructions:

This commit requires a puppet apply after deployment on both staging
and prod.

(imported from commit 2d10e33c6db2f5e9cc1204cdd5f2c91833da2a8e)
2013-12-20 12:50:23 -05:00
Tim Abbott d62ca820db puppet: Fix permissions on /etc/cron.d/log2zulip.
(imported from commit 33ee5ae97b09b3925849940262ecd0bcbce38a3f)
2013-12-17 16:22:14 -05:00
Tim Abbott 6ccf19bed6 Run log2zulip on load balancers too.
(imported from commit 74c8be20d2d03aa524f05b7681febe9a9be9cdff)
2013-12-17 13:46:00 -05:00
Tim Abbott 8dcf7d4cc3 [puppet] Add log2zulip tool for sending log files to Zulip.
This tool is a little crude; it runs out of a cron job and will
forward to staging a notice about any new lines in the declared log
files, truncating if there are more than 10 lines.

(imported from commit 6748ddff1def0907b061dc278a3a848bd2e933f1)
2013-12-17 11:02:55 -05:00
Jessica McKellar 8bb1caec8f [manual] digest: add the cron job that will trigger digest emails.
Manual deployment instructions:

On staging, do a puppet apply.

No action needs to be taken for the prod deploy.

(imported from commit 0f6e5ab22aaeacfcc69d57de12f2bb6fac6f0635)
2013-12-17 10:47:16 -05:00
Tim Abbott b6acbe040c Fix missing nginx service notifications on configuration changes.
(imported from commit 0bfce276bab3704e508f6c8a58c9434e9fc224cd)
2013-12-16 13:44:50 -05:00
Luke Faraone b0a0853bd2 Specify full fingerprint rather than short key ID
(imported from commit fc4e9d51c440000e469f8e3882739215a3bcb022)
2013-12-11 10:54:30 -05:00
Luke Faraone 510b3349a7 Switch to downloading keys via SSL in puppet
(imported from commit 05d2b0626338f09370614e916050cfcee7f14829)
2013-12-11 10:54:30 -05:00
Tim Abbott 64807c0628 nginx: Ensure zulip-include files are distributed to the right systems.
There were a few recently introduced bugs, and this also cuts down on
our having to review diffs that don't actually affect the relevant
server when doing updates.

(imported from commit 43f3cff9a414bc1632f45a8222012846353e8501)
2013-12-05 15:25:25 -05:00
Tim Abbott b8a151ca4e Revert "[puppet] Add cron job to restart our workers daily."
This reverts commit 0b0180b0751f6c618d877b9c9ffc2b8287254e4d.

(imported from commit a81c552100345d369ffcaf69f28a86dea0893128)
2013-12-04 10:27:45 -05:00
Tim Abbott 3971f18de8 loadbalancer: Fix missing location-sockjs config.
(imported from commit 27b168e73014d7b7c71fb00ce5b75271393fc491)
2013-11-26 12:22:17 -05:00
Zev Benjamin d7d98aaacc puppet: Move /etc/iptables/rules to /etc/iptables/rules.v4
The /etc/init.d/iptables-persistent initfile changed to expect there to be two
files in /etc/iptables (rules.v4 and rules.v6) instead of a single rules file.
Several of our machines are currently running without iptables rules as a
result.

(imported from commit 266c2ff26b77f7c9ae793690b0d544ee4cfa5020)
2013-11-23 08:10:44 -05:00
Tim Abbott 8bfbaab1d5 Fix typo in puppet directories for zulip_internal.
(imported from commit 52627a9e71dfc28bedd6c955069da46d3ef56e83)
2013-11-21 10:06:40 -05:00
Leo Franchi c6b862d26d Add desktop sso puppet folder dependencies
(imported from commit 63d8cad722cf015a5925a28ab860431be31175be)
2013-11-20 19:12:53 -05:00
Tim Abbott f0c6b63526 [puppet] Add cron job to restart our workers daily.
This is to ensure that if we have an interval where we're not doing
prod deploys, we don't have to worry about worker memory leaks killing
us.

(imported from commit 0b0180b0751f6c618d877b9c9ffc2b8287254e4d)
2013-11-20 18:34:16 -05:00
Tim Abbott b50db26a18 puppet: Add monitoring for camo.
(imported from commit b3cf29b02de285cf860fc173183cb6f4f3a17c74)
2013-11-19 15:25:14 -05:00
Tim Abbott 1b009c47fc Clean up our nginx configuraiton to make better use of app.d.
Now app.d is something that any app frontend will read, and we just
have secondary manifests add additional files to the app.d directory
for custom stuff.

This fixes the issue that we were incorrectly including the
lb0-related app configuration in the enterprise version.

(imported from commit dec8dcdf2506b82e51186ff936c26dc1cd6cf61b)
2013-11-15 15:04:13 -05:00
Tim Abbott 6826ef4e9a puppet: Switch from nginx to nginx-full.
(imported from commit 38dd5966d75946842b39e4e619d82ebbb0fb041c)
2013-11-15 15:04:13 -05:00
Leo Franchi be3257de62 Add sso sparkle paths and files
(imported from commit f8a953f4262b170931792ed223f1ddc29c5fe5ed)
2013-11-14 10:22:35 -05:00
Jessica McKellar 1cb339a23d Expose password-protected /enterprise/download through nginx.
(imported from commit 8fba915bf0d0c718138ac62bd4333aef4e845d55)
2013-11-13 16:20:43 -05:00
Tim Abbott a16a7a028c Fix internal postgres_common.pp.
We didn't remove python-argparse from the requirements when that was
removed, and we also still need python-pip to install wal-e :(.

(imported from commit b82d3b429cffe0a3993819358511e11268ee2fef)
2013-11-13 15:35:45 -05:00
Zev Benjamin c1bdb0c7a8 puppet: Move the removal of 30-postgresql-shm.conf into zulip::postgres_appdb
(imported from commit 6a4d089fd47a8ba6ea92eeac321e3077fa0d8cc4)
2013-11-13 12:02:01 -05:00
Tim Abbott 17a98b3afd puppet: Move embedly to the internal puppet configuration.
(imported from commit 9b5c2f2726f2cac5bba5619ee9b7371dada0ea35)
2013-11-12 09:34:25 -05:00
Tim Abbott 5de6b879b7 puppet: Move minify-js and statsd dependencies to internal manifest.
(imported from commit 3363fb962a4fde575591b42db888b92bb6edd0f5)
2013-11-12 09:34:25 -05:00
Tim Abbott bbf8424b3e puppet: Move our ops repository apt configuration to internal module.
(imported from commit 2aca8e8c2edbd87a77fd5f00b3ae250484721fb4)
2013-11-12 09:34:25 -05:00
Tim Abbott 138d7053b7 puppet: Move the wal-e dependencies to the internal postgres config.
(imported from commit 67251263ec98e5b141f4c7587042b4db7aed36f2)
2013-11-09 07:28:19 -05:00
Tim Abbott 43e1c5e47d puppet: Move our camo installation setup to its own manifest.
(imported from commit 401018c3ff49fea485c3c8b4adb42574bb0b54b0)
2013-11-09 07:28:19 -05:00
Tim Abbott b902cc6eb1 puppet: Fix loadbalancer dependencies to include nodejs.
(imported from commit 9962e8ee774b13abdb95028d44cd687cee63aa13)
2013-11-09 07:28:19 -05:00
Tim Abbott a3b813e7af puppet: Comment the dependency lists for some service manifests.
(imported from commit 49817eb7daeab7cb83799d116ac829c7cd8e84e7)
2013-11-09 07:28:19 -05:00
Tim Abbott f552149772 puppet: Organize the puppet configuration for various internal services.
(imported from commit ad3525f608dbc5ceb04e6829fb1da0b3baba3258)
2013-11-09 07:28:19 -05:00
Tim Abbott 709850ab9e puppet: Organize the puppet configuration for zmirror.
(imported from commit 4e9d1771fd5198b1f47f8fac187b915287568510)
2013-11-09 07:28:18 -05:00
Tim Abbott 399d28777a puppet: Move python-html2text to the app frontend dependencies.
Otherwise, the email mirror won't work on local server.

(imported from commit ddd388eeb8943f1ce84cf3d113525c1fc7b7b826)
2013-11-09 07:28:18 -05:00
Tim Abbott 180107588f puppet: Organize the zulip_internal package list.
(imported from commit 53b4d28316e41d59f75bf070acfd776267682141)
2013-11-09 07:28:18 -05:00
Tim Abbott 657756b7e1 puppet: Move python-netifaces to zulip_internal (only used there).
(imported from commit 303f32f20b5d2f6962a04134eb569c69ab216c7c)
2013-11-09 07:28:18 -05:00
Tim Abbott 17db94e728 puppet: Move various dependencies to zulip_internal.
These are things that don't make sense to require on our local server
appliance systems.

(imported from commit 66a3ab750b0d27fa011b55c8f7ef9b22511de56c)
2013-11-09 07:28:18 -05:00