To deploy this, the zulip_internal::base and zulip_internal::munin classes must
be added to nagios.zulip.net.
(imported from commit 50d6a4ed19fcc9c62c7104977d69043bf5b9bbf9)
Before we deploy this commit, we must migrate the data from the staging redis
server to the new, dedicated redis server. The steps for doing so are the
following:
* Remove the zulip::redis puppet class from staging's zulip.conf
* ssh once from staging to redis-staging.zulip.net so that the host key is known
* Create a tunnel from redis0.zulip.net to staging.zulip.net
* zulip@redis0:~$ ssh -N -L 127.0.0.1:6380:127.0.0.1:6379 -o ServerAliveInterval=30 -o ServerAliveCountMax=3 staging.zulip.net
* Set the redis instance on redis0.zulip.net to replicate the one on staging.zulip.net
* redis 127.0.0.1:6379> slaveof 127.0.0.1 6380
* Stop the app on staging
* Stop redis-server on staging
* Promote the redis server on redis0.zulip.net to a master
* redis 127.0.0.1:6379> slaveof no one
* Do a puppet apply at this commit on staging (this will bring up the tunnel to redis0)
* Deploy this commit to staging (start the app on staging)
* Kill the tunnel from redis0.zulip.net to staging.zulip.net
* Uninstall redis-server on staging
The steps for migrating prod will be the same modulo s/staging/prod0/.
(imported from commit 546d258883ac299d65e896710edd0974b6bd60f8)
The zulip::redis puppet class should be added to all our frontends' zulip.conf
after this is deployed. No puppet apply is required.
(imported from commit ccea89f4779c6c49c0cbe837adcb5be21bfe55ab)
Otherwise, we will enable the postfix config on all frontends,
regardless of whether Enterprise deployments requested it.
(imported from commit 9592be3706adcee7547f6795f32fe7b8d85e71ee)
This removed the cronjob from all app_frontend servers and enables the
local Postfix mail server on the same.
This is a no-op on staging if the parent commit has already been
applied.
To deploy this commit, run a puppet-apply on prod.
(imported from commit 6d3977fd12088abcd33418279e9fa28f9b2a2006)
This will cause us to recieve messages sent to streams.staging.zulip.com
via the local Postfix daemon running on staging.
This commit does not impact prod. To deploy, a puppet-apply is needed on
staging.
(imported from commit 9eaedc28359f55a65b672a2e078c57362897c0de)
This will allow us to roll out the Postfix-based mirror on staging in
the future without impacting production mirroring.
This branch should be puppet-deployed first on prod, then staging.
(imported from commit eceaa6c02a06f7074cacc19c6439e5928eef3ae4)
This allows the utility to run on trac.zulip.net, which doesn't have a 'zulip'
database.
(imported from commit c8eabb89e5e161191d6f2c92ca2b1428b17a9aa0)
We don't do the sequence check because that requires read access to the database
itself, which the zulip user doesn't have.
(imported from commit fba7604826353b2974e9757f01dcb426297993b3)
We cannot use SNI for these legacy domains because old plugins still
connect to them.
This commit (along with the three previous commits) requires a lb0 nginx
deployment to function.
(imported from commit f47f3d7b597666508b3817d965fe8ce19d50c2c0)
To deploy, the certs need to manually be copied to lb0's /etc/ssl/certs
directory, the nginx config updated, and the server restarted
(imported from commit c70c7678cd010a1b2b0aba830ab3d862005bd627)
This replaces the old www.humbughq.com cert.
Contains these hostnames:
* www.humbughq.com
* api.humbughq.com
* humbughq.com
Generated per 9d674d6a0.
(imported from commit 0ef3f0ff2a02996246868466b5e634ebf45439a2)
These are redirect hosts, so they don't need their own IP. Supporting
non-SNI clients isn't a priority for us.
(imported from commit b1a8de8763ab944885518c868e4e30307d84c11d)
This will let us do normal puppet applies on our postgres hosts again.
Crudini is already installed and /etc/zulip/zulip.conf has already been edited
on the relevant hosts.
(imported from commit 8e2b88d2fe2f7b2367ecb73a50a299200fe381a0)
Note that this change can not currently be applied on postgres hosts due to the
postgres puppet config currently being slightly broken.
(imported from commit 5d8ddeabfd9612d469a048256d22949c0bfa6aba)
This is used by the Android app to authenticate without prompting for a
password.
To do so, we implement a custom authentication backend that validates
the ID token provided by Google and then tries to see if we have a
corresponding UserProfile on file for them.
If the attestation is valid but the user is unregistered, we return that
fact by modifying a dictionary passed in as a parameter. We then return
the appropriate error message via the API.
This commit adds a dependency on the "googleapi" module. On Debian-based
systems with the Zulip APT repository:
sudo apt-get install python-googleapi
For OS X and other platforms:
pip install googleapi
(imported from commit dbda4e657e5228f081c39af95f956bd32dd20139)
MIT implemented NTP rate-limiting to defend against on-going reflection attacks,
which was causing our nagios checks to fail intermittently. When the attacks
die down or when external sites fix their NTP configurations, checking against
time.mit.edu will stop failing. However, there also isn't much of a reason to
stick with checking against a single server.
(imported from commit 2c2a1a04646b880b010cbb4b6d94016b1eccd1a0)
Manual instructions:
This commit requires a puppet apply after deployment on both staging
and prod.
(imported from commit 2d10e33c6db2f5e9cc1204cdd5f2c91833da2a8e)
The manual step here is that we need to do the `puppet apply` before
pushing this commit, or `restart-server` will crash.
Previously we shut down everything in one group, which performed
poorly with supervisor's bad performance on restarting many daemons at
once. Now we shut down the unimportant stuff, then the important
stuff, bring back the important stuff, and then bring back the
unimportant stuff.
This new model has a little over 5s of downtime for the core
user-facing daemons -- which is still far more than would be ideal,
but a lot less than the 13s or so that we had before.
Here's some logs with the current setup for the tornado/django downtime:
2013-12-19 20:16:51,995 restart-server: Stopping daemons
2013-12-19 20:16:53,461 restart-server: Starting daemons
2013-12-19 20:16:57,146 restart-server: Starting workers
Compare with the behavior on master today:
2013-12-19 20:21:45,281 restart-server: Stopping daemons
2013-12-19 20:21:49,225 restart-server: Starting daemons
2013-12-19 20:21:58,463 restart-server: Done!
(imported from commit b2c1ba77f3dc989551d0939779208465a8410435)
Leo Franchi
Zev Benjamin
Luke Faraone
Tim Abbott
Jessica McKellar