This change will make it so that processes related to the app.humbughq.com
server are run under supervisord, which uses a state machine model to ensure
that programs are running. It also ensure process startup order.
We will need to manually switch the old way of running server (in screen) into
this new way of doing things, on both staging and prod (app_frontend.pp has been
updated appropriately). This means:
1) cp servers/puppet/modules/humbug/files/supervisord/conf.d/humbug.conf /etc/supervisord/conf.d
2) installing the supervisor package.
3) killing those while loops in that screen session
4) mkdir /var/log/humbug (as root)
5) /etc/init.d/supervisord start
6) check that nothing broke
(imported from commit 055269a70973db89acd69049e01b185fabdc8f90)
Text search was not that great partially because Postgres wasn't
using a ispell dictionary (Postgres term) before. We now pull in
Hunspell and use its dictionary and affix rules.
It is Ok to run with this new configuration before updating our full
text column and index that will be coming in the next few commits.
Manual steps for deploy:
1) On both postgres0 and postgres1 (both before moving on to step 2),
install the hunspell-en-us package
2) On staging, run migration 0022
3) On both postgres0 and postgres1, copy the appropriate postgresql.conf
file over
4) On both postgres0 and postgres1, run `pg_ctlcluster 9.1 main reload`
(imported from commit 706bf0f6ecc46c712cea10b73c34fd9d1dfd4767)
Note that this file needs to be copied over manually as part of the
process of starting up a new replica.
(imported from commit a9f14b695ef2b6b4d48b6180d187c3babf5a667c)
This requires manual steps on deploy to each of staging and prod:
(1) Run the new update-deployment code to setup the initial deployment directory.
(2) Restart all the programs running in screen sessions.
(3) Deploy the nginx changes and restart nginx.
(imported from commit 1ffe27933ee79274dc0a93d35c9938712de0ef36)
I've already installed these packages on staging, and will install them on prod
when the commit hits master. So there is no manual prod deployment step here.
(imported from commit 1b77e771f938305dfbeb797c3ea2a7e3897e78a7)
The new nginx configuration file needs to be copied to
/etc/nginx/humbug-include and nginx needs to be restarted when this
commit is deployed.
(imported from commit 6c43f3c2c7a6acee6a852c672c96a38bda01dd0d)
Because we're storing 25,000+ message content objects in memcached on
server restart, we were sometimes exceeding the 64MB cap even during
the restart process. 256MB should be safely large enough to not have
the issue but not so large as to seriously consume resources on our
app frontends (which currently have 7GB of RAM).
(imported from commit 1a64319e50c9dadf0bc65e2e4dbf08f4cc1b9c38)
Leo Franchi
acrefoot
Zev Benjamin
Luke Faraone
Tim Abbott
Jessica McKellar
Luke Faraone
Keegan McAllister