Commit Graph

418 Commits

Author SHA1 Message Date
Tim Abbott 682d78bc30 puppet: Add default logrotate configuration.
This should prevent Zulip from eating all the disk now that we are
using the race-free WatchedFileHandler rather than
TimedRotatingFileHandler.
2016-08-24 14:30:54 -07:00
Tim Abbott 1c40df9363 static_asset_compiler: Use correct package name on trusty. 2016-08-22 23:47:03 -07:00
Tim Abbott 97b622dffa puppet: Fix postgres_appdb_tuned manifest on trusty. 2016-08-22 23:41:07 -07:00
Tim Abbott 750e43518f puppet: Ensure supervisord starts on boot with systemd.
This is important to ensure that Zulip comes up properly after a
reboot on Ubuntu Xenial.
2016-08-22 23:25:54 -07:00
Tim Abbott 2be597dd09 puppet: Move support for accepting a loadbalancer to zulip manifests.
Previously, we didn't have an easy process for how one should do this.

Now, it's an (undocumented) zulip.conf setting!
2016-08-22 17:19:03 -07:00
Tim Abbott 88a123d5e0 Fix excessive CPU usage by rabbitmq-numconsumers Nagios checks.
The previous model for these Nagios checks was kinda crazy -- every
minute, we'd run a full `rabbitmctl list_consumers` for each of the
dozen+ consumers that we have, and then do the exact same parsing
logic for each to determine whether the target queue has a running
consumer to write out a state file.

Because `rabbitmctl list_consumers` takes a small amount of resources,
on systems where CPU is very limited (e.g. t2 style AWS instances),
this minor CPU wastage could be problematic.

Now we just do that `rabbitmqctl list_consumers` once per minute, and
output all the state files from a single command.

Further TODO items on this front include removing the hardcoded list
of queues.
2016-08-12 14:09:36 -07:00
Tim Abbott 852af83d3c check_postgres_replication_lag: Fix psql command line.
This allows the plugin to be run as users other than the "zulip" user,
where the "zulip" database would not be the default.
2016-08-12 13:19:08 -07:00
Tim Abbott 6496fe2a53 travis: Remove rabbitmq nodename dependency on hostname.
Because rabbitmq doesn't support changing the nodename of a running
rabbitmq node, Zulip installations suffered a plague of issues where
e.g. a Zulip server would reboot, the hostname would change, and
suddenly the local rabbitmq instance being used by Zulip would stop
working.

We address this problem by using, by default, a fixed rabbitmq
nodename, but providing server administrators the option to set the
rabbitmq nodename used by Zulip however they choose.

To upgrade an existing server to use this new configuration, one will
need to add something like the following to /etc/zulip/zulip.conf:

[rabbitmq]
nodename = zulip@localhost

However, I don't believe we have the puppet code in place to make this
work correctly at initial installation without rabbitmq-server being
already installed (but off), as we can easily setup in Travis CI but I
haven't been willing to do for the installer.  So for now, this just
fixes our Travis CI problems.

Fixes: #1579.
2016-08-12 09:38:23 -07:00
Tim Abbott 2581ac166c puppet: Automatically scale default memcached memory allocation.
Previously, we used a fixed memcached memory allocation of 512MB,
regardless of the size of the server.  While that is a good allocation
for a server with 4GB of RAM, for servers with less, we should
decrease the allocation, and for a large server with much more RAM, we
should increase it.  We still support the user overriding the
configuration setting, but this produces more sensible defaults.
2016-08-11 20:27:45 -07:00
Tim Abbott 2e0e8193d4 puppet: Make memcached memory allocation a variable. 2016-08-11 20:27:45 -07:00
Tim Abbott 3edf880c9b Remove moreutils from zulip_internal::base.
We recently added this to zulip::base, so it would be a duplicate.
2016-08-11 18:12:59 -07:00
Tim Abbott 9afb1c7a71 puppet: Stop wholesale replacing /etc/redis/redis.conf.
Zulip had only patches the redis configuration in one small way, which
resulted in unnecessary portability issues for using Redis on
different versions of Linux.  We replace this with just a adding an
include mechanism to the redis config.

While we're at it, we configure this to take advantage of the
new REDIS_PASSWORD secret to automatically configure redis passwords.
2016-08-11 17:26:04 -07:00
Tim Abbott cb21584ffe check_send_receive_time: Support being run in a cron job. 2016-08-11 14:48:21 -07:00
Tim Abbott 4dcbaf1e6b check_send_receive_time: Cleanup unnecessary print statement. 2016-08-11 14:48:21 -07:00
Tim Abbott 1d6ebd2b3d check_fts_update_log: Fix psycopg2 arguments.
* Fixes passing a string argument rather than an actual Python
  argument.
* Switches to hardcoding the database to connect to rather than the
  user, so this check can be run as an arbitrary user.
2016-08-03 14:58:46 -07:00
Tim Abbott 1c65508624 puppet: Add postgres user to Zulip group. 2016-08-02 14:59:25 -07:00
Tim Abbott 04fc3ff1e1 pg_backup_and_purge: Don't try to use a virtualenv.
This makes this actually work if the postgres server is different from
the Zulip app frontend.
2016-08-02 14:59:25 -07:00
Tim Abbott 4bdd9d3769 check_postgres_backup: Don't try to use a virtualenv.
The dependencies for this tool are installed using apt.
2016-08-02 14:59:25 -07:00
Tim Abbott a5115d54ee env-wal-e: Rename s3_backup_bucket to s3_backups_bucket.
This makes it consistent with the other variables in this file.
2016-08-02 14:59:15 -07:00
Tim Abbott ff80daef16 puppet: Move zulipsecret into its own file.
Apparently puppet doesn't support declaring multiple functions in the
same file.
2016-08-02 14:55:51 -07:00
Tim Abbott 75b5d021fa Remove unused puppet-common third-party module code. 2016-07-31 19:24:42 -07:00
Tim Abbott 6158acb41b nagios: Fix path to check_debian_packages. 2016-07-31 14:25:07 -07:00
Tim Abbott 6954dd84ab postgres_appdb_tuned: Add SSL certificate path configs. 2016-07-31 14:25:07 -07:00
Tim Abbott d66f6b8176 Fix postgresql configuration template for Ubuntu Xenial.
I incorrectly assumed that this was working because Travis CI passed
the build, whereas in fact Travis CI only tests the Trusty templates.
2016-07-31 14:24:34 -07:00
Tim Abbott 8b285ec0ff puppet: Read camo key from zulip-secrets.conf. 2016-07-31 00:23:24 -07:00
Tim Abbott 2b40309029 puppet: Add new zulipsecret function. 2016-07-31 00:23:24 -07:00
Tim Abbott 298e040bac puppet: Strip newlines in zulipconf function.
This avoids creating unnecessary extra newlines in config generated
using this.
2016-07-31 00:23:24 -07:00
Tim Abbott 16a4ce1bd2 puppet: Fix Nagios check_disk flagging special filesystems. 2016-07-30 23:57:31 -07:00
Tim Abbott c764b46cef puppet: Fix missing base include in nagios manifest. 2016-07-30 23:48:41 -07:00
Tim Abbott bc9bdd53aa puppet: Use hostgroup for postgres_other monitoring. 2016-07-30 23:46:05 -07:00
Tim Abbott 6b34bee806 puppet: Remove unused ancient humbug-default apache site. 2016-07-30 23:43:55 -07:00
Tim Abbott 217faed3b3 Update check_postgres plugin to version 2.22. 2016-07-30 23:19:40 -07:00
Tim Abbott 1f549dcfab check_fts_update_log: Fix use on non-appbd machines. 2016-07-30 21:48:17 -07:00
Tim Abbott 35edd8c9dd puppet: Move crudini dependency to base.pp.
This allows us to use the new crudini-based zulipconf function in any
of our manifests in the future.
2016-07-30 21:23:12 -07:00
Tim Abbott 922a4acdc5 puppet: Add zulip.conf setting for listen_addresses tuning. 2016-07-30 21:23:12 -07:00
Tim Abbott b2e32e6e3f puppet: Add postgres replication option for postgres_appdb_tuned.pp.
This adds a setting that controls whether to include the Zulip default
streaming replication configuration in the postgres configuration file.
2016-07-30 21:23:12 -07:00
Tim Abbott 3ab0295061 puppet: Add zulip.conf setting for effective_io_concurrency tuning. 2016-07-30 21:23:12 -07:00
Tim Abbott f3db368a3c puppet: Add zulip.conf setting for random_page_cost tuning. 2016-07-30 21:23:12 -07:00
Tim Abbott 63c757eac3 puppet: add zulipconf function for reading settings from zulip.conf.
This makes it convenient for us to have optional user-defined settings
in the main Zulip puppet configuration.
2016-07-30 21:23:12 -07:00
Eklavya Sharma 3b3b5c7c16 Rename management command email-mirror to email_mirror.
All other zulip management command names have underscores, so
rename email-mirror to email_mirror.

This will also make it possible to import this module, which will
help in writing tests for it.
2016-07-28 14:52:09 -07:00
Tim Abbott 69528790a5 check_worker_memory: Fix handling of no queue workers running. 2016-07-28 13:26:31 -07:00
Eklavya Sharma 408d070170 puppet/: Make subprocess calls unicode-aware. 2016-07-26 12:06:41 -07:00
Eklavya Sharma 3d87c376c2 Make zulip-ec2-configure-interfaces pass mypy check. 2016-07-24 12:52:58 +05:30
Eklavya Sharma 0a5aa2ccc7 Make check_send_receive_time pass mypy check. 2016-07-24 12:41:06 +05:30
Tim Abbott 2a209e46dd puppet: Fix remaining hardcoding of postgres 9.1 in manifests. 2016-07-20 21:11:22 -07:00
Tim Abbott d529bc12ef pg_backup_and_purge: Fix hardcoding of postgres version. 2016-07-19 19:19:42 -07:00
Tim Abbott bdb1ce04a2 puppet: Move pg_backup_and_purge to main config. 2016-07-19 19:19:42 -07:00
Tim Abbott 81136ff092 env-wal-e: Eliminate hardcoding of AWS keys.
Pre-Zulip being open sourced, this file just had the AWS keys for
backups hardcoded.

Instead, these are simply read from zulip-secrets.conf.
2016-07-19 19:19:42 -07:00
Tim Abbott 3023745ed7 puppet: Move env-wal-e to Zulip main distribution.
This doesn't yet include the dependencies for env-wal-e.
2016-07-19 19:19:42 -07:00
Tim Abbott d0dcc8bf26 puppet: Create static_asset_compiler manifest. 2016-07-19 16:38:09 -07:00