If there is a replication primary configured, and no current database,
then we check all of the required secrets are in place, then pull down
the latest backup and trigger a PostgreSQL restart, which will pick up
downloading the remaining WAL logs to catch up, then start streaming
from the configured primary.
The `tidy` parameter is buggy, and ignores all ordering
metaparameters. This is fixed in Puppet 7[^1], but it's helpful to
resolve it now. Specifically, this fixes bugs with tidy running too
early, and deleting the old version of a package before its new
version is installed or symlinked, leaving a race condition if
anything tries to run the binary in this window.
This is mostly not a problem for Supervisor-managed processes, since
the binary is already running, and can continue to run if it is tidied
out from under the running process. For stand-alone tools like wal-g,
which are run frequently by PostgreSQL, this may cause issues if
PostgreSQL tries to call them during a puppet run.
Remove all complicated uses of tidy, and replace them with an `exec`
which does the equivalent. We also generate `file` resources for
binaries, making them easier (and clearer) to specify as dependencies.
[^1]: https://puppet.atlassian.net/browse/PUP-10688
By default, autossh writes to syslog; setting AUTOSSH_DEBUG is the
only way to produce output to STDERR. Timestamp that and log that to
the logfile, making the logs perhaps useful.
puppet hard-fails if it can't find the binary to run in `$PATH`, so we
need to make the `unless` short-circuit to false if puppet itself is
not installed yet (as during initial installation).
These default to off, because in situations with thousands of queues,
consumers, and producers, they cause unreasonable overhead. Our use
case has few enough queues that we do want to be able to inspect them
individually.
Enable per-object Prometheus metrics, per [1].
[1]: 78851828ec/deps/rabbitmq_prometheus (configuration)
We require a `pg_dump` whose version matches the version of the server
we are configured against (see 3a8b4b0205). Installing the latest
`postgresql-client` does not guarantee that we have such a binary
present.
Alex Vandiver
Anders Kaseorg