puppet: wal-g backups are required for replication.

Previously, it was possible to configure `wal-g` backups without
replication enabled; this resulted in only daily backups, not
streaming backups.  It was also possible to enable replication without
configuring the `wal-g` backups bucket; this simply failed to work.

Make `wal-g` backups always streaming, and warn loudly if replication
is enabled but `wal-g` is not configured.
This commit is contained in:
Alex Vandiver 2022-03-11 02:39:34 +00:00 committed by Tim Abbott
parent 6496d43148
commit 6f5ae8d13d
5 changed files with 19 additions and 28 deletions

View File

@ -508,23 +508,15 @@ things you need to be careful about when configuring it:
Zulip's configuration allows for [warm standby database Zulip's configuration allows for [warm standby database
replicas][warm-standby] as a disaster recovery solution; see the replicas][warm-standby] as a disaster recovery solution; see the
linked PostgreSQL documentation for details on this type of linked PostgreSQL documentation for details on this type of
deployment. Zulip's configuration leverages `wal-g`, our [database deployment. Zulip's configuration builds on top of `wal-g`, our
backup solution][wal-g], and thus requires that it be configured for [database backup solution][wal-g], and thus requires that it be
the primary and all secondary warm standby replicas. configured for the primary and all secondary warm standby replicas.
The primary should have log-shipping enabled, with: Warm spare replicas should also have `wal-g` backups configured, and
their primary replica and replication username set:
```ini ```ini
[postgresql] [postgresql]
replication = yes
```
Warm spare replicas should have log-shipping enabled, and their
primary replica and replication username configured:
```ini
[postgresql]
replication = yes
replication_user = replicator replication_user = replicator
replication_primary = hostname-of-primary.example.com replication_primary = hostname-of-primary.example.com
``` ```
@ -688,14 +680,6 @@ setting](https://www.postgresql.org/docs/current/runtime-config-connection.html#
Override PostgreSQL's [`random_page_cost` Override PostgreSQL's [`random_page_cost`
setting](https://www.postgresql.org/docs/current/runtime-config-query.html#GUC-RANDOM-PAGE-COST) setting](https://www.postgresql.org/docs/current/runtime-config-query.html#GUC-RANDOM-PAGE-COST)
#### `replication`
Set to true to enable replication to enable [log shipping replication
between PostgreSQL servers](#postgresql-warm-standby). This should be
enabled on the primary, as well as any replicas, and further requires
configuration of
[wal-g](export-and-import.md#backup-details).
#### `replication_primary` #### `replication_primary`
On the [warm standby replicas](#postgresql-warm-standby), set to the On the [warm standby replicas](#postgresql-warm-standby), set to the

View File

@ -13,7 +13,7 @@ class zulip::profile::postgresql {
$listen_addresses = zulipconf('postgresql', 'listen_addresses', undef) $listen_addresses = zulipconf('postgresql', 'listen_addresses', undef)
$replication = zulipconf('postgresql', 'replication', undef) $s3_backups_bucket = zulipsecret('secrets', 's3_backups_bucket', '')
$replication_primary = zulipconf('postgresql', 'replication_primary', undef) $replication_primary = zulipconf('postgresql', 'replication_primary', undef)
$replication_user = zulipconf('postgresql', 'replication_user', undef) $replication_user = zulipconf('postgresql', 'replication_user', undef)
@ -38,6 +38,13 @@ class zulip::profile::postgresql {
} }
if $replication_primary != '' and $replication_user != '' { if $replication_primary != '' and $replication_user != '' {
if $s3_backups_bucket == '' {
$message = @(EOT/L)
Replication is enabled, but s3_backups_bucket is not set in zulip-secrets.conf! \
Streaming replication requires wal-g backups be configured.
|-EOT
warning($message)
}
if $zulip::postgresql_common::version in ['10', '11'] { if $zulip::postgresql_common::version in ['10', '11'] {
# PostgreSQL 11 and below used a recovery.conf file for replication # PostgreSQL 11 and below used a recovery.conf file for replication
file { "${zulip::postgresql_base::postgresql_confdir}/recovery.conf": file { "${zulip::postgresql_base::postgresql_confdir}/recovery.conf":

View File

@ -787,8 +787,8 @@ effective_io_concurrency = <%= @effective_io_concurrency %>
listen_addresses = <%= @listen_addresses %> listen_addresses = <%= @listen_addresses %>
<% end -%> <% end -%>
<% if @replication != '' || (@replication_primary != '' && @replication_user != '') -%> <% if @s3_backups_bucket != '' -%>
# Replication # Streaming backups and replication
max_wal_senders = 5 max_wal_senders = 5
archive_mode = on archive_mode = on
archive_command = '/usr/bin/timeout 10m /usr/local/bin/env-wal-g wal-push %p' archive_command = '/usr/bin/timeout 10m /usr/local/bin/env-wal-g wal-push %p'

View File

@ -818,8 +818,8 @@ effective_io_concurrency = <%= @effective_io_concurrency %>
listen_addresses = <%= @listen_addresses %> listen_addresses = <%= @listen_addresses %>
<% end -%> <% end -%>
<% if @replication != '' || (@replication_primary != '' && @replication_user != '') -%> <% if @s3_backups_bucket != '' -%>
# Replication # Streaming backups and replication
max_wal_senders = 5 max_wal_senders = 5
archive_mode = on archive_mode = on
archive_command = '/usr/bin/timeout 10m /usr/local/bin/env-wal-g wal-push %p' archive_command = '/usr/bin/timeout 10m /usr/local/bin/env-wal-g wal-push %p'

View File

@ -839,8 +839,8 @@ effective_io_concurrency = <%= @effective_io_concurrency %>
listen_addresses = <%= @listen_addresses %> listen_addresses = <%= @listen_addresses %>
<% end -%> <% end -%>
<% if @replication != '' || (@replication_primary != '' && @replication_user != '') -%> <% if @s3_backups_bucket != '' -%>
# Replication # Streaming backups and replication
max_wal_senders = 5 max_wal_senders = 5
archive_mode = on archive_mode = on
archive_command = '/usr/bin/timeout 10m /usr/local/bin/env-wal-g wal-push %p' archive_command = '/usr/bin/timeout 10m /usr/local/bin/env-wal-g wal-push %p'