Commit Graph

144 Commits

Author SHA1 Message Date
Alex Vandiver 814198d649 installer: Abstract out version of postgres installed.
This allows for installing from-scratch with a different pinned
version of PostgreSQL, and provides a single place to change when the
default should increase.
2020-06-24 12:57:38 -07:00
Alex Vandiver ca9d27175b installer: Write PostgreSQL version based on puppet classes.
Using `/etc/init.d/postgresql` as the detection of if Postgres is on
the server is incorrect, because this line runs _before_ puppet and
any packages are installed.  Thus, it cannot tell the difference
between a new Ubuntu one-host first-time-install without PostgreSQL
yet, and one which is merely a front-end and will never have
PostgreSQL.  This leads to failures in first-time installs:

```
Error: Evaluation Error: Error while evaluating a Function Call,
  Could not find template 'zulip/postgresql//postgresql.conf.template.erb'
```

The only way to detect if PostgreSQL will be present in the _end_
state of the install is to examine the puppet classes that are
applied.

To do this, we must inspect `PUPPET_CLASSES`.  Unfortunately, this can
be fragile to subclassing (e.g. `zulip_ops::postgres_appdb`).  We
might desire to use `puppet apply --write-catalog-summary` to deduce
the _applied_ classes, which would unroll the inheritance; however,
this causes a chicken-and-egg problem, because `zulip.conf` must be
already written out (including a value for `postgresql.version`, if
necessary!) before such a puppet run could successfully complete.

Switch to predicating the `postgresql.version` key on the puppet
classes that are known to install postgres.
2020-06-24 12:57:38 -07:00
Alex Vandiver 253246185f installer: Update documentation.
Where appropriate, documentation wording is shared with
docs/production/install.md
2020-06-24 12:57:38 -07:00
Alex Vandiver 85dbb13c56 installer: Abstract out apt/yum divide into a variable.
This check is done in several places, using a somewhat fragile `case`
statement; move it into an explicit variable.
2020-06-24 12:57:38 -07:00
Alex Vandiver e4899eae8b installer: Sync the claimed supported distros with the check.
0f4b1076ad removed Ubuntu 16.04 "xenial" and Debian 9 "stretch" from
the printed list of supported operating systems, but left them in the
verification check that controls if that message is printed,
effectively continuing to support them.

Conversely, 439f0d3004 added Ubuntu 20.04 "focal" to the check, but
not to the printed list.

Synchronize to check and print the right supported distributions:
Ubuntu 18.04 "bionic", Ubuntu 20.04 "focal", and Debian 10 "buster".
2020-06-24 12:57:38 -07:00
Alex Vandiver 58cb7cecd8 installer: Remove `--remote-postgres`, redundant with `--no-init-db`.
The previous commit removed the only behavior difference between the
two flags; both of them skip user/database creation, and the tables
therein.

Of the two options `--no-init-db` is more explicit as to what it does,
as opposed to just one facet of when it might be used; remove
`--remote-postgres`.
2020-06-24 12:57:38 -07:00
Alex Vandiver 7c6a25a43d installer: Group and unify ordering of installer options.
This also adds the missing `--no-overwrite-settings` option to
`--help`.
2020-06-24 12:57:38 -07:00
Alex Vandiver b165b4144d installer: Prevent flags which conflict with `--no-overwrite-settings`.
Since `--postgres-missing-dictionaries` edits `/etc/zulip/zulip.conf`,
it interferes with the intent of `--no-overwrite-settings`.

Make the two settings conflict, to prevent this unclear state.
2020-06-23 13:40:28 -07:00
Alex Vandiver 7f4a2527c0 installer: Make `--no-overwrite-settings` also preserve `zulip.conf`.
This allows a path through the installer for places that have already
configured `zulip.conf`, by extending the existing flag and behavior.
2020-06-23 13:40:28 -07:00
Alex Vandiver 27100b4507 installer: Fix mis-indentation. 2020-06-23 13:36:26 -07:00
Alex Vandiver 5b7be7ba5d installer: Do not initialize db with --no-init-db.
The `--no-init-db` option previously only controlled if
`initialize-database` was run, which sets up the tables inside the
database.  If PostgreSQL was installed locally, it still attempted to
create the user and empty database.

This fails on hosts which are remote PostgreSQL hosts, and not
application hosts, as:

 - They may already have a local database, and while
 `initialize-datbase` will detect and offer to abort if one is
 found,`--no-init-db` seems like it should be the option to not
 overwrite it

 - `flush-memcached` requires that a local venv be installed, which it
 often is not on non-frontend machines.

Skip the database configuration when run with `--no-init-db`.
2020-06-23 13:36:26 -07:00
Alex Vandiver 49a7a66004 install: Pin new apt-based installs to the latest postgresql.
Since we now support Postgres versions from 10 to 12, we might as well
have new installations start on Postgres 12 to avoid unnecessary
migration/upgrade work.
2020-06-16 17:08:16 -07:00
arpit551 439f0d3004 install: Ad production support for Zulip on Ubuntu Focal.
Install script now runs on Focal.  Python 2 is now installed via the
`python2` package in Focal.
2020-05-25 16:58:42 -07:00
Tim Abbott 5187d5032c update-prod-static: Remove unused authors-not-required.
This argument hasn't done anything since we moved constructing the
/team page to a cron job.
2020-04-25 15:39:03 -07:00
Aman Agrawal 0f4b1076ad scripts: Remove Xenial and Stretch support from installation scripts.
Note that we leave support for them in `setup-apt-repo` and puppet,
since we're still supporting systems using Xenial for non-appserver
puppet rules.
2020-04-22 10:00:38 -07:00
arpit551 e6edf469ee ci: Remove the need of using TRAVIS in env.
Since now we want to use production suites on Circle CI so there
is no need to set TRAVIS in env while running scripts.

CIRCLECI is set default in the enviroment of Circle CI builds
so we can use it directly.

Also Travis CI had rabbitmq-server installed so we had to add workaround
in install script to avoid the error. That workaround is removed.
2020-04-21 14:46:40 -07:00
rht 7c9954afc6 install: Reduce the RAM limit to accommodate some definition of "2GB".
See also the discussion at
https://chat.zulip.org/#narrow/stream/3-backend/topic/2GB.20ram.20gotcha.
2020-01-24 12:35:37 -08:00
rht 6f5cbed5f1 prod install: Ignore reading VERSION_CODENAME if not available. 2020-01-22 23:19:18 -08:00
rht c2dcaf48d8 prod install: Add the CentOS version of SUPERVISOR_CONF_DIR. 2020-01-22 23:19:18 -08:00
Anders Kaseorg 3360df7ad1 generate_secrets: Enable memcached authentication in production.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-01-15 17:35:15 -08:00
rht cd3907648d prod install: Use ID_LIKE to help select os family. 2020-01-07 13:25:25 -08:00
rht bc94e8e815 prod install: Use /etc/os-release for Ubuntu/Debian to get os_id, os_version_id. 2020-01-07 13:25:25 -08:00
rht 9898c07e0d prod install: Add the CentOS version of the step to do dist-upgrade. 2020-01-07 13:25:25 -08:00
rht bf76696d67 prod install: Add the CentOS version of the step to install preparatory packages. 2020-01-07 13:25:25 -08:00
rht 6dd5dc32fc prod install: Add the CentOS version of the step to upgrade packages. 2020-01-07 13:25:25 -08:00
rht d88a7bbb42 prod install: Add the CentOS version of the step to update packages. 2020-01-07 13:25:25 -08:00
rht 49d7adb3cb prod install: Parse CentOS os identifications from /etc/os-release. 2020-01-07 13:25:25 -08:00
rht 771f6d213f prod install: Rename os_codename into os_version_id 2020-01-07 13:25:25 -08:00
Anders Kaseorg a78f8647d8 install: Run generate_secrets.py before zulip-puppet-apply.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-01-05 22:48:08 -08:00
Vishnu KS 6901087246 install: Use crudini for storing value of POSTGRES_MISSING_DICTIONARIES.
This simplifies the RDS installation process to avoid awkwardly
requiring running the installer twice, and also is significantly more
robust in handling issues around rerunning the installer.

Finally, the answer for whether dictionaries are missing is available
to Django for future use in warnings/etc. around full-text search not
being great with this configuration, should they be required.
2019-12-13 12:05:39 -08:00
Vishnu KS 6c97a36355 install: Support remote database services like RDS.
Documentation and variable names edited by tabbott.
2019-12-12 12:59:45 -08:00
Hemanth V. Alluri dac068df31 production: Finish adding production support for Zulip on Debian Buster.
This commit finishes adding end-to-end support for the install script
on Debian Buster (making it production ready). Some support for this
was already added in prior commits such as
99414e2d96.

We plan to revert the postgres hunks of this once we've built
tsearch_extras for our packagecloud archive.

Fixes #9828.
2019-08-17 12:22:32 -07:00
Anders Kaseorg caecd1c2ad install: Disable installation and provisioning on Ubuntu 14.04 Trusty.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-06-26 15:58:53 -07:00
Tim Abbott d2fbd62b63 install: Fix argument parsing for no-overwrite-settings option.
This had the same bug as our other recent change to the install
script.
2019-06-16 15:24:20 -07:00
Vishnu Ks 6e52779ed5 install: Include no-dist-upgrade in args list.
This was missed out in 2e51ac8c49
2019-06-14 14:24:08 -07:00
Tim Abbott 8a53686f41 install: Add --no-overwrite-settings option.
This commit needs more work to make this option reasonable.
2019-06-13 14:39:25 -07:00
Vishnu Ks 14ed0e283d install: Add option to skip dist-upgrade. 2019-06-05 15:50:02 -07:00
Rohitt Vashishtha 22a3b4230c Revert "install: Ensure that sudo is installed."
Now that we have the run_as_root helper function, we don't need to
install sudo to run Zulip in production

This reverts commit a7d7d181ea.

Fixes #10036.
2019-04-04 16:52:20 -07:00
Anders Kaseorg 4fef10f4e6 install: Improve error handling.
On usage errors (except --help), write usage message to stderr and
exit with nonzero status.

Forbid setting the hostname and email to the example values.  Those
are specifically checked for and would fail later.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 11:18:42 -08:00
Tim Abbott df436a55b4 scripts: Recommend apt update after enabling universe.
One needs to manually do an apt update after add-apt-repository, or it
won't actually work.
2019-01-26 12:29:05 -08:00
Tim Abbott 630968b632 provision: Use a more efficient approach for getting yarn version.
Since yarn has a package.json conveniently available, we can parse
that with jq, saving the expensive operation of starting up yarn.

This saves ~300ms in a no-op provision.
2018-12-07 14:36:54 -08:00
Tim Abbott 1303f8df20 docs: Fix missing quotes in `su zulip -c` documentation.
This fixes an actual user-facing issue in our mobile push
notifications documentation (where we were incorrectly failing to
quote the argument to `./manage.py register_server` making it not
work), as well as preventing future similar issues from occurring
again via a linter rule.
2018-11-30 12:12:27 -08:00
Anders Kaseorg 1597511de0 install: Check whether universe repository is enabled on Ubuntu.
Fixes #10417.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-11-28 17:18:43 -08:00
Rohitt Vashishtha 95ba947f13 setup-cerbot: Allow issuing certificates for multiple domains.
This commit allows specifying Subject Alternative Names to issue certs
for multiple domains using certbot. The first name passed to certbot-auto
becomes the common name for the certificate; common name and the other
names are then added to the SAN field. All of these arguments are now
positional. Also read the following for the certbot syntax reference:

https://community.letsencrypt.org/t/how-to-specify-subject-name-on-san/

Fixes #10674.
2018-11-13 12:47:31 -08:00
Tim Abbott 453c35d0f7 install: Provide a suggestive error message when missing Universe.
By far the dominant cause of errors when installing apt packages is
not having the Universe repository enabled in Ubuntu bionic (this
seems to have started happening a lot recently; I wonder if Ubuntu
changed the defaults for new server installs or something?).

In any case, providing that suggestion in the error output should help
reduce these a lot.
2018-11-12 10:56:39 -08:00
Tim Abbott e7bb833a37 install: Improve some error output for common errors.
This uses `set +x` to hide the `echo` output, and then sets the font
color to red.
2018-11-12 10:55:06 -08:00
Anders Kaseorg d0fb34e8af install: Fix shellcheck warnings.
In scripts/lib/install line 71:
ZULIP_PATH="$(readlink -f $(dirname $0)/../..)"
                          ^-- SC2046: Quote this to prevent word splitting.
                                    ^-- SC2086: Double quote to prevent globbing and word splitting.

In scripts/lib/install line 105:
mem_kb=$(cat /proc/meminfo | head -n1 | awk '{print $2}')
             ^-- SC2002: Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead.

In scripts/lib/install line 141:
apt-get -y dist-upgrade $APT_OPTIONS
                        ^-- SC2086: Double quote to prevent globbing and word splitting.

In scripts/lib/install line 145:
    $ADDITIONAL_PACKAGES
    ^-- SC2086: Double quote to prevent globbing and word splitting.

In scripts/lib/install line 254:
    if [ -n "ZULIP_ADMINISTRATOR" ]; then
             ^-- SC2157: Argument to -n is always true due to literal strings.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-18 15:26:21 -07:00
Tim Abbott a7d7d181ea install: Ensure that sudo is installed.
We use it to drop privileges from root to other users in the installer
process (which ideally, we would remove, but it will take some
annoying refactoring).

This should generally be safe to do, since the default sudo
permissions only allow root to use it anyway.

See https://github.com/zulip/zulip/issues/10036 for the follow-up
issue of removing the need to do this.
2018-10-16 15:34:04 -07:00
Tim Abbott 9054ce278f locale: Fix incorrect use of locale-gen to generate locales.
Previously, we were having issues installing on Debian Stretch with
non-English locales, because `locale-gen` actually doesn't take a
locale as an argument (and thus `locale-gen en_US.UTF-8` did nothing).
We should instead be calling localedef directly.

Thanks to Tom Daff for debugging this.

Fixes #10629.
2018-10-11 14:42:24 -07:00
xificurC 9e053c74cf scripts: Allow configuring a custom CA bundle for build process.
For building Zulip in an environment where a custom CA certificate is
required to access the public Internet, one needs to be able to
specify that CA certificate for all network access done by the Zulip
installer/build process.  This change allows configuring that via the
environment.
2018-08-13 15:45:28 -07:00