Commit Graph

171 Commits

Author SHA1 Message Date
Anders Kaseorg 392175d6e8 Use #!/usr/bin/env for bash shebangs.
/bin/sh and /usr/bin/env are the only two binaries that NixOS provides
at a fixed path (outside a buildFHSUserEnv sandbox).

This discussion was split from #11004.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-12-17 17:21:08 -08:00
Tim Abbott 2558f101af docs: Add documentation for `if False` mypy pattern in scripts.
This should help make it clear what's going on with these scripts.
2018-12-17 11:12:53 -08:00
rht 711b0fda8b scripts: Initialize setup-apt-repo-debathena. 2018-12-11 13:01:26 -08:00
Tim Abbott adf27aae4c python: Remove now-unnecessary str_utils library.
This library was absolutely essential as part of our Python 2->3
migration process, but all of its calls should be either no-ops or
encode/decode operations.

Note also that the library has been wrong since the incorrect
refactoring in 1f9244e060.

Fixes #10807.
2018-11-27 11:57:54 -08:00
Rohitt Vashishtha 95ba947f13 setup-cerbot: Allow issuing certificates for multiple domains.
This commit allows specifying Subject Alternative Names to issue certs
for multiple domains using certbot. The first name passed to certbot-auto
becomes the common name for the certificate; common name and the other
names are then added to the SAN field. All of these arguments are now
positional. Also read the following for the certbot syntax reference:

https://community.letsencrypt.org/t/how-to-specify-subject-name-on-san/

Fixes #10674.
2018-11-13 12:47:31 -08:00
Anders Kaseorg 310027f970 terminate-psql-sessions: Fix shellcheck warnings.
In scripts/setup/terminate-psql-sessions line 16:
major=$(echo "$version" | cut -d. -f1,2)
^-- SC2034: major appears unused. Verify use (or export if used externally).

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:48:57 -07:00
Tim Abbott 24f3419449 postgres-init-db: Throw an error if not run as root.
Fixes part of the dicsussion in #552.
2018-08-21 11:36:50 -07:00
Tim Abbott 460ea76d96 terminate-psql-sessions: Remove postgres <9.2 support.
Those older versions of postgres reached end-of-life upstream, and it
makes this shell script easier to read.
2018-08-09 15:24:47 -07:00
Anders Kaseorg 26d378fb7d terminate-psql-sessions: Fix shellcheck warnings.
In scripts/setup/terminate-psql-sessions line 5:
    [ "$1" = "`echo -e "$1\n$2" | sort -V | tail -n1`" ]
              ^-- SC2006: Use $(..) instead of legacy `..`.
                          ^-- SC1117: Backslash is literal in "\n". Prefer explicit escaping: "\\n".

In scripts/setup/terminate-psql-sessions line 20:
major=$(echo $version | cut -d. -f1,2)
             ^-- SC2086: Double quote to prevent globbing and word splitting.

In scripts/setup/terminate-psql-sessions line 24:
tables=$(echo "'$@'" | sed "s/ /','/g")
                ^-- SC2145: Argument mixes string and array. Use * or separate argument.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:26 -07:00
Anders Kaseorg 5a5497c6a1 setup-certbot: Fix shellcheck warnings.
In scripts/setup/setup-certbot line 64:
if [ -z "$DOMAIN" -o -z "$EMAIL" ]; then
                  ^-- SC2166: Prefer [ p ] || [ q ] as [ p -o q ] is not well defined.

In scripts/setup/setup-certbot line 73:
        method_args=(--webroot --webroot-path=/var/lib/zulip/certbot-webroot/)
                               ^-- SC2191: The = here is literal. To assign by index, use ( [index]=value ) with no spaces. To keep as literal, quote it.

In scripts/setup/setup-certbot line 112:
if [ -z "$deploy_hook" ]; then
         ^-- SC2128: Expanding an array without an index only gives the first element.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:26 -07:00
Anders Kaseorg e495eaa339 postgres-init-db: Fix shellcheck warnings.
In scripts/setup/postgres-init-db line 12:
records=`su "$POSTGRES_USER" -c "psql -Atc 'SELECT COUNT(*) FROM zulip.zerver_message;' zulip" | cat`
        ^-- SC2006: Use $(..) instead of legacy `..`.

In scripts/setup/postgres-init-db line 35:
source "$(dirname "$0")/terminate-psql-sessions" postgres zulip zulip_base
^-- SC1090: Can't follow non-constant source. Use a directive to specify location.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:26 -07:00
Anders Kaseorg edc5a7bdd0 install: Fix shellcheck warnings.
In scripts/setup/install line 18:
if [ $failed = 1 ]; then
     ^-- SC2086: Double quote to prevent globbing and word splitting.

In scripts/setup/install line 19:
    echo -e "\033[0;31m"
             ^-- SC1117: Backslash is literal in "\0". Prefer explicit escaping: "\\0".

In scripts/setup/install line 25:
    echo -e "\033[0m"
             ^-- SC1117: Backslash is literal in "\0". Prefer explicit escaping: "\\0".

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:26 -07:00
Anders Kaseorg bb19fed5a7 initialize-database: Fix shellcheck warnings.
In scripts/setup/initialize-database line 38:
    echo -e "\033[32mPopulating default database failed."
             ^-- SC1117: Backslash is literal in "\0". Prefer explicit escaping: "\\0".

In scripts/setup/initialize-database line 42:
    echo -e "\033[0m"
             ^-- SC1117: Backslash is literal in "\0". Prefer explicit escaping: "\\0".

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:26 -07:00
Anders Kaseorg 68261c7e29 generate-self-signed-cert: Fix shellcheck warnings.
In scripts/setup/generate-self-signed-cert line 36:
if [ -n "$EXISTS_OK" ] && [ -e "$KEYFILE" -a -e "$CERTFILE" ]; then
                                          ^-- SC2166: Prefer [ p ] && [ q ] as [ p -a q ] is not well defined.

In scripts/setup/generate-self-signed-cert line 40:
if [ -z "$FORCE" ] && [ -e "$KEYFILE" -o -e "$CERTFILE" ]; then
                                      ^-- SC2166: Prefer [ p ] || [ q ] as [ p -o q ] is not well defined.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:26 -07:00
Anders Kaseorg 78225e4d04 configure-rabbitmq: Fix shellcheck warnings.
In scripts/setup/configure-rabbitmq line 13:
sudo rabbitmqctl $RABBITMQ_FLAGS delete_user "$RABBITMQ_USERNAME" || true
                 ^-- SC2086: Double quote to prevent globbing and word splitting.

In scripts/setup/configure-rabbitmq line 14:
sudo rabbitmqctl $RABBITMQ_FLAGS delete_user zulip || true
                 ^-- SC2086: Double quote to prevent globbing and word splitting.

In scripts/setup/configure-rabbitmq line 15:
sudo rabbitmqctl $RABBITMQ_FLAGS delete_user guest || true
                 ^-- SC2086: Double quote to prevent globbing and word splitting.

In scripts/setup/configure-rabbitmq line 16:
sudo rabbitmqctl $RABBITMQ_FLAGS add_user "$RABBITMQ_USERNAME" "$RABBITMQ_PASSWORD"
                 ^-- SC2086: Double quote to prevent globbing and word splitting.

In scripts/setup/configure-rabbitmq line 17:
sudo rabbitmqctl $RABBITMQ_FLAGS set_user_tags "$RABBITMQ_USERNAME" administrator
                 ^-- SC2086: Double quote to prevent globbing and word splitting.

In scripts/setup/configure-rabbitmq line 18:
sudo rabbitmqctl $RABBITMQ_FLAGS set_permissions -p / "$RABBITMQ_USERNAME" '.*' '.*' '.*'
                 ^-- SC2086: Double quote to prevent globbing and word splitting.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:26 -07:00
Anders Kaseorg a5407e1c7d scripts: Replace node-wrapper with a symlink.
Commit 00e057bf44 (#4727) simplified
node-wrapper to a one-line wrapper script for performance.  Copying
the binary was proposed and rejected because node finds some of its
modules relative to its own path.  But a symlink doesn’t have that
issue, as you can verify with

    node -e 'console.log(require.resolve.paths("foo"))'

(To find its own path, node uses `process.execPath`, which resolves
symlinks, and there’s no plausible reason for that behavior to change.
https://github.com/nodejs/node/blob/v8.11.1/lib/module.js#L708-L717
https://github.com/nodejs/node/blob/v10.7.0/lib/internal/modules/cjs/loader.js#L761-L770)

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-30 11:48:10 -07:00
jeaye 4999474cce install: Add a couple Docker-specific options to the certbot scripts.
--agree-tos is useful for the Docker environment, where we won't have
an interactive shell present for agreeing to the ToS.

--deploy-hook is also useful for the Docker environment; it makes it
possible to customize what deploy hook (if any) we pass into the
underlying cerbot command.
2018-07-23 17:45:59 -07:00
Sampriti Panda 8bf80f699f docker: Add PGroonga support to Postgresql Dockerfile.
Fixes #9466.
2018-05-23 11:53:32 -07:00
Jason Michalski 24fbd7205d docker: Add postgres docker build with full text search.
This is multi-stage build which first builds tsearch-extras with the
current version of postgres and then configs postgres for zulip. The
zulip config installs the hunspell dictionaries, stop words file,
tsearch-extras, and creates the initial database.

**Testing Plan:**
1) `docker-compose up` the existing config.
2) Build the new image
3) Edit docker-compose.yml to use the new image id
4) `docker-compose up` and verify full text search is still working.
2018-05-16 13:14:24 -07:00
Aditya Bansal e14974ff2c scripts: Change use of typing.Text to str. 2018-05-10 14:19:49 -07:00
Tim Abbott b3e4c702d1 generate_secrets: Fix handling of an empty secrets file.
This is now a condition that happens during installation, because we
now create an empty file for this in puppet.
2018-05-06 22:12:32 -07:00
Tim Abbott 4ee762a52c apt: Add packagecloud repository for Debian. 2018-05-05 10:03:03 -07:00
Rhea Parekh ccefaf7b26 scripts: Remove the depreciated script 'postgres-reset-sequences'. 2018-04-10 13:07:14 -07:00
Tim Abbott 5fb3bff67e certbot: Don't prompt when installing apt packages.
The comment included in this commit explains the somewhat messy
situation that requires running certbot twice as part of this
installer.

Fixes #8486.
2018-03-28 17:09:38 -07:00
Greg Price eb7eef02e8 install: Suppress initialize-database instructions when redundant.
To do it, add a blob of getopt boilerplate at the top of the script;
and also fix a couple of nits of shell style while here.
2018-03-06 19:43:02 -08:00
Greg Price 08135e3666 install: Don't try to chown a self-signed cert to zulip.
The zulip user has no need to see this file; it's used by nginx.
And when we set up the cert early in install, there's no zulip user
yet anyway, so this fails.
2018-02-08 17:08:16 -08:00
rht 2b19f17fc9 scripts: Remove u prefix from strings. 2018-02-05 12:11:33 -08:00
Greg Price 0f836f51a5 generate-self-signed-cert: Fix a mostly-harmless bug.
Thanks to the magic of `set -x`, I noticed this:
```
+ cat
++ ssl-cert
/tmp/src/zulip-server/scripts/setup/generate-self-signed-cert: line 49: ssl-cert: command not found
+ apt-get install -y openssl
[...]
```

In other words, we were trying to run `ssl-cert` -- the name of a
Debian package I meant to refer to in a comment inside the templated
temporary config file for `openssl req` -- as if it were a command.
It wasn't, hence the error.

Because `set -e` has loopholes like a sieve, this didn't cause the
script to exit, just produced this funny output and presumably caused
the config file's comment to be missing a word.  In principle, it
could do something surprising if for some reason there were a command
named `ssl-cert` on PATH.

Fix it.
2018-01-23 18:08:52 -08:00
Greg Price f26b34405e install --self-signed-cert: Generate our own, rather than use system's.
This gives us just one way of adopting a self-signed cert, rather than
one script which would generate a new one and an option to another
which would symlink to the system's snakeoil cert.  Now those two
codepaths converge, and do the same thing.

The small advantage of generating our own over the alternative is that
it lets us set the name in the cert to EXTERNAL_HOST, rather than the
system's hostname as embedded in the system snakeoil certs.  Not a big
deal, but might make things go slightly smoother if some browsers are
lenient (in a way that they probably shouldn't be.)
2018-01-23 18:08:52 -08:00
Greg Price c27f36a82a install: Rename generate-self-signed-cert.
This script generates one certificate.
2018-01-23 18:08:52 -08:00
Greg Price d258e48f3d install: Replace our generate-self-signed-certs script.
Take the core of the logic from how Debian generates the system's
/etc/ssl/certs/ssl-cert-snakeoil.pem ; that gives me more confidence
in the various config choices, and it also demonstrates a much cleaner
way to use the `openssl` tool.  Also replace the outer shell logic for
CLI and logging with a cleaner version.
2018-01-23 18:08:52 -08:00
Greg Price ac88f8ae1b setup-certbot: Stop automatically "agreeing" to the LE TOS.
It's not appropriate for our script to pass the `--agree-tos` flag
without any evidence of the user actually having any knowledge of,
let alone intent to agree to, any such ToS.  Stop doing that.
Fortunately this script hasn't been part of any release, so it's
likely that no users have gone down this path.
2018-01-22 18:55:46 -08:00
Rhea Parekh 460e9b4634 Scripts: Add script for resetting postgres sequences for imports. 2017-12-01 06:38:08 -08:00
Aditya Bansal 79576797b0 thumbor: Auto generate thumbor_key and add it to zulip settings. 2017-11-16 22:34:14 -08:00
Greg Price a56fca81f1 setup-certbot: Require hostname and email.
The script already won't work without them; so if the user gets the
invocation wrong, give a halfway-reasonable error rather than just
crash into the ground.
2017-11-15 21:50:41 -08:00
Greg Price df8548aaf1 setup-certbot: Fix the usage message, and add the recently-added options. 2017-11-15 21:50:41 -08:00
Greg Price 8f387ba4d4 setup-certbot: Add option to choose verification method.
This allows the installer to continue using this script for the
`standalone` method, while the no-argument form now uses the same
`webroot` method as the renewal cron job, suitable for running
by hand to adopt Certbot after initial install.
2017-11-15 21:50:41 -08:00
Greg Price 7c887a6741 setup-certbot: Use set -x.
When there's a failure, this can make it much less confusing
to figure out.
2017-11-15 21:50:41 -08:00
Greg Price ae901309fc certbot: Control auto-renew with a zulip.conf setting.
This causes the cron job to run only when a Zulip-managed certbot
install is actually set up.

Inside `install`, zulip.conf doesn't yet exist when we run
setup-certbot, so we write the setting later.  But we also give
setup-certbot the ability to write the setting itself, so that we
can recommend it in instructions for adopting certbot in an
existing Zulip installation.
2017-11-15 21:50:41 -08:00
Greg Price dd32348fca setup-certbot: Eliminate obnoxious wget spew. 2017-11-15 21:50:41 -08:00
Greg Price 3f1f70fae2 setup-certbot: Treat potential existing certs with kid gloves.
This helps make this script suitable to run on existing installations,
by mitigating any worry about clobbering existing certs with links to
the new ones, in case the admin changes their mind or was using the
certs for something else too.
2017-11-15 21:50:41 -08:00
Tim Abbott 2afc3b9e50 certbot: Move path to /usr/local/sbin.
[greg: fixed typo bug]
2017-11-15 21:50:41 -08:00
Tim Abbott 38dc43fdf8 certbot: Use --standalone mode during installer.
This should help avoid problems, since we don't need to install
`nginx` before using this mode.
2017-11-08 12:32:26 -08:00
rht 8b6b4e043f install: Add option to get certs via certbot.
While this doesn't quite complete our plans for certbot support (it's
not documented, etc.), this is a great stride forward.
2017-10-27 17:19:34 -07:00
Tim Abbott be619fe881 lint: Wrap many very long lines in the Python codebase.
This decreases the maximum line length in our Python codebase to 130.
2017-10-26 17:31:58 -07:00
Tim Abbott 8c8613c3c7 install: Add comments documenting installer wrapper. 2017-10-01 15:48:36 -07:00
rht 71188d7b0a scripts: Remove import print_function. 2017-09-29 15:43:30 -07:00
rht b937e1fede Remove the rest of `import six` outside zerver.lib.
We no longer need the six module, now that we're Python 3 only.
2017-09-27 17:09:15 -07:00
julianasobreira df2d448f7b python: Add lint rule banning 'from os.path import'.
This enforces our use of a consistent style in how we access Python
modules; "from os.path import dirname" is a particularly popular
abbreviation inconsistent with our style, and so it deserves a lint
rule.

Commit message and error text tweaked by tabbott.

Fixes #6543.
2017-09-22 04:55:38 -07:00
Tim Abbott b8e7369dee mypy: Remove type: ignores not needed in Python 3. 2017-08-25 11:04:20 -07:00