Commit Graph

45 Commits

Author SHA1 Message Date
Anders Kaseorg 684dad8145 tools: Use root-based absolute import for tools.lib, etc.
Mypy can’t follow absolute imports based on directories other than the
root.  This was hiding some type errors due to ignore_missing_imports.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-05 12:21:52 -07:00
Alex Vandiver 88c19bf65a puppet: Catch when a comma is left out of puppet_classes.
With two space-separated classes in `puppet_classes`, the second one
is silently ignored.  With three of more, puppet generates the
following very opaque error message:

```
Error: Could not parse for environment production: This
Name has no effect. A value was produced and then forgotten (one or
more preceding expressions may have the wrong form)
```

Catch when this has happened, and give an error message to the user.

Fixes #18992.
2021-06-28 20:58:56 -04:00
Alex Vandiver 87a109e3e0 puppet: Pull in pinned puppet modules.
Using puppet modules from the puppet forge judiciously will allow us
to simplify the configuration somewhat; this specifically pulls in the
stdlib module, which we were already using parts of.
2021-05-27 21:14:48 -07:00
Alex Vandiver 3314fefaec puppet: Do not require a venv for zulip-puppet-apply.
0663b23d54 changed zulip-puppet-apply to
use the venv, because it began using `yaml` to parse the output of
puppet to determine if changes would happen.

However, not every install ends with a venv; notably, non-frontend
servers do not have one.  Attempting to run zulip-puppet-apply on them
hence now fails.

Remove this dependency on the venv, by installing a system
python3-yaml package -- though in reality, this package is already an
indirect dependency of the system.  Especially since pyyaml is quite
stable, we're not using it in any interesting way, and it does not
actually add to the dependencies, it is preferable to parsing the YAML
by hand in this instance.
2021-03-14 17:50:57 -07:00
Alex Vandiver 0663b23d54 puppet: Only prompt to apply if there are changes to apply.
Since `yaml` is not a module in the standard library, this requires
makings `zulip-puppet-apply` use the venv.
2021-02-23 18:16:02 -08:00
Anders Kaseorg 6e4c3e41dc python: Normalize quotes with Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 11741543da python: Reformat with Black, except quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 72d6ff3c3b docs: Fix more capitalization issues.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-23 11:46:55 -07:00
Anders Kaseorg bb4fc3c4c7 python: Prefer --flag=option over --flag option.
For less inflation by Black.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 17:51:09 -07:00
Anders Kaseorg fbfd4b399d python: Elide action="store" for argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg 1ded51aa9d python: Replace list literal concatenation with * unpacking.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:15:41 -07:00
Alex Vandiver c1923e19b0 puppet: --noop implies --force (i.e. no prompt).
The combination of `--force --noop` is potentially confusing, but
currently `--noop` makes no sense without `--force`, as it will prompt
and then not make changes.

Make `--noop` skip the prompt as well.
2020-08-02 12:47:31 -07:00
Alex Vandiver 17002f2a0e puppet: Allow passing an alternate config path to zulip-puppet-apply.
When temporary configuration changes are desired, this lets one set up
an alternate `zulip.conf` to apply while leaving the true one in place.
2020-07-06 18:30:16 -07:00
Alex Vandiver efe2b6e5cd puppet: Switch `zulip-puppet-apply` to argparse.
This allows additional arguments other than `-f` or `--force`.
2020-07-06 18:30:16 -07:00
Alex Vandiver 876ee4a8ed installer: Remove code specific to stretch or xenial.
Support for Xenial and Stretch was removed (5154ddafca, 0f4b1076ad,
8944e0ad53, 79acd5ae40, 1219a2e854), but not all codepaths were
updated to remove their conditionals on it.

Remove all code predicated on Xenial or Stretch.  debathena support
was migrated to Bionic, since that appears to be the current state of
existing debathena servers.
2020-06-24 12:57:38 -07:00
Anders Kaseorg 365fe0b3d5 python: Sort imports with isort.
Fixes #2665.

Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.

Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start.  I expect this change will increase pressure for us to split
those files, which isn't a bad thing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:45:32 -07:00
Anders Kaseorg 67e7a3631d python: Convert percent formatting to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 15:02:09 -07:00
arpit551 3971824d04 puppet: suppress puppet warnings with ruby 2.7.
Ubuntu Focal comes with ruby 2.7 and the latest puppet
has some issues with it so to suppress puppet
warnings with ruby 2.7 we added  RUBYOPT = "-W0" in
the environment.
2020-05-25 16:56:11 -07:00
rht 07808e35be parse_lsb_release: Use /etc/os-release instead of /etc/lsb-release. 2019-08-28 17:53:27 -07:00
Anders Kaseorg caecd1c2ad install: Disable installation and provisioning on Ubuntu 14.04 Trusty.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-06-26 15:58:53 -07:00
rht c9d54f7854 puppet: Remove vendored puppetlabs apt and stdlibs dependencies.
This commit works by vendoring the couple functions we still use from
puppetlabs stdlib (join and range), but removing the rest of the
puppetlabs codebase, and of course cleaning up our linter rules in the
process.

Fixes #7423.
2018-12-11 13:03:26 -08:00
Tim Abbott b3a4b78c6d puppet: Pass zulip scripts path as a FACTER to puppet.
This makes it possible for the Puppet codebase to access the path to
the relevant /home/zulip/deployments type directory that puppet was
run from, which in turn makes it possible to safely call scripts from
here.

Based on work by Rein Zustand.
2018-12-06 11:38:55 -08:00
Tim Abbott 3e3eb2aa7f scripts: Clarify names of running-as-root assertions.
This should make it more obvious that these functions will exit the
script if the check fails.
2018-11-19 10:58:34 -08:00
Rohitt Vashishtha 0975bbb39e scripts: Make zulip-puppet-apply check if the user is root.
Fixes #10833.
2018-11-19 10:58:34 -08:00
Anders Kaseorg fdc3b62d5f scripts/zulip-puppet-apply: Avoid shelling out for touch.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-25 16:54:46 -07:00
Raymond Akornor 5d39a0f0fc scripts: Replace calls to lsb release with our own parsing.
This improves the performance of these operations, by saving a ~50ms
Python process startup.  While not a major performance improvement, it
seems worth it, given how often these commands get run.

Fixes #9571.
2018-05-29 10:57:36 -07:00
Tim Abbott 37335960a0 puppet: Stop automatically creating hiera.yaml on newer distros.
This fixes some unnecessary warnings on systems running Ubuntu bionic.
2018-05-24 10:39:39 -07:00
rht ccf2792c1c refactor: Remove six.moves.configparser import. 2017-11-07 10:51:44 -08:00
rht 71188d7b0a scripts: Remove import print_function. 2017-09-29 15:43:30 -07:00
Tim Abbott 6d5dac96c6 zulip-puppet-apply: Fix flushing of stdout for Python 3.
Apparently, this UI worked just fine with Python 2, but failed to
actually print usefully with Python 3.
2017-08-27 18:36:39 -07:00
Greg Price a099e698e2 py3: Switch almost all shebang lines to use `python3`.
This causes `upgrade-zulip-from-git`, as well as a no-option run of
`tools/build-release-tarball`, to produce a Zulip install running
Python 3, rather than Python 2.  In particular this means that the
virtualenv we create, in which all application code runs, is Python 3.

One shebang line, on `zulip-ec2-configure-interfaces`, explicitly
keeps Python 2, and at least one external ops script, `wal-e`, also
still runs on Python 2.  See discussion on the respective previous
commits that made those explicit.  There may also be some other
third-party scripts we use, outside of this source tree and running
outside our virtualenv, that still run on Python 2.
2017-08-16 17:54:43 -07:00
Tim Abbott f6f2c62caf zulip-puppet-apply: Fix running it as ./zulip-puppet-apply. 2017-02-27 15:25:14 -08:00
Tim Abbott 609082c475 puppet: Remove hardcoding of /root/zulip from puppet path.
This is an important prerequisite to being able to remove dependence
on the /root/zulip symlink altogether.
2017-02-08 11:13:19 -08:00
Tim Abbott 3239197fdb puppet: Silence puppet warnings about missing hiera.yaml. 2016-07-30 21:25:19 -07:00
Eklavya Sharma 11732f9ab0 Make all scripts in scripts/ pass mypy check. 2016-07-24 00:17:21 +05:30
Tim Abbott 12028339a3 puppet: Use --detailed-exitcodes to return nonzero exit code on failure.
Apparently, puppet has messed up exit codes and doesn't by default
return the usual 0=success, nonzero=failure codes.  By default, it
seems to always return 0; and with `--detailed-exitcodes`, it returns
the complicated thing documented in the comments.

We fix this by checking the exit code and translating it to what we
actually care about, namely whether errors occurred.

See https://tickets.puppetlabs.com/browse/PUP-2754 for details.

Fixes #1094.
2016-07-08 12:11:44 -07:00
Eklavya Sharma 149938d468 Change shebangs from python2.7 to python. 2016-05-29 05:03:08 -07:00
Eklavya Sharma 7b8cb105bf Apply Python 3 futurize transform libmodernize.fixes.fix_imports_six
Refer to #256
2016-03-10 22:03:30 -08:00
Steven Oud d5435fad1d Consistently use /usr/bin/env python2.7 in shebangs and commands. 2015-10-21 22:58:21 +00:00
Tim Abbott 9748780192 Remove unnecessary puppet.conf configuration.
Fixes #23.
2015-09-26 21:34:19 -07:00
Tim Abbott c38bad0b65 zulip-puppet-apply: Fix the force option handling.
(imported from commit e3a0f3902ac183c7c30ba0e8ed27e7bd9924ff2b)
2013-11-05 17:06:31 -05:00
Zev Benjamin b3e6d49d2f puppet: Switch puppet to using include-style class declarations
This allows us to declare classes more than once, which we need as
we're making our class dependency graph more complicated.  See
http://docs.puppetlabs.com/puppet/2.7/reference/lang_classes.html#aside-history-the-future-and-best-practices

(imported from commit d4199b0df0d92d166fbc3559957f67ee9cf2ecdc)
2013-11-05 14:14:19 -05:00
Zev Benjamin 787215d743 [manual] Switch over to new /etc/zulip/zulip.conf config file
Run the following commands as root before deploying this branch:
 # /root/zulip/tools/migrate-server-config
 # rm /etc/zulip/machinetype /etc/zulip/server /etc/zulip/local /etc/humbug-machinetype /etc/humbug-server /etc/humbug-local

(imported from commit aa7dcc50d2f4792ce33834f14761e76512fca252)
2013-11-05 14:14:19 -05:00
Zev Benjamin 8ca76ba13f Split our puppet configuration in two
The 'zulip' module contains the general setup for any server that
will run our app.  The 'zulip-internal' module contains configuration
that is internal to Zulip the company.

(imported from commit 23c99dd889577c2917ddbf76892cf8f6cc66a13e)
2013-10-30 15:42:25 -04:00
Tim Abbott 7c6ca81df5 Move zulip-puppet-apply to scripts/.
(imported from commit c4e4c98f0a50feb31f7a716e04009d814cf047e7)
2013-10-28 10:54:48 -04:00