Commit Graph

436 Commits

Author SHA1 Message Date
Harshit Bansal 48662a79a5 clean-venv-cache: Fix an exception with unusual deployment dirs. 2017-10-31 00:01:09 -07:00
Harshit Bansal 8c9ea94878 scripts: Fix an issue in `purge-old-deployments` script.
We were not including the real path of the symlinks due to which we
were incorrectly deleting deployments pointed by last/current/next.
2017-10-30 23:09:51 -07:00
Harshit Bansal 1871d6fe1f minor: Remove unnecessary path juggling in `get_recent_deployments()`. 2017-10-29 14:38:20 -07:00
Tim Abbott 11ab545f3b install: Set the locale so our dependencies can install.
Many pip packages don't install properly without a US locale.
2017-10-29 11:49:08 -07:00
rht 8b6b4e043f install: Add option to get certs via certbot.
While this doesn't quite complete our plans for certbot support (it's
not documented, etc.), this is a great stride forward.
2017-10-27 17:19:34 -07:00
Tim Abbott be619fe881 lint: Wrap many very long lines in the Python codebase.
This decreases the maximum line length in our Python codebase to 130.
2017-10-26 17:31:58 -07:00
Shekh Ataul d239f77966 refactor: Replace mkdir_p functions with Python 3 builtin.
This didn't exist in Python 2, but it does in Python 3, so we get to
reap the rewards of dropping Python 2 support.

Fixes #7082.
2017-10-25 11:06:11 -07:00
Tim Abbott 6b1eb647e4 nginx: Fix bugs in new nginx configuration checks. 2017-10-24 14:29:36 -07:00
Tim Abbott 730c77c7df docs: Document scripts/setup/generate-self-signed-certs.
And more generally clean up our non-LetsEncrypt SSL docs.

This should make it a bit easier to setup a Zulip server.
2017-10-24 13:48:14 -07:00
Tim Abbott 1b653409f4 install: Provide a nicer error message for bad nginx configuration.
This also covers missing SSL configuration errors nicely.
2017-10-24 13:39:39 -07:00
rht 5755f475f4 upgrade: Add "must be run as root" check.
The logic and text is based on scripts/lib/upgrade-zulip.
2017-10-23 23:01:54 -07:00
rht dcb46afa90 upgrade: Remove an extraneous `sudo` prefix.
Because scripts/upgrade-zulip-from-git must be run from root anyway.
2017-10-23 23:01:46 -07:00
rht c8090aafa4 lint: Remove several unused imports. 2017-10-18 10:56:11 -07:00
rht a603a4f9f5 Remove `from __future__ import absolute_import`.
Except in:
- docs/writing-bots-guide.md, because bots are supposed to be Python 2
  compatible
- puppet/zulip_ops/files/zulip-ec2-configure-interfaces, because this
  script is still on python2.7
- tools/lint
- tools/linter_lib
- tools/lister.py

For the latter two, because they might be yanked away to a separate repo
for general use with other FLOSS projects.
2017-10-17 22:59:42 -07:00
Tim Abbott 2ae2a94444 provision: Stop using shared var/ for caching apt state.
This didn't work at all when one did a `vagrant destroy` and then
`vagrant up`, because the cache state would be preserved even though
the machine is gone.

Fixes #5981.
2017-10-17 21:15:58 -07:00
rht c77c2f084c
upgrade-zulip-from-git: Fix rerun script and script description. 2017-10-18 04:41:22 +02:00
rht 9ab54e5bd7 scripts/lib/install: Add flag to specify key settings.
This should make it easier to script the installation process, and
also conveniently are the options one would want for the --certbot
option.

Significantly modified by tabbott to have a sane right interface,
include --help, and avoid printing all the `set -x` garbage before the
usage notices.
2017-10-03 16:56:45 -07:00
Tim Abbott 8c8613c3c7 install: Add comments documenting installer wrapper. 2017-10-01 15:48:36 -07:00
Alexander Trost 6a7d34316c Add scripts to be used in docker-entrypoint.sh.
Based on #450, with commits
restructured by Rein Zustand.
Tweaks by Rein Zustand:
- Replace configure-cert with generate-self-signed-certs
- `mv scripts/lib/create-zulip-admin.sh scripts/lib/create-zulip-admin`
2017-09-30 09:55:48 -07:00
rht 71188d7b0a scripts: Remove import print_function. 2017-09-29 15:43:30 -07:00
Alexander Trost dea507f4b8 Add script to create admin from CLI. 2017-09-27 20:15:41 -07:00
rht b937e1fede Remove the rest of `import six` outside zerver.lib.
We no longer need the six module, now that we're Python 3 only.
2017-09-27 17:09:15 -07:00
Tim Abbott 358cb40ed1 cache: Add backwards compatibility for emoji cache.
This allows the emoji cache cleaning code to run against old emoji
caches.
2017-09-25 17:06:02 -07:00
Tim Abbott db982672b1 caches: Don't depend on yarn.lock existing.
This allows our cache cleaning code to run on servers that have copies
of Zulip 1.6 and older around.
2017-09-25 17:05:45 -07:00
Tim Abbott 0f19e501a6 caches: Suppress unnecessary output when cleaning caches.
This should make the cache cleaning process a lot less spammy.
2017-09-25 16:34:03 -07:00
Tim Abbott 703b87c116 caches: Move clean-unused-caches to scripts/lib/. 2017-09-25 16:34:03 -07:00
Tim Abbott 86a07baf40 zulip_tools: Skip the lock directory.
This is the one special directory that usually lives in deployments/
and is not a deployment.  Make sure we don't treat it as a deployment.
2017-09-25 15:15:32 -07:00
Harshit Bansal 90f6ffa23d purge-old-deployments: Remove an unnecessary check.
Deployments whose name is not in the format of a timestamp are
always included in the `recent_deployments` and are not deleted,
hence we don't need to check for them explicitly.
2017-09-25 11:51:24 -07:00
Harshit Bansal a6caf30ca7 scripts: Fix an issue in `get_recent_deployments()` due to relative paths.
We were checking for whether an item in the deployments directory
represents a directory but were using its relative path which was
causing a false value to be returned for all items irrespective of
their being a directory or not if the script was invoked from some
where other than the deployments directory.
2017-09-25 11:51:24 -07:00
Harshit Bansal 6ff7da04de emoji: Remove `NotoColorEmoji.ttf`.
We no longer use glyphs from `NotoColorEmoji.ttf` so removing this.
2017-09-24 04:51:33 -07:00
Harshit Bansal c8c1c8ef43 emoji: Remove `AndroidEmoji.ttf`. 2017-09-24 04:51:33 -07:00
Tim Abbott 7e7b80f5e8 purge_old_deployments: Fix running from any path. 2017-09-24 04:42:14 -07:00
Harshit Bansal 57161a92a1 scripts: Rearrange the arguments of `purge_unused_caches()`.
This commit re-arranges the arguments of `purge_unused_caches()`
function in order to remain consistent with other similar functions
in the library like `may_be_perform_caching()`.
2017-09-24 04:37:31 -07:00
Harshit Bansal df7ea375c1 scripts: Make default mode of cache-cleaning scripts much less verbose.
Print a detailed report only if `--verbose` flag is specified.

Fixes: #6632.
2017-09-24 04:37:31 -07:00
Harshit Bansal 480ae626fb scripts: Add `--verbose` flag to purge-old-deployments script. 2017-09-24 04:37:31 -07:00
Harshit Bansal 3e8469a717 zulip_tools: Remove the now unused `GENERIC_CACHE_SCRIPT_PARSER`.
This has been replaced by `parse_cache_script_args()`.
2017-09-24 04:37:31 -07:00
Harshit Bansal 7f752f0942 scripts: Use `parse_cache_script_args()` in various cache cleaning scripts.
Instead of using `GENERIC_CACHE_SCRIPT_PARSER` and defining `parse_args()`
function in each script separately, use `parse_cache_script_args()`.
2017-09-24 04:37:31 -07:00
Harshit Bansal fe80330708 zulip_tools: Add `parse_cache_script_args()`.
This function will replace the repetitive definition of `parse_args()`
in various cache cleaning scripts. Also adds a `--verbose` argument
to the parser.
2017-09-24 04:37:31 -07:00
Harshit Bansal 4e6b68d02f zulip_tools: Change `purge_unused_caches()` API.
Instead of accepting individual arguments, accept `argparse.Namespace`
object as an argument.
2017-09-24 04:37:31 -07:00
Tim Abbott 0a91a5510c install: Fix check for whether update-prod-static is needed.
The previous version seems to be created without update-prod-static.
2017-09-22 19:52:40 -07:00
Tim Abbott 304bd86173 install: Support installing a Zulip server from a Git checkout.
Historically, one has needed to build a release tarball in order to
use/test the Zulip installer, but you could upgrade a Zulip server
from Git.  However, the only reason for that requirement was that we
didn't run `tools/update-prod-static` as part of the install script if
it's required.  A good test for that case is whether we're in a Git
repository, but a better one is to check whether the prod-static
content exists in the tarball paths.

Fixes #3704.
2017-09-22 15:47:42 -07:00
Juliana Bacelar 928dd06cc8 linter: Add lint rule banning 'import os.path' 2017-09-22 10:32:21 -07:00
julianasobreira df2d448f7b python: Add lint rule banning 'from os.path import'.
This enforces our use of a consistent style in how we access Python
modules; "from os.path import dirname" is a particularly popular
abbreviation inconsistent with our style, and so it deserves a lint
rule.

Commit message and error text tweaked by tabbott.

Fixes #6543.
2017-09-22 04:55:38 -07:00
Tim Abbott 901e0258df purge-old-deployments: Fix purging the symlinks etc.
The recent rewrite of purge-old-deployments accidentally attempted to
purge the symlinks, sockets, lock, and other files in the deployment
directory.

The new version has been tested out in production successfully.
2017-09-16 08:49:05 -07:00
Harshit Bansal 26915bc54f scripts: Make `purge-old-deployments` configurable.
Expands `purge-old-deployments` such that now it accepts the threshold
days as argument. Also `clean-unused-caches` script is automatically
run after purging the old deployments so that the orphaned caches
gets automatically cleaned.

Fixes: #5726.
2017-09-16 08:28:57 -07:00
Harshit Bansal 20f062f726 zulip_tools.py: Extract `may_be_perform_purging()` function.
Based on the `dry_run` flag, this function either purges the list
of directories passed to them or prints a listing of the directories
it would have purged/kept_back, had the `dry_run` flag been false.
2017-09-16 08:28:57 -07:00
Tim Abbott 600b164130 setup-apt-repo: Fix failure to rerun properly on provision failure.
Apparently, the refactoring to make this script only run when changes
are present was buggy, in that if `apt-get update` failed, running
provision against wouldn't rerun `apt-get update`, resulting in a
broken state that requires expertise to fix.  This closes that gap, by
using a stamp file to ensure we always successfully update apt before
proceeding.

It doesn't fix existing installations.
2017-09-12 07:08:54 -07:00
Tim Abbott 6d5dac96c6 zulip-puppet-apply: Fix flushing of stdout for Python 3.
Apparently, this UI worked just fine with Python 2, but failed to
actually print usefully with Python 3.
2017-08-27 18:36:39 -07:00
Tim Abbott 1a1df29053 get_recent_deployments: Skip uwsgi socket and friends.
This fixes an exception when running clean-venv-caches in production.
2017-08-27 18:18:53 -07:00
Tim Abbott e38ac00f3c caches: Move cache cleaning helper tools under scripts/lib.
This helps keep the root of scripts/ uncluttered.
2017-08-27 17:59:49 -07:00