Commit Graph

430 Commits

Author SHA1 Message Date
Shekh Ataul d239f77966 refactor: Replace mkdir_p functions with Python 3 builtin.
This didn't exist in Python 2, but it does in Python 3, so we get to
reap the rewards of dropping Python 2 support.

Fixes #7082.
2017-10-25 11:06:11 -07:00
Tim Abbott 6b1eb647e4 nginx: Fix bugs in new nginx configuration checks. 2017-10-24 14:29:36 -07:00
Tim Abbott 730c77c7df docs: Document scripts/setup/generate-self-signed-certs.
And more generally clean up our non-LetsEncrypt SSL docs.

This should make it a bit easier to setup a Zulip server.
2017-10-24 13:48:14 -07:00
Tim Abbott 1b653409f4 install: Provide a nicer error message for bad nginx configuration.
This also covers missing SSL configuration errors nicely.
2017-10-24 13:39:39 -07:00
rht 5755f475f4 upgrade: Add "must be run as root" check.
The logic and text is based on scripts/lib/upgrade-zulip.
2017-10-23 23:01:54 -07:00
rht dcb46afa90 upgrade: Remove an extraneous `sudo` prefix.
Because scripts/upgrade-zulip-from-git must be run from root anyway.
2017-10-23 23:01:46 -07:00
rht c8090aafa4 lint: Remove several unused imports. 2017-10-18 10:56:11 -07:00
rht a603a4f9f5 Remove `from __future__ import absolute_import`.
Except in:
- docs/writing-bots-guide.md, because bots are supposed to be Python 2
  compatible
- puppet/zulip_ops/files/zulip-ec2-configure-interfaces, because this
  script is still on python2.7
- tools/lint
- tools/linter_lib
- tools/lister.py

For the latter two, because they might be yanked away to a separate repo
for general use with other FLOSS projects.
2017-10-17 22:59:42 -07:00
Tim Abbott 2ae2a94444 provision: Stop using shared var/ for caching apt state.
This didn't work at all when one did a `vagrant destroy` and then
`vagrant up`, because the cache state would be preserved even though
the machine is gone.

Fixes #5981.
2017-10-17 21:15:58 -07:00
rht c77c2f084c
upgrade-zulip-from-git: Fix rerun script and script description. 2017-10-18 04:41:22 +02:00
rht 9ab54e5bd7 scripts/lib/install: Add flag to specify key settings.
This should make it easier to script the installation process, and
also conveniently are the options one would want for the --certbot
option.

Significantly modified by tabbott to have a sane right interface,
include --help, and avoid printing all the `set -x` garbage before the
usage notices.
2017-10-03 16:56:45 -07:00
Tim Abbott 8c8613c3c7 install: Add comments documenting installer wrapper. 2017-10-01 15:48:36 -07:00
Alexander Trost 6a7d34316c Add scripts to be used in docker-entrypoint.sh.
Based on #450, with commits
restructured by Rein Zustand.
Tweaks by Rein Zustand:
- Replace configure-cert with generate-self-signed-certs
- `mv scripts/lib/create-zulip-admin.sh scripts/lib/create-zulip-admin`
2017-09-30 09:55:48 -07:00
rht 71188d7b0a scripts: Remove import print_function. 2017-09-29 15:43:30 -07:00
Alexander Trost dea507f4b8 Add script to create admin from CLI. 2017-09-27 20:15:41 -07:00
rht b937e1fede Remove the rest of `import six` outside zerver.lib.
We no longer need the six module, now that we're Python 3 only.
2017-09-27 17:09:15 -07:00
Tim Abbott 358cb40ed1 cache: Add backwards compatibility for emoji cache.
This allows the emoji cache cleaning code to run against old emoji
caches.
2017-09-25 17:06:02 -07:00
Tim Abbott db982672b1 caches: Don't depend on yarn.lock existing.
This allows our cache cleaning code to run on servers that have copies
of Zulip 1.6 and older around.
2017-09-25 17:05:45 -07:00
Tim Abbott 0f19e501a6 caches: Suppress unnecessary output when cleaning caches.
This should make the cache cleaning process a lot less spammy.
2017-09-25 16:34:03 -07:00
Tim Abbott 703b87c116 caches: Move clean-unused-caches to scripts/lib/. 2017-09-25 16:34:03 -07:00
Tim Abbott 86a07baf40 zulip_tools: Skip the lock directory.
This is the one special directory that usually lives in deployments/
and is not a deployment.  Make sure we don't treat it as a deployment.
2017-09-25 15:15:32 -07:00
Harshit Bansal 90f6ffa23d purge-old-deployments: Remove an unnecessary check.
Deployments whose name is not in the format of a timestamp are
always included in the `recent_deployments` and are not deleted,
hence we don't need to check for them explicitly.
2017-09-25 11:51:24 -07:00
Harshit Bansal a6caf30ca7 scripts: Fix an issue in `get_recent_deployments()` due to relative paths.
We were checking for whether an item in the deployments directory
represents a directory but were using its relative path which was
causing a false value to be returned for all items irrespective of
their being a directory or not if the script was invoked from some
where other than the deployments directory.
2017-09-25 11:51:24 -07:00
Harshit Bansal 6ff7da04de emoji: Remove `NotoColorEmoji.ttf`.
We no longer use glyphs from `NotoColorEmoji.ttf` so removing this.
2017-09-24 04:51:33 -07:00
Harshit Bansal c8c1c8ef43 emoji: Remove `AndroidEmoji.ttf`. 2017-09-24 04:51:33 -07:00
Tim Abbott 7e7b80f5e8 purge_old_deployments: Fix running from any path. 2017-09-24 04:42:14 -07:00
Harshit Bansal 57161a92a1 scripts: Rearrange the arguments of `purge_unused_caches()`.
This commit re-arranges the arguments of `purge_unused_caches()`
function in order to remain consistent with other similar functions
in the library like `may_be_perform_caching()`.
2017-09-24 04:37:31 -07:00
Harshit Bansal df7ea375c1 scripts: Make default mode of cache-cleaning scripts much less verbose.
Print a detailed report only if `--verbose` flag is specified.

Fixes: #6632.
2017-09-24 04:37:31 -07:00
Harshit Bansal 480ae626fb scripts: Add `--verbose` flag to purge-old-deployments script. 2017-09-24 04:37:31 -07:00
Harshit Bansal 3e8469a717 zulip_tools: Remove the now unused `GENERIC_CACHE_SCRIPT_PARSER`.
This has been replaced by `parse_cache_script_args()`.
2017-09-24 04:37:31 -07:00
Harshit Bansal 7f752f0942 scripts: Use `parse_cache_script_args()` in various cache cleaning scripts.
Instead of using `GENERIC_CACHE_SCRIPT_PARSER` and defining `parse_args()`
function in each script separately, use `parse_cache_script_args()`.
2017-09-24 04:37:31 -07:00
Harshit Bansal fe80330708 zulip_tools: Add `parse_cache_script_args()`.
This function will replace the repetitive definition of `parse_args()`
in various cache cleaning scripts. Also adds a `--verbose` argument
to the parser.
2017-09-24 04:37:31 -07:00
Harshit Bansal 4e6b68d02f zulip_tools: Change `purge_unused_caches()` API.
Instead of accepting individual arguments, accept `argparse.Namespace`
object as an argument.
2017-09-24 04:37:31 -07:00
Tim Abbott 0a91a5510c install: Fix check for whether update-prod-static is needed.
The previous version seems to be created without update-prod-static.
2017-09-22 19:52:40 -07:00
Tim Abbott 304bd86173 install: Support installing a Zulip server from a Git checkout.
Historically, one has needed to build a release tarball in order to
use/test the Zulip installer, but you could upgrade a Zulip server
from Git.  However, the only reason for that requirement was that we
didn't run `tools/update-prod-static` as part of the install script if
it's required.  A good test for that case is whether we're in a Git
repository, but a better one is to check whether the prod-static
content exists in the tarball paths.

Fixes #3704.
2017-09-22 15:47:42 -07:00
Juliana Bacelar 928dd06cc8 linter: Add lint rule banning 'import os.path' 2017-09-22 10:32:21 -07:00
julianasobreira df2d448f7b python: Add lint rule banning 'from os.path import'.
This enforces our use of a consistent style in how we access Python
modules; "from os.path import dirname" is a particularly popular
abbreviation inconsistent with our style, and so it deserves a lint
rule.

Commit message and error text tweaked by tabbott.

Fixes #6543.
2017-09-22 04:55:38 -07:00
Tim Abbott 901e0258df purge-old-deployments: Fix purging the symlinks etc.
The recent rewrite of purge-old-deployments accidentally attempted to
purge the symlinks, sockets, lock, and other files in the deployment
directory.

The new version has been tested out in production successfully.
2017-09-16 08:49:05 -07:00
Harshit Bansal 26915bc54f scripts: Make `purge-old-deployments` configurable.
Expands `purge-old-deployments` such that now it accepts the threshold
days as argument. Also `clean-unused-caches` script is automatically
run after purging the old deployments so that the orphaned caches
gets automatically cleaned.

Fixes: #5726.
2017-09-16 08:28:57 -07:00
Harshit Bansal 20f062f726 zulip_tools.py: Extract `may_be_perform_purging()` function.
Based on the `dry_run` flag, this function either purges the list
of directories passed to them or prints a listing of the directories
it would have purged/kept_back, had the `dry_run` flag been false.
2017-09-16 08:28:57 -07:00
Tim Abbott 600b164130 setup-apt-repo: Fix failure to rerun properly on provision failure.
Apparently, the refactoring to make this script only run when changes
are present was buggy, in that if `apt-get update` failed, running
provision against wouldn't rerun `apt-get update`, resulting in a
broken state that requires expertise to fix.  This closes that gap, by
using a stamp file to ensure we always successfully update apt before
proceeding.

It doesn't fix existing installations.
2017-09-12 07:08:54 -07:00
Tim Abbott 6d5dac96c6 zulip-puppet-apply: Fix flushing of stdout for Python 3.
Apparently, this UI worked just fine with Python 2, but failed to
actually print usefully with Python 3.
2017-08-27 18:36:39 -07:00
Tim Abbott 1a1df29053 get_recent_deployments: Skip uwsgi socket and friends.
This fixes an exception when running clean-venv-caches in production.
2017-08-27 18:18:53 -07:00
Tim Abbott e38ac00f3c caches: Move cache cleaning helper tools under scripts/lib.
This helps keep the root of scripts/ uncluttered.
2017-08-27 17:59:49 -07:00
Harshit Bansal f90f9c7404 scripts: Add `clean-unused-caches` script.
This script can be used for running all the three cache cleaning scripts
in one go.
2017-08-27 17:51:24 -07:00
Harshit Bansal e5361b4403 clean_emoji_cache: Expand `clean-emoji-cache`.
Expands `clean-emoji-cache` so that it can be used in production
environment as well. Also moves it to `scripts/` from `tools/`.
2017-08-27 17:51:24 -07:00
Harshit Bansal facb5dbe85 zulip_tools.py: Extract `generate_sha1sum_emoji()`.
Given the path of a zulip installation, it returns a hash corresponding
to the emoji infrastructure of that installation.
2017-08-27 17:51:24 -07:00
Harshit Bansal 948cf54ee3 clean-npm-cache: Expand `clean-npm-cache`.
Expands `clean-npm-cache` so that it can be used in production
environment as well. Also moves it to `scripts/` from `tools/`.
2017-08-27 17:51:24 -07:00
Harshit Bansal 8e41bbe2b0 node_cache.py: Modify `generate_sha1sum_node_modules()`.
Modify `generate_sha1sum_node_modules()` such that it can calculate
the hash for a particular installation.

Tweaked by tabbott to use os.path.realpath in the setup_dir
calculation, to ensure it's consistent.
2017-08-27 17:51:24 -07:00
Harshit Bansal 8aadbc258d clean-venv-cache: Don't remove the currently active cache in dev.
In dev always include the currently active cache in order not to break
current installation in case dependencies are updated with bumping the
provision version.
2017-08-27 17:51:24 -07:00