Commit Graph

243 Commits

Author SHA1 Message Date
rht 01eb22abd9 refactor: Remove all the remaining six.moves import. 2017-11-07 10:51:44 -08:00
rht 9c7d5812ce refactor: Remove six.moves.urllib.parse import. 2017-11-07 10:51:44 -08:00
rht ccf2792c1c refactor: Remove six.moves.configparser import. 2017-11-07 10:51:44 -08:00
neiljp (Neil Pilgrim) 221be326a9 mypy: Set type parameters for IO to Any in scripts/lib/node_cache.py. 2017-11-04 19:47:44 -07:00
Harshit Bansal 48662a79a5 clean-venv-cache: Fix an exception with unusual deployment dirs. 2017-10-31 00:01:09 -07:00
Harshit Bansal 8c9ea94878 scripts: Fix an issue in `purge-old-deployments` script.
We were not including the real path of the symlinks due to which we
were incorrectly deleting deployments pointed by last/current/next.
2017-10-30 23:09:51 -07:00
Harshit Bansal 1871d6fe1f minor: Remove unnecessary path juggling in `get_recent_deployments()`. 2017-10-29 14:38:20 -07:00
Tim Abbott 11ab545f3b install: Set the locale so our dependencies can install.
Many pip packages don't install properly without a US locale.
2017-10-29 11:49:08 -07:00
rht 8b6b4e043f install: Add option to get certs via certbot.
While this doesn't quite complete our plans for certbot support (it's
not documented, etc.), this is a great stride forward.
2017-10-27 17:19:34 -07:00
Shekh Ataul d239f77966 refactor: Replace mkdir_p functions with Python 3 builtin.
This didn't exist in Python 2, but it does in Python 3, so we get to
reap the rewards of dropping Python 2 support.

Fixes #7082.
2017-10-25 11:06:11 -07:00
Tim Abbott 6b1eb647e4 nginx: Fix bugs in new nginx configuration checks. 2017-10-24 14:29:36 -07:00
Tim Abbott 730c77c7df docs: Document scripts/setup/generate-self-signed-certs.
And more generally clean up our non-LetsEncrypt SSL docs.

This should make it a bit easier to setup a Zulip server.
2017-10-24 13:48:14 -07:00
Tim Abbott 1b653409f4 install: Provide a nicer error message for bad nginx configuration.
This also covers missing SSL configuration errors nicely.
2017-10-24 13:39:39 -07:00
rht c8090aafa4 lint: Remove several unused imports. 2017-10-18 10:56:11 -07:00
rht a603a4f9f5 Remove `from __future__ import absolute_import`.
Except in:
- docs/writing-bots-guide.md, because bots are supposed to be Python 2
  compatible
- puppet/zulip_ops/files/zulip-ec2-configure-interfaces, because this
  script is still on python2.7
- tools/lint
- tools/linter_lib
- tools/lister.py

For the latter two, because they might be yanked away to a separate repo
for general use with other FLOSS projects.
2017-10-17 22:59:42 -07:00
Tim Abbott 2ae2a94444 provision: Stop using shared var/ for caching apt state.
This didn't work at all when one did a `vagrant destroy` and then
`vagrant up`, because the cache state would be preserved even though
the machine is gone.

Fixes #5981.
2017-10-17 21:15:58 -07:00
rht 9ab54e5bd7 scripts/lib/install: Add flag to specify key settings.
This should make it easier to script the installation process, and
also conveniently are the options one would want for the --certbot
option.

Significantly modified by tabbott to have a sane right interface,
include --help, and avoid printing all the `set -x` garbage before the
usage notices.
2017-10-03 16:56:45 -07:00
Alexander Trost 6a7d34316c Add scripts to be used in docker-entrypoint.sh.
Based on #450, with commits
restructured by Rein Zustand.
Tweaks by Rein Zustand:
- Replace configure-cert with generate-self-signed-certs
- `mv scripts/lib/create-zulip-admin.sh scripts/lib/create-zulip-admin`
2017-09-30 09:55:48 -07:00
rht 71188d7b0a scripts: Remove import print_function. 2017-09-29 15:43:30 -07:00
Alexander Trost dea507f4b8 Add script to create admin from CLI. 2017-09-27 20:15:41 -07:00
Tim Abbott 358cb40ed1 cache: Add backwards compatibility for emoji cache.
This allows the emoji cache cleaning code to run against old emoji
caches.
2017-09-25 17:06:02 -07:00
Tim Abbott db982672b1 caches: Don't depend on yarn.lock existing.
This allows our cache cleaning code to run on servers that have copies
of Zulip 1.6 and older around.
2017-09-25 17:05:45 -07:00
Tim Abbott 0f19e501a6 caches: Suppress unnecessary output when cleaning caches.
This should make the cache cleaning process a lot less spammy.
2017-09-25 16:34:03 -07:00
Tim Abbott 703b87c116 caches: Move clean-unused-caches to scripts/lib/. 2017-09-25 16:34:03 -07:00
Tim Abbott 86a07baf40 zulip_tools: Skip the lock directory.
This is the one special directory that usually lives in deployments/
and is not a deployment.  Make sure we don't treat it as a deployment.
2017-09-25 15:15:32 -07:00
Harshit Bansal a6caf30ca7 scripts: Fix an issue in `get_recent_deployments()` due to relative paths.
We were checking for whether an item in the deployments directory
represents a directory but were using its relative path which was
causing a false value to be returned for all items irrespective of
their being a directory or not if the script was invoked from some
where other than the deployments directory.
2017-09-25 11:51:24 -07:00
Harshit Bansal 6ff7da04de emoji: Remove `NotoColorEmoji.ttf`.
We no longer use glyphs from `NotoColorEmoji.ttf` so removing this.
2017-09-24 04:51:33 -07:00
Harshit Bansal c8c1c8ef43 emoji: Remove `AndroidEmoji.ttf`. 2017-09-24 04:51:33 -07:00
Harshit Bansal 57161a92a1 scripts: Rearrange the arguments of `purge_unused_caches()`.
This commit re-arranges the arguments of `purge_unused_caches()`
function in order to remain consistent with other similar functions
in the library like `may_be_perform_caching()`.
2017-09-24 04:37:31 -07:00
Harshit Bansal df7ea375c1 scripts: Make default mode of cache-cleaning scripts much less verbose.
Print a detailed report only if `--verbose` flag is specified.

Fixes: #6632.
2017-09-24 04:37:31 -07:00
Harshit Bansal 3e8469a717 zulip_tools: Remove the now unused `GENERIC_CACHE_SCRIPT_PARSER`.
This has been replaced by `parse_cache_script_args()`.
2017-09-24 04:37:31 -07:00
Harshit Bansal 7f752f0942 scripts: Use `parse_cache_script_args()` in various cache cleaning scripts.
Instead of using `GENERIC_CACHE_SCRIPT_PARSER` and defining `parse_args()`
function in each script separately, use `parse_cache_script_args()`.
2017-09-24 04:37:31 -07:00
Harshit Bansal fe80330708 zulip_tools: Add `parse_cache_script_args()`.
This function will replace the repetitive definition of `parse_args()`
in various cache cleaning scripts. Also adds a `--verbose` argument
to the parser.
2017-09-24 04:37:31 -07:00
Harshit Bansal 4e6b68d02f zulip_tools: Change `purge_unused_caches()` API.
Instead of accepting individual arguments, accept `argparse.Namespace`
object as an argument.
2017-09-24 04:37:31 -07:00
Tim Abbott 0a91a5510c install: Fix check for whether update-prod-static is needed.
The previous version seems to be created without update-prod-static.
2017-09-22 19:52:40 -07:00
Tim Abbott 304bd86173 install: Support installing a Zulip server from a Git checkout.
Historically, one has needed to build a release tarball in order to
use/test the Zulip installer, but you could upgrade a Zulip server
from Git.  However, the only reason for that requirement was that we
didn't run `tools/update-prod-static` as part of the install script if
it's required.  A good test for that case is whether we're in a Git
repository, but a better one is to check whether the prod-static
content exists in the tarball paths.

Fixes #3704.
2017-09-22 15:47:42 -07:00
Juliana Bacelar 928dd06cc8 linter: Add lint rule banning 'import os.path' 2017-09-22 10:32:21 -07:00
julianasobreira df2d448f7b python: Add lint rule banning 'from os.path import'.
This enforces our use of a consistent style in how we access Python
modules; "from os.path import dirname" is a particularly popular
abbreviation inconsistent with our style, and so it deserves a lint
rule.

Commit message and error text tweaked by tabbott.

Fixes #6543.
2017-09-22 04:55:38 -07:00
Harshit Bansal 20f062f726 zulip_tools.py: Extract `may_be_perform_purging()` function.
Based on the `dry_run` flag, this function either purges the list
of directories passed to them or prints a listing of the directories
it would have purged/kept_back, had the `dry_run` flag been false.
2017-09-16 08:28:57 -07:00
Tim Abbott 600b164130 setup-apt-repo: Fix failure to rerun properly on provision failure.
Apparently, the refactoring to make this script only run when changes
are present was buggy, in that if `apt-get update` failed, running
provision against wouldn't rerun `apt-get update`, resulting in a
broken state that requires expertise to fix.  This closes that gap, by
using a stamp file to ensure we always successfully update apt before
proceeding.

It doesn't fix existing installations.
2017-09-12 07:08:54 -07:00
Tim Abbott 1a1df29053 get_recent_deployments: Skip uwsgi socket and friends.
This fixes an exception when running clean-venv-caches in production.
2017-08-27 18:18:53 -07:00
Tim Abbott e38ac00f3c caches: Move cache cleaning helper tools under scripts/lib.
This helps keep the root of scripts/ uncluttered.
2017-08-27 17:59:49 -07:00
Harshit Bansal facb5dbe85 zulip_tools.py: Extract `generate_sha1sum_emoji()`.
Given the path of a zulip installation, it returns a hash corresponding
to the emoji infrastructure of that installation.
2017-08-27 17:51:24 -07:00
Harshit Bansal 8e41bbe2b0 node_cache.py: Modify `generate_sha1sum_node_modules()`.
Modify `generate_sha1sum_node_modules()` such that it can calculate
the hash for a particular installation.

Tweaked by tabbott to use os.path.realpath in the setup_dir
calculation, to ensure it's consistent.
2017-08-27 17:51:24 -07:00
Harshit Bansal 36420ab636 zulip_tools.py: Add `purge_caches()` function.
This function can be used for purging unused cache directories.
2017-08-27 17:37:08 -07:00
Tim Abbott fa97dd1408 setup_path_on_import: Drop Python 2 support. 2017-08-23 19:21:50 -07:00
Tim Abbott 2424819749 install: Move upstart checks a bit earlier.
This should make it much more likely that users see this before
waiting a long time for other things to happen, since the `apt-get
dist-upgrade` step is really slow.  We can't move further to the top,
since this requires `lsb_release` to be installed.
2017-08-23 14:55:01 -07:00
Harshit Bansal 504abfce63 zulip_tools.py: Add `GENERIC_CACHE_SCRIPT_PARSER`.
This parser will act as a parent parser for all the cache cleaning scripts.
2017-08-23 00:00:34 -07:00
Harshit Bansal 6936bb1ba0 zulip_tools.py: Add `get_caches_to_be_purged()` function.
Given the path of directory containing all the caches, a list of
caches in use and threshold days, this function returns a list
of caches which can be removed safely.
2017-08-22 23:59:45 -07:00
Harshit Bansal e71f92b09e zulip_tools.py: Add `get_threshold_timestamp()` function.
Given `threshold_days` this function returns a timestamp corresponding
to the time before threshold number of days.
2017-08-22 23:57:20 -07:00