Commit Graph

33 Commits

Author SHA1 Message Date
Riken Shah c5580607a7 purge-old-deployments: Use the `clean_unused_caches.main` function.
We currently run the `clean_unused_caches.py` as a
script to clean the unused caches.

This commit replaces that with
`clean_unused_caches.main` function as it would be
faster.
2021-06-12 07:28:16 -07:00
Riken Shah 4f54e15993 refactor: Convert `clean-unused-caches` to`clean_unused_caches.py`.
We convert the `clean-unused-caches` script to a
python file so we can run it in provision by importing it
instead of running the script, hence saving some time.
2021-06-12 07:28:16 -07:00
Anders Kaseorg 6766a3f780 purge-old-deployments: Check /srv/zulip.git existence before pruning it.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-13 20:36:27 -07:00
Anders Kaseorg 3f83b843c2 upgrade-zulip-from-git: Create deployment directories with git worktree.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-13 13:42:05 -07:00
Anders Kaseorg 11741543da python: Reformat with Black, except quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 5028c081cb python: Merge concatenated string literals that Black would uglify.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 3c5b39da9c python: Elide nargs for argparse flag arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg 1ded51aa9d python: Replace list literal concatenation with * unpacking.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:15:41 -07:00
Anders Kaseorg a5dbab8fb0 python: Remove redundant dest for argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:04:10 -07:00
Anders Kaseorg 365fe0b3d5 python: Sort imports with isort.
Fixes #2665.

Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.

Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start.  I expect this change will increase pressure for us to split
those files, which isn't a bad thing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:45:32 -07:00
Anders Kaseorg 5901e7ba7e python: Convert function type annotations to Python 3 style.
Generated by com2ann (slightly patched to avoid also converting
assignment type annotations, which require Python 3.6), followed by
some manual whitespace adjustment, and six fixes for runtime issues:

-    def __init__(self, token: Token, parent: Optional[Node]) -> None:
+    def __init__(self, token: Token, parent: "Optional[Node]") -> None:

-def main(options: argparse.Namespace) -> NoReturn:
+def main(options: argparse.Namespace) -> "NoReturn":

-def fetch_request(url: str, callback: Any, **kwargs: Any) -> Generator[Callable[..., Any], Any, None]:
+def fetch_request(url: str, callback: Any, **kwargs: Any) -> "Generator[Callable[..., Any], Any, None]":

-def assert_server_running(server: subprocess.Popen[bytes], log_file: Optional[str]) -> None:
+def assert_server_running(server: "subprocess.Popen[bytes]", log_file: Optional[str]) -> None:

-def server_is_up(server: subprocess.Popen[bytes], log_file: Optional[str]) -> bool:
+def server_is_up(server: "subprocess.Popen[bytes]", log_file: Optional[str]) -> bool:

-    method_kwarg_pairs: List[FuncKwargPair],
+    method_kwarg_pairs: "List[FuncKwargPair]",

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-18 20:42:48 -07:00
Anders Kaseorg c734bbd95d python: Modernize legacy Python 2 syntax with pyupgrade.
Generated by `pyupgrade --py3-plus --keep-percent-format` on all our
Python code except `zthumbor` and `zulip-ec2-configure-interfaces`,
followed by manual indentation fixes.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-09 16:43:22 -07:00
ab1nash 71f0fecda7 scripts: Clean up output from 'clean_unused_caches'.
The output log from running clean_unused_caches was too verbose as
part of the `upgrade-zulip` overall output.  While this output is
potentially helpful when running it directly for debugging, it's
certainly redundant for the main production use case.

So a new flag --no-print-headers is introduced.  It suppresses the
header outputs for the subtools.

Fixes #13214.
2019-09-30 10:51:00 -07:00
Wyatt Hoodes a109508e34 typing: Remove now-unnecessary conditional import.
As a result of dropping support for trusty, we can remove our old
pattern of putting `if False` before importing the typing module,
which was essential for Python 3.4 support, but not required and maybe
harmful on newer versions.

cron_file_helper
check_rabbitmq_consumers
hash_reqs
check_zephyr_mirror
check_personal_zephyr_mirrors
check_cron_file
zulip_tools
check_postgres_replication_lag
api_test_helpers
purge-old-deployments
setup_venv
node_cache
clean_venv_cache
clean_node_cache
clean_emoji_cache
pg_backup_and_purge
restore-backup
generate_secrets
zulip-ec2-configure-interfaces
diagnose
check_user_zephyr_mirror_liveness
2019-07-29 15:18:22 -07:00
Anders Kaseorg e984107966 scripts: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:02:58 -08:00
Tim Abbott 2558f101af docs: Add documentation for `if False` mypy pattern in scripts.
This should help make it clear what's going on with these scripts.
2018-12-17 11:12:53 -08:00
Aditya Bansal cb9d8f6d48 scripts: Change use of typing.Text to str. 2018-05-14 05:16:22 +05:30
Tim Abbott 0f19e501a6 caches: Suppress unnecessary output when cleaning caches.
This should make the cache cleaning process a lot less spammy.
2017-09-25 16:34:03 -07:00
Tim Abbott 703b87c116 caches: Move clean-unused-caches to scripts/lib/. 2017-09-25 16:34:03 -07:00
Tim Abbott 86a07baf40 zulip_tools: Skip the lock directory.
This is the one special directory that usually lives in deployments/
and is not a deployment.  Make sure we don't treat it as a deployment.
2017-09-25 15:15:32 -07:00
Harshit Bansal 90f6ffa23d purge-old-deployments: Remove an unnecessary check.
Deployments whose name is not in the format of a timestamp are
always included in the `recent_deployments` and are not deleted,
hence we don't need to check for them explicitly.
2017-09-25 11:51:24 -07:00
Tim Abbott 7e7b80f5e8 purge_old_deployments: Fix running from any path. 2017-09-24 04:42:14 -07:00
Harshit Bansal df7ea375c1 scripts: Make default mode of cache-cleaning scripts much less verbose.
Print a detailed report only if `--verbose` flag is specified.

Fixes: #6632.
2017-09-24 04:37:31 -07:00
Harshit Bansal 480ae626fb scripts: Add `--verbose` flag to purge-old-deployments script. 2017-09-24 04:37:31 -07:00
Tim Abbott 901e0258df purge-old-deployments: Fix purging the symlinks etc.
The recent rewrite of purge-old-deployments accidentally attempted to
purge the symlinks, sockets, lock, and other files in the deployment
directory.

The new version has been tested out in production successfully.
2017-09-16 08:49:05 -07:00
Harshit Bansal 26915bc54f scripts: Make `purge-old-deployments` configurable.
Expands `purge-old-deployments` such that now it accepts the threshold
days as argument. Also `clean-unused-caches` script is automatically
run after purging the old deployments so that the orphaned caches
gets automatically cleaned.

Fixes: #5726.
2017-09-16 08:28:57 -07:00
Greg Price a099e698e2 py3: Switch almost all shebang lines to use `python3`.
This causes `upgrade-zulip-from-git`, as well as a no-option run of
`tools/build-release-tarball`, to produce a Zulip install running
Python 3, rather than Python 2.  In particular this means that the
virtualenv we create, in which all application code runs, is Python 3.

One shebang line, on `zulip-ec2-configure-interfaces`, explicitly
keeps Python 2, and at least one external ops script, `wal-e`, also
still runs on Python 2.  See discussion on the respective previous
commits that made those explicit.  There may also be some other
third-party scripts we use, outside of this source tree and running
outside our virtualenv, that still run on Python 2.
2017-08-16 17:54:43 -07:00
Taranjeet Singh d606b95242 zulip_tools.py: Move zulip_tools.py in scripts/lib.
This commit moves zulip_tools.py as part of cleaning the root directory
and organizing proejct into better directory structure.
2016-08-15 16:44:50 -07:00
Tim Abbott 91b1521578 purge-old-deployments: Avoid purging last/next deployments. 2016-08-04 17:02:48 -07:00
Eklavya Sharma 149938d468 Change shebangs from python2.7 to python. 2016-05-29 05:03:08 -07:00
Steven Oud d5435fad1d Consistently use /usr/bin/env python2.7 in shebangs and commands. 2015-10-21 22:58:21 +00:00
Tim Abbott bc19be977a purge-old-deployments: Never purge the currently running deployment.
(imported from commit edccfc908f50a66e5c214bca8a5418625ed64bf0)
2013-11-04 13:22:41 -05:00
Tim Abbott adc5771062 Rename purge-deployments to purge-old-deployments.
Since this is a user-facing script, we should make it more clear about
what it does.

(imported from commit eecaed8462a92fe7c688df5c32fea2e088b25058)
2013-11-04 13:22:41 -05:00