Commit Graph

61 Commits

Author SHA1 Message Date
Anders Kaseorg e0a51948d9 script: Add ready-to-run tooling for doing backups.
Based on an initial version by Tim Abbott (#11204).

Fixes #552.
2019-02-11 17:30:37 -08:00
Anders Kaseorg ebad0b7cbf zulip_tools: Get the zulip uid from the owner of DEPLOY_ROOT.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-11 17:00:37 -08:00
Anders Kaseorg 70bfcd3402 zulip_tools: Extract get_deploy_root function.
Modified by tabbott from the original to preserve the implementation;
see https://github.com/zulip/zulip/pull/11295#discussion_r254925032
for why this is correct.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-07 17:09:29 -08:00
Anders Kaseorg e984107966 scripts: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:02:58 -08:00
rht 15763f8545 provision: Include DISTRIB_FAMILY in parse_lsb_release output. 2019-01-07 18:52:09 -08:00
rht 105732ab1f parse_lsb_release: Fix vendor name matching for CentOS. 2019-01-04 14:09:48 -08:00
rht acbb174100 provision: Add RHEL 7 support. 2018-12-18 17:13:56 -08:00
rht b732fe819e provision: Add Fedora support. 2018-12-17 16:23:44 -08:00
Tim Abbott 2558f101af docs: Add documentation for `if False` mypy pattern in scripts.
This should help make it clear what's going on with these scripts.
2018-12-17 11:12:53 -08:00
shubham-padia 29dce7c9b9 upgrade-zulip-from-git: Refactor deploy_options logic to zulip_tools.py.
This a preparatory commit moving the deploy_options logic to
zulip_tools.py so it can be imported and used in upgrade-zulip.
2018-12-16 07:52:47 -08:00
rht e0ec288928 parse_lsb_release: Add CentOS support. 2018-12-11 13:08:26 -08:00
Anders Kaseorg ed0292629b zulip_tools.run: Remove shell=True support.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-11-28 17:48:23 -08:00
Anders Kaseorg 33a4d12101 scripts: Add zulip_tools.overwrite_symlink function to replace ln -nsf.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-11-28 17:24:59 -08:00
Tim Abbott 3e3eb2aa7f scripts: Clarify names of running-as-root assertions.
This should make it more obvious that these functions will exit the
script if the check fails.
2018-11-19 10:58:34 -08:00
Rohitt Vashishtha 767acfa2ac scripts: Add util functions for checking root to zulip_tools. 2018-11-19 10:58:16 -08:00
Anders Kaseorg 510c97d861 scripts: Use shell quoting when displaying commands to be run.
This way, commands with arguments containing whitespace or
metacharacters are unambiguously readable.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-30 22:39:08 -07:00
Anders Kaseorg 09c64f260b scripts/lib/zulip_tools.py: Avoid shelling out for touch.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-25 16:54:46 -07:00
Harshit Bansal f636882e04 build_emoji: Migrate to use `emoji_names.py` file.
This migrates Zulip to use a dramatically better set of names and
aliases for our emoji set, defined in emoji_names.py (which is in turn
manually generated from our hand-curated CSV file).

This should significantly improve the experience of using Zulip's
emoji picker and emoji typeahead for finding what one is looking for.
2018-07-13 21:18:02 +05:30
Tim Abbott 8f9b5633b8 zulip_tools: Fix accessing LSB data on Debian stretch.
Apparently, at least some Debian stretch systems don't have an
/etc/lsb-release, so the optimization that we did in
5d39a0f0fc broke our installer on
Debian.

We fix this, by falling back to calling the lsb_release command on
systems that don't have a faster way to do it.

Fixes part of #9946.
2018-07-13 18:00:38 +05:30
Vishnu Ks 109fa85614 provision: Rename file_hash_updated to file_or_package_hash_updated.
Check for changes in package version as well along
with the files.
2018-06-22 23:40:31 +05:30
Vishnu Ks 7b8e79ae48 provision: Refactor hashing of compilemessages into a function.
This allows it to be reused for other tools.

Edited by tabbott to remove the use of "compilemessages" in variable
names.
2018-06-18 06:55:36 -07:00
Raymond Akornor 5d39a0f0fc scripts: Replace calls to lsb release with our own parsing.
This improves the performance of these operations, by saving a ~50ms
Python process startup.  While not a major performance improvement, it
seems worth it, given how often these commands get run.

Fixes #9571.
2018-05-29 10:57:36 -07:00
Aditya Bansal e14974ff2c scripts: Change use of typing.Text to str. 2018-05-10 14:19:49 -07:00
Harshit Bansal 40958e0824 emoji: Switch to 64px 256 color indexed sprite sheets.
This commit switches our emoji infrastructure to use 256 color indexed
64px spritesheets. Earlier we were using non-indexed 32px spritesheets
which were blurry on high dpi displays. These indexed spritesheets not
only provide a crispier display but are also smaller in size.

This commit also removes the `emoji-datasource` package as a dependency
as all the data is now sourced from individual datasource packages.

Fixes: #7862.
2018-03-14 10:28:45 -07:00
rht 53e37aa511 scripts: Text-wrap long lines exceeding 110. 2017-11-10 16:22:26 -08:00
Harshit Bansal 8c9ea94878 scripts: Fix an issue in `purge-old-deployments` script.
We were not including the real path of the symlinks due to which we
were incorrectly deleting deployments pointed by last/current/next.
2017-10-30 23:09:51 -07:00
Harshit Bansal 1871d6fe1f minor: Remove unnecessary path juggling in `get_recent_deployments()`. 2017-10-29 14:38:20 -07:00
Shekh Ataul d239f77966 refactor: Replace mkdir_p functions with Python 3 builtin.
This didn't exist in Python 2, but it does in Python 3, so we get to
reap the rewards of dropping Python 2 support.

Fixes #7082.
2017-10-25 11:06:11 -07:00
Tim Abbott 2ae2a94444 provision: Stop using shared var/ for caching apt state.
This didn't work at all when one did a `vagrant destroy` and then
`vagrant up`, because the cache state would be preserved even though
the machine is gone.

Fixes #5981.
2017-10-17 21:15:58 -07:00
rht 71188d7b0a scripts: Remove import print_function. 2017-09-29 15:43:30 -07:00
Tim Abbott 358cb40ed1 cache: Add backwards compatibility for emoji cache.
This allows the emoji cache cleaning code to run against old emoji
caches.
2017-09-25 17:06:02 -07:00
Tim Abbott 0f19e501a6 caches: Suppress unnecessary output when cleaning caches.
This should make the cache cleaning process a lot less spammy.
2017-09-25 16:34:03 -07:00
Tim Abbott 86a07baf40 zulip_tools: Skip the lock directory.
This is the one special directory that usually lives in deployments/
and is not a deployment.  Make sure we don't treat it as a deployment.
2017-09-25 15:15:32 -07:00
Harshit Bansal a6caf30ca7 scripts: Fix an issue in `get_recent_deployments()` due to relative paths.
We were checking for whether an item in the deployments directory
represents a directory but were using its relative path which was
causing a false value to be returned for all items irrespective of
their being a directory or not if the script was invoked from some
where other than the deployments directory.
2017-09-25 11:51:24 -07:00
Harshit Bansal 6ff7da04de emoji: Remove `NotoColorEmoji.ttf`.
We no longer use glyphs from `NotoColorEmoji.ttf` so removing this.
2017-09-24 04:51:33 -07:00
Harshit Bansal c8c1c8ef43 emoji: Remove `AndroidEmoji.ttf`. 2017-09-24 04:51:33 -07:00
Harshit Bansal 57161a92a1 scripts: Rearrange the arguments of `purge_unused_caches()`.
This commit re-arranges the arguments of `purge_unused_caches()`
function in order to remain consistent with other similar functions
in the library like `may_be_perform_caching()`.
2017-09-24 04:37:31 -07:00
Harshit Bansal df7ea375c1 scripts: Make default mode of cache-cleaning scripts much less verbose.
Print a detailed report only if `--verbose` flag is specified.

Fixes: #6632.
2017-09-24 04:37:31 -07:00
Harshit Bansal 3e8469a717 zulip_tools: Remove the now unused `GENERIC_CACHE_SCRIPT_PARSER`.
This has been replaced by `parse_cache_script_args()`.
2017-09-24 04:37:31 -07:00
Harshit Bansal fe80330708 zulip_tools: Add `parse_cache_script_args()`.
This function will replace the repetitive definition of `parse_args()`
in various cache cleaning scripts. Also adds a `--verbose` argument
to the parser.
2017-09-24 04:37:31 -07:00
Harshit Bansal 4e6b68d02f zulip_tools: Change `purge_unused_caches()` API.
Instead of accepting individual arguments, accept `argparse.Namespace`
object as an argument.
2017-09-24 04:37:31 -07:00
Harshit Bansal 20f062f726 zulip_tools.py: Extract `may_be_perform_purging()` function.
Based on the `dry_run` flag, this function either purges the list
of directories passed to them or prints a listing of the directories
it would have purged/kept_back, had the `dry_run` flag been false.
2017-09-16 08:28:57 -07:00
Tim Abbott 1a1df29053 get_recent_deployments: Skip uwsgi socket and friends.
This fixes an exception when running clean-venv-caches in production.
2017-08-27 18:18:53 -07:00
Harshit Bansal facb5dbe85 zulip_tools.py: Extract `generate_sha1sum_emoji()`.
Given the path of a zulip installation, it returns a hash corresponding
to the emoji infrastructure of that installation.
2017-08-27 17:51:24 -07:00
Harshit Bansal 36420ab636 zulip_tools.py: Add `purge_caches()` function.
This function can be used for purging unused cache directories.
2017-08-27 17:37:08 -07:00
Harshit Bansal 504abfce63 zulip_tools.py: Add `GENERIC_CACHE_SCRIPT_PARSER`.
This parser will act as a parent parser for all the cache cleaning scripts.
2017-08-23 00:00:34 -07:00
Harshit Bansal 6936bb1ba0 zulip_tools.py: Add `get_caches_to_be_purged()` function.
Given the path of directory containing all the caches, a list of
caches in use and threshold days, this function returns a list
of caches which can be removed safely.
2017-08-22 23:59:45 -07:00
Harshit Bansal e71f92b09e zulip_tools.py: Add `get_threshold_timestamp()` function.
Given `threshold_days` this function returns a timestamp corresponding
to the time before threshold number of days.
2017-08-22 23:57:20 -07:00
Harshit Bansal 8954605726 zulip_tools.py: Add `get_recent_deployments()` function.
This function returns a list of all the deployments directories
which are newer than some threshold number of days including the
`/root/zulip` directory if it exists.
2017-08-22 23:57:20 -07:00
Harshit Bansal 931e4752aa zulip_tools.py: Add `get_environment()` function.
This function can be used to determine the environment in which a
script is being executed.
2017-08-22 23:57:20 -07:00