Commit Graph

534 Commits

Author SHA1 Message Date
Greg Price 08135e3666 install: Don't try to chown a self-signed cert to zulip.
The zulip user has no need to see this file; it's used by nginx.
And when we set up the cert early in install, there's no zulip user
yet anyway, so this fails.
2018-02-08 17:08:16 -08:00
Greg Price f68c485a86 install: Clean up some shell style: use here-docs, and compact getopt. 2018-02-08 17:05:13 -08:00
Greg Price 8f4b7f6e1e install: On --express, check up front for --hostname and --email. 2018-02-08 16:34:49 -08:00
Greg Price 31ec9a4627 install: Implement --self-signed-cert earlier.
Otherwise when puppet tries to install and start nginx, that can
give an error.
2018-02-08 16:34:49 -08:00
Greg Price 70d203c1c8 install: Add experimental option to go straight through more install steps.
We'll make this the normal behavior soon, once we're satisfied with
our arrangements for sending the admin straight to realm creation and
using the app without configuring email.  The instructions in the docs
will also have to change accordingly, of course.
2018-02-05 12:59:12 -08:00
rht 2b19f17fc9 scripts: Remove u prefix from strings. 2018-02-05 12:11:33 -08:00
Umair Khan 8b33c46165 i18n: Add script to process mobile locales.
This script iterates over all the mobile.json resources and creates a
single file at static/locale/mobile_info.json which contains total and
not-translated strings information against each language. After doing
this, it deletes all the mobile i18n resources downloaded by
tools/sync-translations because we neither want to check them in our
repository nor we want to make our repository dirty.
2018-01-26 16:55:40 -08:00
Greg Price 0ba3fc3c2f install: Slightly tighten up CLI parsing.
This causes us to give an error if you pass the installer any
positional arguments, e.g. with `--`.  There's no reason you'd want
to do this, but I accidentally did it by passing an extra `--` to
the `test-install/install` wrapper and spent a few minutes on
confused debugging.
2018-01-24 14:34:30 -08:00
Greg Price 0f836f51a5 generate-self-signed-cert: Fix a mostly-harmless bug.
Thanks to the magic of `set -x`, I noticed this:
```
+ cat
++ ssl-cert
/tmp/src/zulip-server/scripts/setup/generate-self-signed-cert: line 49: ssl-cert: command not found
+ apt-get install -y openssl
[...]
```

In other words, we were trying to run `ssl-cert` -- the name of a
Debian package I meant to refer to in a comment inside the templated
temporary config file for `openssl req` -- as if it were a command.
It wasn't, hence the error.

Because `set -e` has loopholes like a sieve, this didn't cause the
script to exit, just produced this funny output and presumably caused
the config file's comment to be missing a word.  In principle, it
could do something surprising if for some reason there were a command
named `ssl-cert` on PATH.

Fix it.
2018-01-23 18:08:52 -08:00
Greg Price f26b34405e install --self-signed-cert: Generate our own, rather than use system's.
This gives us just one way of adopting a self-signed cert, rather than
one script which would generate a new one and an option to another
which would symlink to the system's snakeoil cert.  Now those two
codepaths converge, and do the same thing.

The small advantage of generating our own over the alternative is that
it lets us set the name in the cert to EXTERNAL_HOST, rather than the
system's hostname as embedded in the system snakeoil certs.  Not a big
deal, but might make things go slightly smoother if some browsers are
lenient (in a way that they probably shouldn't be.)
2018-01-23 18:08:52 -08:00
Greg Price d76c2d77f4 install: Tighten an SSL-cert help message.
The option's name now explains for itself some of what we'd
had in prose.
2018-01-23 18:08:52 -08:00
Greg Price 841a5f3152 install: Say --self-signed-cert instead of --snakeoil-cert.
Less evocative, but requires less explanation to document because
it's a well-known term on the Internet.
2018-01-23 18:08:52 -08:00
Greg Price c27f36a82a install: Rename generate-self-signed-cert.
This script generates one certificate.
2018-01-23 18:08:52 -08:00
Greg Price d258e48f3d install: Replace our generate-self-signed-certs script.
Take the core of the logic from how Debian generates the system's
/etc/ssl/certs/ssl-cert-snakeoil.pem ; that gives me more confidence
in the various config choices, and it also demonstrates a much cleaner
way to use the `openssl` tool.  Also replace the outer shell logic for
CLI and logging with a cleaner version.
2018-01-23 18:08:52 -08:00
Greg Price 2a59b2d2ac install: Work around a bug in the (our) Debian package for camo.
Before this fix, the installer has an extremely annoying bug where
when run inside a container with `lxc-attach`, when the installer
finishes, the `lxc-attach` just hangs and doesn't respond even to
C-c or C-z.  The only way to get the terminal back is to root around
from some other terminal to find the PID and kill it; then run
something like `stty sane` to fix the messed-up terminal settings
left behind.

After bisecting pieces of the install script to locate which step
was causing the issue, it comes down to the `service camo restart`.
The comment here indicates that we knew about an annoying bug here
years ago, and just swept it under the rug by skipping this step
when in Travis. >_<

The issue can be reproduced by running simply `service camo restart`
under `lxc-attach` instead of the installer; or `service camo start`,
following a `service camo stop`.  If `lxc-attach` is used to get an
interactive shell, these commands appear to work fine; but then when
that shell exits, the same hang appears.  So, when we start camo
we're evidently leaving some kind of mess that entangles the daemon
with our shell.

Looking at the camo initscript where it starts the daemon, there's
not much code, and one flag jumps out as suspicious:

  start-stop-daemon --start --quiet --pidfile $PIDFILE -bm \
    --exec $DAEMON --no-close -c nobody --test > /dev/null 2>&1 \
    || return 1
  start-stop-daemon --start --quiet --pidfile $PIDFILE -bm \
    --no-close -c nobody --exec $DAEMON -- \
    $DAEMON_ARGS >> /var/log/camo/camo.log 2>&1 \
    || return 2

What does `--no-close` do?

 -C, --no-close
     Do not close any file descriptor when forcing the daemon
     into  the  background  (since version 1.16.5).  Used for
     debugging purposes to see  the  process  output,  or  to
     redirect  file  descriptors  to  log the process output.

And in fact, looking in /proc/PID/fd while a hang is happening finds
that fd 0 on the camo daemon process, aka stdin, is connected to our
terminal.

So, stop that by denying the initscript our stdin in the first place.
This fixes the problem.

The Debian maintainer turns out to be "Zulip Debian Packaging Team",
at debian@zulip.com; so this package and its bugs are basically ours.
2018-01-22 18:55:46 -08:00
Greg Price cef8549ec6 install: Add --snakeoil-cert option.
This provides a major simplification for non-production installs,
including our own testing (it's already in both the test-install
harness script and the "production" test suite) as well as potential
admins evaluating Zulip.

Ultimately this should probably be the default behavior, with perhaps
something shown to admins on the web as a reminder and link to help on
installing a better certificate.  For now, pending working through
that, just get the behavior in and leave it opt-in.
2018-01-22 18:55:46 -08:00
Greg Price ac88f8ae1b setup-certbot: Stop automatically "agreeing" to the LE TOS.
It's not appropriate for our script to pass the `--agree-tos` flag
without any evidence of the user actually having any knowledge of,
let alone intent to agree to, any such ToS.  Stop doing that.
Fortunately this script hasn't been part of any release, so it's
likely that no users have gone down this path.
2018-01-22 18:55:46 -08:00
Greg Price 525b136f10 install: Install curl.
The third-party `install-yarn.sh` script uses `curl`, and we invoke it
in `install-node`.  So we need to install it as a dependency.

We've mostly gotten away with this because it's common for `curl` to
already be installed; but it isn't always.
2018-01-22 18:55:46 -08:00
Aditya Bansal c770bdaa3a reminder_bot: Add infra for adding reminder bot to every realm. 2018-01-19 11:33:11 -05:00
Umair Khan 68513952fb email-worker: Create EmailSendingWorker.
This commit just copies all the code from MissedMessageSendingWorker
class to a new EmailSendingWorker class. All the logic to send an email
through a queue was already there. This commit only makes the logic
generic. It does so by creating a special purpose queue called
'email_senders' to send any type of email. To make
MissedMessageSendingWorker still work we derive it from
EmailSendingWorker. All the tests that were testing
MissedMessageSendingWorker now run against EmailSendingWorker.
2017-12-20 19:36:27 -08:00
Rhea Parekh 460e9b4634 Scripts: Add script for resetting postgres sequences for imports. 2017-12-01 06:38:08 -08:00
Tim Abbott 15cf87f424 clean-npm-cache: Fix path to yarn in Travis CI.
Apparently, this was checking the wrong path in Travis CI, and thus
never actually running (meaning we'd accumulate every `node_modules`
directory ever in the Travis caches, which in turn resulted in very
slow builds).
2017-11-23 12:01:20 -08:00
Greg Price 64c608a51a install: Clarify how we set locale during install, and why.
This updates commit 11ab545f3 "install: Set the locale ..."
to be somewhat cleaner, and to explain more in the commit message.

In some environments, either pip itself fails or some packages fail to
install, and setting the locale to en_US.UTF-8 resolves the issue.

We heard reports of this kind of behavior with at least two different
sets of symptoms, with 1.7.0 or its release candidates:
  https://chat.zulip.org/#narrow/stream/general/subject/Trusty.201.2E7.20Upgrade/near/302214
  https://chat.zulip.org/#narrow/stream/production.20help/subject/1.2E6.20to.201.2E7/near/306250

In all reported cases, commit 11ab545f3 or equivalent fixed the issue.

Setting LC_CTYPE is redundant when also setting LC_ALL, because LC_ALL
overrides all `LC_*` environment variables; so skip that.  Also move
the line in `install` to a more appropriate spot, and adjust the
comments.
2017-11-22 18:11:20 -08:00
rht 585b71bc00 email-mirror-postfix: Replace optparse with argparse. 2017-11-21 21:34:38 -08:00
rht 54fb88f331 scripts: Replace optparse with argparse. 2017-11-21 21:23:41 -08:00
Vishnu Ks 766511e519 actions: Mark all messages as read when user unsubscribes from stream.
This fixes a bug where, when a user is unsubscribed from a stream,
they might have unread messages on that stream leak.  While it might
seem to be a minor problem, it can cause significant problems for
computing the `unread_msgs` data structures, since it means we need to
add an extra filter for whether the user is still subscribed, either
in the backend or in the UI.

Fixes #7095.
2017-11-21 20:09:17 -08:00
Harshit Bansal e75f0c1ee4 requirements: Rename requirements files.
This commit renames various source requirements files like `dev.txt`,
`mypy.txt` etc to `dev.in`, `mypy.in` etc and various locked requirements
files like `dev_lock.txt`, `mypy_lock.txt` etc to `dev.txt`, `mypy.txt`
etc. This will help in emphasizing to the user that *.in are actually
input to `update-locked-requirements` tool which should be run after
updating any of these.
2017-11-21 02:38:26 +05:30
Aditya Bansal dd037df2c4 thumbor: Add dependencies and virtualenv setup script.
In this commit we add new dependencies needed for running thumbor.
Also we add the script for creating the virtual environment ready
for thumbor.
Note: Thumbor will use python2 and thus have different virtualenv
dedicated to it.
Credits to @TigorC and @joshland as well for there work on this.
2017-11-16 22:38:29 -08:00
Aditya Bansal 79576797b0 thumbor: Auto generate thumbor_key and add it to zulip settings. 2017-11-16 22:34:14 -08:00
Tim Abbott 054952a44a docs: Update links from codebase to point to ReadTheDocs. 2017-11-16 10:53:49 -08:00
Greg Price a56fca81f1 setup-certbot: Require hostname and email.
The script already won't work without them; so if the user gets the
invocation wrong, give a halfway-reasonable error rather than just
crash into the ground.
2017-11-15 21:50:41 -08:00
Greg Price df8548aaf1 setup-certbot: Fix the usage message, and add the recently-added options. 2017-11-15 21:50:41 -08:00
Greg Price 8f387ba4d4 setup-certbot: Add option to choose verification method.
This allows the installer to continue using this script for the
`standalone` method, while the no-argument form now uses the same
`webroot` method as the renewal cron job, suitable for running
by hand to adopt Certbot after initial install.
2017-11-15 21:50:41 -08:00
Greg Price 7c887a6741 setup-certbot: Use set -x.
When there's a failure, this can make it much less confusing
to figure out.
2017-11-15 21:50:41 -08:00
Greg Price 9adaf3417a install: Enforce a constraint on the options, and expand usage message. 2017-11-15 21:50:41 -08:00
Greg Price 84f956f5f1 certbot: Use --deploy-hook to get the cert actually served.
Certbot replaces the cert files under /etc/letsencrypt/live/,
which our nginx config refers to symlinks to; but it doesn't
tell nginx there's been an update, so nginx keeps serving the
old cert.

This is fine as long as nginx is restarted, or just told to
reload its config, at some point before the cert actually
expires about 30 days later.  Which is probably the common
case, but of course we should make it just work.  So, if we
actually renew a cert, tell nginx to reload its config now.
2017-11-15 21:50:41 -08:00
Greg Price ae901309fc certbot: Control auto-renew with a zulip.conf setting.
This causes the cron job to run only when a Zulip-managed certbot
install is actually set up.

Inside `install`, zulip.conf doesn't yet exist when we run
setup-certbot, so we write the setting later.  But we also give
setup-certbot the ability to write the setting itself, so that we
can recommend it in instructions for adopting certbot in an
existing Zulip installation.
2017-11-15 21:50:41 -08:00
Greg Price dd32348fca setup-certbot: Eliminate obnoxious wget spew. 2017-11-15 21:50:41 -08:00
Greg Price 3f1f70fae2 setup-certbot: Treat potential existing certs with kid gloves.
This helps make this script suitable to run on existing installations,
by mitigating any worry about clobbering existing certs with links to
the new ones, in case the admin changes their mind or was using the
certs for something else too.
2017-11-15 21:50:41 -08:00
Tim Abbott 2afc3b9e50 certbot: Move path to /usr/local/sbin.
[greg: fixed typo bug]
2017-11-15 21:50:41 -08:00
rht 53e37aa511 scripts: Text-wrap long lines exceeding 110. 2017-11-10 16:22:26 -08:00
rht bb46bea44d Remove usage of six.moves.text_type. 2017-11-09 10:00:00 -08:00
derAnfaenger 19bc55aa45 Fix various typos.
The typos and their corrections were found with the
aid of https://github.com/lucasdemarchi/codespell.
2017-11-09 16:26:38 +01:00
Tim Abbott 38dc43fdf8 certbot: Use --standalone mode during installer.
This should help avoid problems, since we don't need to install
`nginx` before using this mode.
2017-11-08 12:32:26 -08:00
rht 01eb22abd9 refactor: Remove all the remaining six.moves import. 2017-11-07 10:51:44 -08:00
rht 9c7d5812ce refactor: Remove six.moves.urllib.parse import. 2017-11-07 10:51:44 -08:00
rht ccf2792c1c refactor: Remove six.moves.configparser import. 2017-11-07 10:51:44 -08:00
neiljp (Neil Pilgrim) 221be326a9 mypy: Set type parameters for IO to Any in scripts/lib/node_cache.py. 2017-11-04 19:47:44 -07:00
Harshit Bansal 48662a79a5 clean-venv-cache: Fix an exception with unusual deployment dirs. 2017-10-31 00:01:09 -07:00
Harshit Bansal 8c9ea94878 scripts: Fix an issue in `purge-old-deployments` script.
We were not including the real path of the symlinks due to which we
were incorrectly deleting deployments pointed by last/current/next.
2017-10-30 23:09:51 -07:00
Harshit Bansal 1871d6fe1f minor: Remove unnecessary path juggling in `get_recent_deployments()`. 2017-10-29 14:38:20 -07:00
Tim Abbott 11ab545f3b install: Set the locale so our dependencies can install.
Many pip packages don't install properly without a US locale.
2017-10-29 11:49:08 -07:00
rht 8b6b4e043f install: Add option to get certs via certbot.
While this doesn't quite complete our plans for certbot support (it's
not documented, etc.), this is a great stride forward.
2017-10-27 17:19:34 -07:00
Tim Abbott be619fe881 lint: Wrap many very long lines in the Python codebase.
This decreases the maximum line length in our Python codebase to 130.
2017-10-26 17:31:58 -07:00
Shekh Ataul d239f77966 refactor: Replace mkdir_p functions with Python 3 builtin.
This didn't exist in Python 2, but it does in Python 3, so we get to
reap the rewards of dropping Python 2 support.

Fixes #7082.
2017-10-25 11:06:11 -07:00
Tim Abbott 6b1eb647e4 nginx: Fix bugs in new nginx configuration checks. 2017-10-24 14:29:36 -07:00
Tim Abbott 730c77c7df docs: Document scripts/setup/generate-self-signed-certs.
And more generally clean up our non-LetsEncrypt SSL docs.

This should make it a bit easier to setup a Zulip server.
2017-10-24 13:48:14 -07:00
Tim Abbott 1b653409f4 install: Provide a nicer error message for bad nginx configuration.
This also covers missing SSL configuration errors nicely.
2017-10-24 13:39:39 -07:00
rht 5755f475f4 upgrade: Add "must be run as root" check.
The logic and text is based on scripts/lib/upgrade-zulip.
2017-10-23 23:01:54 -07:00
rht dcb46afa90 upgrade: Remove an extraneous `sudo` prefix.
Because scripts/upgrade-zulip-from-git must be run from root anyway.
2017-10-23 23:01:46 -07:00
rht c8090aafa4 lint: Remove several unused imports. 2017-10-18 10:56:11 -07:00
rht a603a4f9f5 Remove `from __future__ import absolute_import`.
Except in:
- docs/writing-bots-guide.md, because bots are supposed to be Python 2
  compatible
- puppet/zulip_ops/files/zulip-ec2-configure-interfaces, because this
  script is still on python2.7
- tools/lint
- tools/linter_lib
- tools/lister.py

For the latter two, because they might be yanked away to a separate repo
for general use with other FLOSS projects.
2017-10-17 22:59:42 -07:00
Tim Abbott 2ae2a94444 provision: Stop using shared var/ for caching apt state.
This didn't work at all when one did a `vagrant destroy` and then
`vagrant up`, because the cache state would be preserved even though
the machine is gone.

Fixes #5981.
2017-10-17 21:15:58 -07:00
rht c77c2f084c
upgrade-zulip-from-git: Fix rerun script and script description. 2017-10-18 04:41:22 +02:00
rht 9ab54e5bd7 scripts/lib/install: Add flag to specify key settings.
This should make it easier to script the installation process, and
also conveniently are the options one would want for the --certbot
option.

Significantly modified by tabbott to have a sane right interface,
include --help, and avoid printing all the `set -x` garbage before the
usage notices.
2017-10-03 16:56:45 -07:00
Tim Abbott 8c8613c3c7 install: Add comments documenting installer wrapper. 2017-10-01 15:48:36 -07:00
Alexander Trost 6a7d34316c Add scripts to be used in docker-entrypoint.sh.
Based on #450, with commits
restructured by Rein Zustand.
Tweaks by Rein Zustand:
- Replace configure-cert with generate-self-signed-certs
- `mv scripts/lib/create-zulip-admin.sh scripts/lib/create-zulip-admin`
2017-09-30 09:55:48 -07:00
rht 71188d7b0a scripts: Remove import print_function. 2017-09-29 15:43:30 -07:00
Alexander Trost dea507f4b8 Add script to create admin from CLI. 2017-09-27 20:15:41 -07:00
rht b937e1fede Remove the rest of `import six` outside zerver.lib.
We no longer need the six module, now that we're Python 3 only.
2017-09-27 17:09:15 -07:00
Tim Abbott 358cb40ed1 cache: Add backwards compatibility for emoji cache.
This allows the emoji cache cleaning code to run against old emoji
caches.
2017-09-25 17:06:02 -07:00
Tim Abbott db982672b1 caches: Don't depend on yarn.lock existing.
This allows our cache cleaning code to run on servers that have copies
of Zulip 1.6 and older around.
2017-09-25 17:05:45 -07:00
Tim Abbott 0f19e501a6 caches: Suppress unnecessary output when cleaning caches.
This should make the cache cleaning process a lot less spammy.
2017-09-25 16:34:03 -07:00
Tim Abbott 703b87c116 caches: Move clean-unused-caches to scripts/lib/. 2017-09-25 16:34:03 -07:00
Tim Abbott 86a07baf40 zulip_tools: Skip the lock directory.
This is the one special directory that usually lives in deployments/
and is not a deployment.  Make sure we don't treat it as a deployment.
2017-09-25 15:15:32 -07:00
Harshit Bansal 90f6ffa23d purge-old-deployments: Remove an unnecessary check.
Deployments whose name is not in the format of a timestamp are
always included in the `recent_deployments` and are not deleted,
hence we don't need to check for them explicitly.
2017-09-25 11:51:24 -07:00
Harshit Bansal a6caf30ca7 scripts: Fix an issue in `get_recent_deployments()` due to relative paths.
We were checking for whether an item in the deployments directory
represents a directory but were using its relative path which was
causing a false value to be returned for all items irrespective of
their being a directory or not if the script was invoked from some
where other than the deployments directory.
2017-09-25 11:51:24 -07:00
Harshit Bansal 6ff7da04de emoji: Remove `NotoColorEmoji.ttf`.
We no longer use glyphs from `NotoColorEmoji.ttf` so removing this.
2017-09-24 04:51:33 -07:00
Harshit Bansal c8c1c8ef43 emoji: Remove `AndroidEmoji.ttf`. 2017-09-24 04:51:33 -07:00
Tim Abbott 7e7b80f5e8 purge_old_deployments: Fix running from any path. 2017-09-24 04:42:14 -07:00
Harshit Bansal 57161a92a1 scripts: Rearrange the arguments of `purge_unused_caches()`.
This commit re-arranges the arguments of `purge_unused_caches()`
function in order to remain consistent with other similar functions
in the library like `may_be_perform_caching()`.
2017-09-24 04:37:31 -07:00
Harshit Bansal df7ea375c1 scripts: Make default mode of cache-cleaning scripts much less verbose.
Print a detailed report only if `--verbose` flag is specified.

Fixes: #6632.
2017-09-24 04:37:31 -07:00
Harshit Bansal 480ae626fb scripts: Add `--verbose` flag to purge-old-deployments script. 2017-09-24 04:37:31 -07:00
Harshit Bansal 3e8469a717 zulip_tools: Remove the now unused `GENERIC_CACHE_SCRIPT_PARSER`.
This has been replaced by `parse_cache_script_args()`.
2017-09-24 04:37:31 -07:00
Harshit Bansal 7f752f0942 scripts: Use `parse_cache_script_args()` in various cache cleaning scripts.
Instead of using `GENERIC_CACHE_SCRIPT_PARSER` and defining `parse_args()`
function in each script separately, use `parse_cache_script_args()`.
2017-09-24 04:37:31 -07:00
Harshit Bansal fe80330708 zulip_tools: Add `parse_cache_script_args()`.
This function will replace the repetitive definition of `parse_args()`
in various cache cleaning scripts. Also adds a `--verbose` argument
to the parser.
2017-09-24 04:37:31 -07:00
Harshit Bansal 4e6b68d02f zulip_tools: Change `purge_unused_caches()` API.
Instead of accepting individual arguments, accept `argparse.Namespace`
object as an argument.
2017-09-24 04:37:31 -07:00
Tim Abbott 0a91a5510c install: Fix check for whether update-prod-static is needed.
The previous version seems to be created without update-prod-static.
2017-09-22 19:52:40 -07:00
Tim Abbott 304bd86173 install: Support installing a Zulip server from a Git checkout.
Historically, one has needed to build a release tarball in order to
use/test the Zulip installer, but you could upgrade a Zulip server
from Git.  However, the only reason for that requirement was that we
didn't run `tools/update-prod-static` as part of the install script if
it's required.  A good test for that case is whether we're in a Git
repository, but a better one is to check whether the prod-static
content exists in the tarball paths.

Fixes #3704.
2017-09-22 15:47:42 -07:00
Juliana Bacelar 928dd06cc8 linter: Add lint rule banning 'import os.path' 2017-09-22 10:32:21 -07:00
julianasobreira df2d448f7b python: Add lint rule banning 'from os.path import'.
This enforces our use of a consistent style in how we access Python
modules; "from os.path import dirname" is a particularly popular
abbreviation inconsistent with our style, and so it deserves a lint
rule.

Commit message and error text tweaked by tabbott.

Fixes #6543.
2017-09-22 04:55:38 -07:00
Tim Abbott 901e0258df purge-old-deployments: Fix purging the symlinks etc.
The recent rewrite of purge-old-deployments accidentally attempted to
purge the symlinks, sockets, lock, and other files in the deployment
directory.

The new version has been tested out in production successfully.
2017-09-16 08:49:05 -07:00
Harshit Bansal 26915bc54f scripts: Make `purge-old-deployments` configurable.
Expands `purge-old-deployments` such that now it accepts the threshold
days as argument. Also `clean-unused-caches` script is automatically
run after purging the old deployments so that the orphaned caches
gets automatically cleaned.

Fixes: #5726.
2017-09-16 08:28:57 -07:00
Harshit Bansal 20f062f726 zulip_tools.py: Extract `may_be_perform_purging()` function.
Based on the `dry_run` flag, this function either purges the list
of directories passed to them or prints a listing of the directories
it would have purged/kept_back, had the `dry_run` flag been false.
2017-09-16 08:28:57 -07:00
Tim Abbott 600b164130 setup-apt-repo: Fix failure to rerun properly on provision failure.
Apparently, the refactoring to make this script only run when changes
are present was buggy, in that if `apt-get update` failed, running
provision against wouldn't rerun `apt-get update`, resulting in a
broken state that requires expertise to fix.  This closes that gap, by
using a stamp file to ensure we always successfully update apt before
proceeding.

It doesn't fix existing installations.
2017-09-12 07:08:54 -07:00
Tim Abbott 6d5dac96c6 zulip-puppet-apply: Fix flushing of stdout for Python 3.
Apparently, this UI worked just fine with Python 2, but failed to
actually print usefully with Python 3.
2017-08-27 18:36:39 -07:00
Tim Abbott 1a1df29053 get_recent_deployments: Skip uwsgi socket and friends.
This fixes an exception when running clean-venv-caches in production.
2017-08-27 18:18:53 -07:00
Tim Abbott e38ac00f3c caches: Move cache cleaning helper tools under scripts/lib.
This helps keep the root of scripts/ uncluttered.
2017-08-27 17:59:49 -07:00
Harshit Bansal f90f9c7404 scripts: Add `clean-unused-caches` script.
This script can be used for running all the three cache cleaning scripts
in one go.
2017-08-27 17:51:24 -07:00
Harshit Bansal e5361b4403 clean_emoji_cache: Expand `clean-emoji-cache`.
Expands `clean-emoji-cache` so that it can be used in production
environment as well. Also moves it to `scripts/` from `tools/`.
2017-08-27 17:51:24 -07:00