Commit Graph

160 Commits

Author SHA1 Message Date
Eklavya Sharma 0365a4a9f2 Factor out apt dependencies for setting up virtualenv.
Move apt dependencies for creating a virtualenv from provision.py
to scripts/lib/setup_venv.py.
2016-06-23 14:17:17 -07:00
Tim Abbott d3f3046629 Add python-six to early installer dependencies.
Since we're now using python-six in zulip-puppet-apply, we need to
install python-six before calling into zulip-puppet-apply.
2016-06-22 08:11:34 -07:00
Eklavya Sharma 121a5f26fb Move hash_reqs.py from tools/ to scripts/lib/.
This is needed because hash_reqs.py is used to create a virtualenv.
Currently we only use virtualenv in development, but we will soon
start using it in production.  Scripts used in production should be
put in scripts/.
2016-06-22 18:12:08 +05:30
Eklavya Sharma fc8d4f9ef5 Factor out venv-creating code from provision.py.
Move setup_virtualenv and do_setup_virtualenv from provision.py to
scripts/lib/setup_venv.py.
2016-06-21 11:25:41 -07:00
Tim Abbott f7ce5fc179 generate_secrets: Silence mypy error with configparser.
See https://github.com/python/typeshed/issues/307.
2016-06-20 09:03:09 -07:00
Umair Khan a9a6687b7d Make generate-secrets script use existing values.
Fixes #1035
2016-06-20 17:00:27 +05:00
Michael Cordover b401ec0af7 Warn on postgres-init-db if >200 messages exist.
Closes #548.
2016-06-13 15:57:51 -07:00
acrefoot cf15b0b4e6 Fix hanging nc and 'invalid wait time' error on Trusty.
Apparently, 0 isn't a supported wait time value in some versions of nc.
2016-06-02 18:02:13 -07:00
Eklavya Sharma 94e4b39112 Replace python2.7 by python everywhere. 2016-05-29 05:03:08 -07:00
Eklavya Sharma 149938d468 Change shebangs from python2.7 to python. 2016-05-29 05:03:08 -07:00
Tim Abbott 6e1872987d Move bin/get-django-setting to scripts/. 2016-05-07 19:37:06 -07:00
Tim Abbott a315849a9e Move bin/log-management-command to scripts/lib/.
We're in the process of eliminating the bin/ subdirectory in favor of
the scripts/ tree, and this one isn't user-facing.
2016-05-07 19:37:06 -07:00
Tim Abbott cb81a59e38 Move write-rabbitmq-consumers-state-file to scripts/nagios/. 2016-05-07 19:37:06 -07:00
Tim Abbott 2761c012e5 Move rabbitmq consumer checks from bots/ to scripts/nagios/. 2016-05-07 19:37:06 -07:00
Tim Abbott be6566dc5c nagios: Move cron_file_helper from bots/ to scripts/lib.
This ensures the tool is available in Zulip production deployments.
2016-05-07 19:37:06 -07:00
Tim Abbott 52c1e8ac7d Run a local camo server in voyager production environments.
Camo is a caching image proxy, used in Zulip to avoid mixed-content
warnings by proxying HTTP image content over HTTPS.  We've been using
it in zulip.com production for years; this change makes it available
in standalone Zulip deployments.
2016-05-02 17:21:31 -07:00
Tim Abbott 6e1e4aaef6 postgres-init-db: Add POSTGRES_USER argument. 2016-04-26 15:27:35 -07:00
Tim Abbott dc772518e7 Don't chown supervisor socket if it doesn't exist. 2016-04-26 15:27:35 -07:00
Tim Abbott 6a3c775842 install: Ensure prod-static/serve is created. 2016-04-26 15:27:35 -07:00
Tim Abbott bb25b6060e install: Avoid unnecessarily storing apt key under /root. 2016-04-26 15:07:53 -07:00
Tim Abbott e9416a9fb2 install: Add PUPPET_CLASSES variable. 2016-04-26 15:06:37 -07:00
Tim Abbott a9d86a3620 install: Add DEPLOYMENT_TYPE variable. 2016-04-26 15:04:32 -07:00
Tim Abbott 68c6d514e8 install: Add ADDITIONAL_PACKAGES option. 2016-04-26 15:02:28 -07:00
Tim Abbott ae48f6394b migrations: Disable prompting about content-type deletion.
The main function of prompting inside `manage.py migrate` is to ask
the user if they want to delete stale content-types, which is
unimportant and likely scary, so we disable doing so.
2016-04-13 13:19:02 -07:00
Tim Abbott 4352a022cd Add type: ignore to wildcard imports in pythonrc.py. 2016-04-03 15:40:24 -07:00
Tim Abbott f9222de83e Auto-load commonly used modules in manage.py shell.
This automatically loads settings, zerver.models.* and
zerver.lib.actions.* when you start `manage.py shell`, which should
save a bit of time basically every time someone uses it.

Fixes #275.
2016-03-19 11:32:49 -07:00
Eklavya Sharma 7b8cb105bf Apply Python 3 futurize transform libmodernize.fixes.fix_imports_six
Refer to #256
2016-03-10 22:03:30 -08:00
Eklavya Sharma c59185e119 Apply Python 3 futurize transform libfuturize.fixes.fix_print_with_import
Refer #256
2016-03-10 22:02:17 -08:00
Luke Faraone 9d9bfb27ef Correct shell quoting around $DEFAULT_USER in terminate-psql-sessions
Previously, we used shell quoting that would result in the shell variable not
being substituted. Instead, we use `"`s that will allow for variable
substitution.
2016-02-19 02:09:50 +00:00
Vladislav Manchev dfbea01c8f Add support for running OpenBSD in development environment. 2016-01-21 22:33:55 -08:00
Alexander Trost 84f7a1f1ea Make rabbitmq, redis, and memcached configurable via user settings.py.
Previously these were hardcoded in zproject/settings.py to be accessed
on localhost.

[Modified by Tim Abbott to adjust comments and fix configure-rabbitmq]
2016-01-21 22:07:56 -08:00
Tim Abbott 26e9d55e16 deployments: Refactor locking libraries into zulip_tools.py.
The code in update-deployment and upgrade-zulip for managing the
deployment lock was nearly identical.
2016-01-11 21:36:42 -08:00
Tim Abbott f871090bb6 upgrade-zulip: Archive release tarballs at /home/zulip/archives.
A common issue when doing a Zulip upgrade is trying to pass
upgrade-zulip a tarball path under /root, which doesn't work because
the Zulip user doesn't have permission to read the tarball.  We
could fix this by just unpacking the tarballs as root, but it seemed
like a nicer approach would be to archive the release tarballs
somewhere readable by the Zulip user (/home/zulip/archives) and unpack
them from there.

Fixes #208.
2016-01-11 21:36:42 -08:00
Tim Abbott c101bf663d Run upgrade-zulip-stage-2 from an absolute path.
This should make it more obvious in tracebacks that we are running the
script from the version of Zulip we're upgrading to, not the old
version.
2016-01-11 21:36:42 -08:00
Tim Abbott 52d0423591 Document structure of upgrade-zulip-stage-2 more clearly. 2016-01-11 21:36:41 -08:00
Tim Abbott 186f563176 Fix deployment locks being leaked when a deployment fails.
The point of the lock is to prevent two deployments happening at the
same time and racing with each other, not to prevent doing any future
deployments after an error happens (which is what the current
implementation does in practice).

Addresses part of #208.
2016-01-11 21:36:41 -08:00
Tim Abbott e7353902df upgrade-zulip: Restart process-fts-updates if also a DB host.
Previoulsy, process-fts-updates wouldn't be restarted on a server
upgrade in Voyager configurations.
2016-01-09 16:59:43 -08:00
Tim Abbott 75b5a1b8da upgrade-zulip: Stop zulip-senders too when shutting down services. 2016-01-09 16:59:43 -08:00
Reid Barton b2a92877ff Don't print echo commands in initialize-database post-success message. 2015-12-25 10:38:44 -08:00
Reid Barton 64a142f0a2 Fix running postgres-init-db via a relative path.
If the user runs ./scripts/setup/postgres-init-db, then dirname "$0"
would no longer refer to the correct directory after cd /.
2015-12-25 10:06:45 -08:00
Javier Ros ab89ef501f Add support for a development environment with Docker. 2015-12-14 18:22:56 -08:00
Tim Abbott 7395003e6a Fix buggy #! lines using "/usr/bin/env python2.7 -u".
The #! line processing interpreted the argument to pass to `env` as
"python2.7 -u", which obviously isn't a real program.

We fix this by setting the PYTHONUNBUFFERED environment variable
inside the program, which has the same effect.

Thanks to Dan Fedele for the bug report and suggested solution!
2015-11-16 08:46:48 -08:00
Tim Abbott 827babdf29 terminate-psql-sessions: Remove dependency on bc.
Fixes #281.
2015-11-11 21:35:16 -08:00
Tim Abbott 6eb670097c Expand testing done via Travis CI to cover production pipeline.
With this change, we are now testing the production static asset
pipeline and installation process in a new testing job (and also run
the frontend/backend tests separately).

This means that changes that break the Zulip static asset pipeline or
production installation process are more likely to fail tests.  The
testing is imperfect in that it does not have proper isolation -- we
build a complete Zulip development environment and then install a
Zulip production environment on top of it, so e.g. any apt
dependencies installed for Zulip development will still be available
for the Zulip production environment.  But, it's better than nothing!

A good v2 of this would be to have the production setup process just
install the minimum stuff needed to run `build-release-tarball` and
then uninstall it / clean it up so that we can do a more clear
production installation, but that's more work.
2015-11-01 18:11:39 -08:00
Tim Abbott 421560af21 postgres-init-db: Stop all services before recreating database. 2015-11-01 18:11:39 -08:00
Tim Abbott 3c31f9a2e3 Drop database users prior to DROP/CREATE database.
This fixes an annoying issue where one tries to rebuild the database,
and it fails due to there being existing connections.

The one thing that is potentially scary about this implementation is
that it means it's now a lot easier to accidentally drop your
production database by running the wrong script; might be worth adding
a "--force" flag controlling this behavior or something.

Thanks to Nemanja Stanarevic and Neeraj Wahi for prototypes of this
implementation!  They did most of the work and testing for this.
2015-11-01 18:11:39 -08:00
Tim Abbott b7cd000af6 install: Check nginx configuration is valid.
It's better to fail here and have the user correct the issue than fail
later.
2015-11-01 18:06:59 -08:00
Tim Abbott f3783fb4a1 Apply Python 3 futurize transform libfuturize.fixes.fix_print_with_import. 2015-11-01 09:26:16 -08:00
Steven Oud d5435fad1d Consistently use /usr/bin/env python2.7 in shebangs and commands. 2015-10-21 22:58:21 +00:00
Tim Abbott e75ba630fb initialize-database: Make management command errors fatal again.
We accidentally made this non-fatal when we added the nice error
output telling users to run postgres-init-db.
2015-10-15 12:21:46 -04:00