Commit Graph

35 Commits

Author SHA1 Message Date
rht c8090aafa4 lint: Remove several unused imports. 2017-10-18 10:56:11 -07:00
rht 71188d7b0a scripts: Remove import print_function. 2017-09-29 15:43:30 -07:00
julianasobreira df2d448f7b python: Add lint rule banning 'from os.path import'.
This enforces our use of a consistent style in how we access Python
modules; "from os.path import dirname" is a particularly popular
abbreviation inconsistent with our style, and so it deserves a lint
rule.

Commit message and error text tweaked by tabbott.

Fixes #6543.
2017-09-22 04:55:38 -07:00
Greg Price 4b58794fe2 provision: Install Python 3 versions of system dependencies.
Also make our dependency on `six` (for e.g. `replace-tarball-shebang`)
explicit -- we've been getting it via `python-pip`, but `python3-pip`
(on trusty) doesn't have that dependency for some reason.
2017-08-09 14:05:52 -07:00
Aditya Bansal cf503017d2 Use requirements file for pip/setuptools/wheels in venv setup.
Fixes: #5158.
2017-06-19 16:37:50 -04:00
Vishnu Ks 7c5dd68414 Upgrade setuptools to 36.0.1. 2017-06-19 06:40:18 -04:00
Aditya Bansal f9cbb0936a setup_venv: Add wrapper to retry pip install if it fails first time. 2017-06-14 21:48:15 -07:00
Tim Abbott be814b940d setup_venv: Pin a version of setuptools for creating venvs.
This works around a bad recent setuptools release:

https://github.com/pypa/setuptools/issues/1042
2017-05-31 23:58:24 -07:00
Tim Abbott 1d5edff927 setup_venv: Give up if virtualenv-clone isn't working.
virtualenv-clone can sometimes fail if the old virtualenv is
broken; in that case, we can just make a new one.
2017-01-09 11:21:42 -08:00
Tim Abbott 4a4664d268 mypy: Remove a bunch of now-unnecessary type: ignore annotations.
Since mypy and typeshed have advanced a lot over the last several
months, we no longer need these `type: ignore` annotations.
2016-10-17 11:48:34 -07:00
Tim Abbott fcde846b58 setup_venv: Fix handling of Git requirements.
get_package_names did not correctly strip the GitHub URLs from package
names, resulting in the "package names" for our dependencies installed
from Git being tracked with the complete sha1sum included in the name.
This meant that upgrading our virtualenvs incorrectly ended up
resorting to creating an entirely new virtualenv whenever we changed a
dependency that had previously been installed from GitHub URLs.
2016-10-16 01:10:43 -07:00
Steve Howell 13d6f52203 provision: Handle VENV_CACHE_PATH not existing.
If VENV_CACHE_PATH does not exist (which can happen if you destroy
your vagrant environment), then do a short circuit return in
try_to_copy_venv().
2016-09-20 06:55:08 -07:00
Umair Khan e3078b226a Optimize incremental virtualenv creation.
This adds a new system for copying packages from old virtualenvs that
are sufficiently similar to the new virtualenv required.

In practice, this results in a huge performance improvement for
re-provisioning Zulip development environments when the requirements
files have changed (which is the dominant performance problem with
provision today).

Fixes: #1507.
2016-09-19 22:35:32 -07:00
Umair Khan 859b1fbb2c setup_venv.py: Remove comma from the end. 2016-09-08 14:14:28 -07:00
Taranjeet Singh d606b95242 zulip_tools.py: Move zulip_tools.py in scripts/lib.
This commit moves zulip_tools.py as part of cleaning the root directory
and organizing proejct into better directory structure.
2016-08-15 16:44:50 -07:00
ukhan 180b438c44 Upgrade setuptools prior to updating mock 2016-08-12 17:56:23 -07:00
Eklavya Sharma baa157344c Patch activate script only in development. 2016-07-21 14:11:41 -07:00
Eklavya Sharma e7813094d7 Patch activate script to set VIRTUAL_ENV correctly.
Patch a virtualenv's activate script to not resolve symlinks when
setting the environment variable VIRTUAL_ENV.

Fixes #1190.
2016-07-21 14:11:41 -07:00
Tim Abbott dd6e92a714 setup_venv: Fix missing dependencies on image libraries. 2016-07-20 14:55:45 -07:00
Eklavya Sharma 576a090499 Add libpq-dev to VENV_DEPENDENCIES. 2016-07-20 22:15:06 +05:30
Tim Abbott 8eaa75b90f setup_venv: Remove unnecessary postgresql-server-dev-all dependency.
This package is a dependency for building tsearch_extras, but isn't
needed for building a Zulip virtualenv.
2016-07-19 19:19:42 -07:00
Umair Khan 9f42fda7f4 Use stripped down version of Talon. 2016-07-13 11:24:18 -07:00
Umair Khan 395e053ce3 Revert "Revert "Extract reply from email.""
This reverts commit f1ba3ded42.
2016-07-13 11:24:18 -07:00
Eklavya Sharma f1ba3ded42 Revert "Extract reply from email."
This reverts commit f1f48f305e.

The use of sklearn unfortunately caused a substantial slowdown to the
Zulip provisioning process, which didn't seem worth it for a
relatively minor feature.
2016-07-10 11:30:30 -07:00
Umair Khan f1f48f305e Extract reply from email. 2016-07-08 10:58:25 -07:00
Eklavya Sharma 7e020e3dae scripts/lib/setup_venv.py: Use universal_newlines=True.
In python 3, subprocess uses bytes for input and output if
universal_newlines=False (the default).  It uses str for input and
output if universal_newlines=True.
Since we're dealing with strings here, add universal_newlines=True
to subprocess.check_output calls.
2016-07-06 13:50:55 -07:00
Eklavya Sharma f1b72c5f41 Upgrade wheel before installing venv requirements.
This is required because an older wheel version causes problems
when building wheels on python 3.
2016-07-02 11:53:26 -07:00
Tim Abbott 4aae4de294 Properly cache production virtualenvs in Travis CI.
Previously, the --travis argument wasn't available to the `setup_venv`
code when creating the production virtualenv.
2016-06-28 12:04:11 -07:00
Tim Abbott 7ed0ab8c4a Ensure Zulip virtualenvs are world-readable.
This is important for both ensuring the Nagios checks work correctly
in production, as well as making sure the `zulip` user can access the
virtualenv (owned by the `travis` user) in Travis CI.
2016-06-28 12:03:01 -07:00
Tim Abbott 4a7913bc22 setup_venv: Use correct VENV_CACHE_PATH for production suite. 2016-06-27 21:16:22 -07:00
Tim Abbott dfd8fede4a setup_venv: Add missing build essential dependencies.
Apparently, c74a74dc74 introduced a bug
where we are no longer correctly depending on build-essential as part
of the Zulip development environment installation process.

Fixes #1111.
2016-06-24 09:11:10 -07:00
Eklavya Sharma 9482cdccf4 scripts/lib/setup-venv.py: Accept None as target_venv_path.
If target_venv_path in setup_virtualenv is None, create a virtualenv
named 'venv' but don't make a symlink to it.  Also return
cached_venv_path.
2016-06-23 14:17:17 -07:00
Eklavya Sharma 0365a4a9f2 Factor out apt dependencies for setting up virtualenv.
Move apt dependencies for creating a virtualenv from provision.py
to scripts/lib/setup_venv.py.
2016-06-23 14:17:17 -07:00
Eklavya Sharma 121a5f26fb Move hash_reqs.py from tools/ to scripts/lib/.
This is needed because hash_reqs.py is used to create a virtualenv.
Currently we only use virtualenv in development, but we will soon
start using it in production.  Scripts used in production should be
put in scripts/.
2016-06-22 18:12:08 +05:30
Eklavya Sharma fc8d4f9ef5 Factor out venv-creating code from provision.py.
Move setup_virtualenv and do_setup_virtualenv from provision.py to
scripts/lib/setup_venv.py.
2016-06-21 11:25:41 -07:00