Commit Graph

54 Commits

Author SHA1 Message Date
Aditya Bansal d5736ef65a provision.py: Remove duplicate install candidated from APT_DEPENDENCIES.
This is necessary, since we now have some dependencies in common
between the Thumbor and main Zulip dependency lists.
2018-02-12 14:40:26 -08:00
rht 396a8b644d provision.py: Colorize error messages for fail condition. 2018-02-05 11:44:33 -08:00
Aditya Bansal 0fcf0c5052 thumbor: Add thumbor on port 9995 in development.
For now, this does nothing in a production environment, but it should
simplify the process of doing testing on the Thumbor implementation,
by integrating a lot of dependency management logic.
2018-01-29 13:10:29 -08:00
Greg Price ff29fe07be provision: Install virtualenv on xenial too.
As the commit message on 680381c9d which added this for stretch says,
we need this dependency when on xenial as well.  Add it there.
2018-01-22 19:36:52 -08:00
Aditya Bansal c770bdaa3a reminder_bot: Add infra for adding reminder bot to every realm. 2018-01-19 11:33:11 -05:00
Rhea Parekh 6e374d30e1 tools/lib/provision: Clean the code to directly read files in binary mode. 2017-12-26 09:09:31 -05:00
Greg Price e5c6c446b5 travis + circleci: Start postgresql during provision.
Otherwise, in CircleCI it isn't running when we need it later.
Travis must special-case starting it automatically.
2017-12-18 22:05:27 -08:00
Greg Price c83c118c6c circleci: Crudely clone a bit of `is_travis` logic.
This isn't really the right way to do this -- commit
  dcd80e665 "travis/setup-backend: Remove the '--travis' flag"
took us in the wrong direction by introducing more magic, deeper in
the stack.

But it's the same way we do it for Travis.  For now, just copy that.

[Thanks to hackerkid for cleaning up my original crude hack.]
2017-12-18 22:04:39 -08:00
Greg Price e8bf7d23bf provision: Remove transitional Python 2->3 logic in setup_venvs.
At this point if we were accidentally using `/srv/zulip-venv` for
anything, we'd have run into it by now.  So just drop the bit of
historical logic that we had to ensure that.
2017-12-18 21:59:51 -08:00
rht bff736868e Generate custom-icon-webfont on each provision or update-prod-static.
Fixes #7354.
2017-11-20 16:36:49 -08:00
Greg Price 13aa23eb40 provision: Run clean-unused-caches.
This prevents the caches in /srv from growing to fill up the disk --
e.g., on my laptop after 6 months of regular development the venv cache
was 12G and the NPM cache 5G, making them by far the largest disk hogs
on the machine.

It costs about 0.4s, apart from any time spent actually removing
things.  This is a little annoyingly slow to be adding to every
provision, and seems like it could be optimized, but I think already
worth it as is.
2017-11-18 15:04:08 -08:00
Tim Abbott 638eb7a8e4 docs: Update links to ReadTheDocs to always use https.
This is better security practice.

We also add a lint rule to enforce this for the future.
2017-11-16 10:59:24 -08:00
Tim Abbott 054952a44a docs: Update links from codebase to point to ReadTheDocs. 2017-11-16 10:53:49 -08:00
rht 72ce1c6501 Use a more concise `mkdir -p` when the situation permits. 2017-11-14 09:47:09 -08:00
rht 7a96b8c7ad tools/lib: Text-wrap long lines exceeding 110. 2017-11-10 16:24:09 -08:00
Tim Abbott 508686bed4 provision: Fix buggy bash_profile management code.
Apparently, the previous version would just append to .bash_profile
every single time :(.
2017-10-17 21:22:32 -07:00
Tim Abbott 3fed995a41 provision_version: Move to under UUID_VAR_PATH. 2017-10-17 21:15:58 -07:00
Tim Abbott 3983ebce9a test_fixtures: Move test_db_status state under UUID_VAR_PATH.
This should mean that maintaining two Zulip development environments
using the same Git checkout no longer has caching problems keeping
track of the migration status.
2017-10-17 21:15:58 -07:00
Tim Abbott a7c639cff7 provision: Use UUID in path to compilemessages hash. 2017-10-17 21:15:58 -07:00
Tim Abbott 2ae2a94444 provision: Stop using shared var/ for caching apt state.
This didn't work at all when one did a `vagrant destroy` and then
`vagrant up`, because the cache state would be preserved even though
the machine is gone.

Fixes #5981.
2017-10-17 21:15:58 -07:00
rht dcd80e6650 travis/setup-backend: Remove the '--travis' flag in tools/provision.
Whether the env is Travis or not, this has been detected via $TRAVIS env var.
Proof:
6fbf41bdbc/tools/provision (L38)
2017-10-06 11:20:25 -07:00
rht 2c2adbf647 tools/lib: Remove unused imports. 2017-10-02 10:13:46 -07:00
rht f15bdce90e tools: Remove print_function.
Tweaked by tabbott to exclude the linter libraries.
2017-09-29 15:44:56 -07:00
Rhea Parekh 0e5fdcea3f provision: Run apt-get update on catching an error.
For broken apt cache, we run the command apt-get update on catching
the error.
2017-09-29 15:38:23 -07:00
Eeshan Garg bb34ba37b5 tools: Rename build_pygments_data.py -> build_pygments_data.
Dropping the file extension (.py) conforms to our naming
convention for such scripts.
2017-09-26 16:00:41 -07:00
Tim Abbott 8e2ae66fe6 provision: Update apt operations text. 2017-09-25 14:47:18 -07:00
Tim Abbott 16ed23414d provision: Fix spelling of apt_dependencies_hash.
Also use a variable to avoid duplicating the path.
2017-09-25 14:46:46 -07:00
Eeshan Garg 390a1fec92 zulip_bots: Generate static files during provisioning.
This commit implements support for copying over static files
for all bots in the zulip_bots package to
static/generated/bots/ during provisioning. This directory
isn't tracked by Git. This allows us to have access to files
stored in an arbitrary zulip_bots package directory somewhere
on the system. For now, logo.* and doc.md files are copied over.

This commit should act as a starting point for extending our
macro-based Markdown framework to our bots/API packages'
documentation and eventually rendering these static files
alongside our webhooks' documentation.
2017-09-22 15:14:42 -07:00
Vishnu Ks 61e48734a7 provision: Add moreutils in APT common dependencies.
This fixes an issue where `tools/update-locked-requirements` would fail.
2017-09-22 14:04:57 -07:00
Vishnu Ks f4c69bb126 tools: Create tool for inlining email CSS during provision. 2017-09-16 07:17:48 -07:00
Greg Price a099e698e2 py3: Switch almost all shebang lines to use `python3`.
This causes `upgrade-zulip-from-git`, as well as a no-option run of
`tools/build-release-tarball`, to produce a Zulip install running
Python 3, rather than Python 2.  In particular this means that the
virtualenv we create, in which all application code runs, is Python 3.

One shebang line, on `zulip-ec2-configure-interfaces`, explicitly
keeps Python 2, and at least one external ops script, `wal-e`, also
still runs on Python 2.  See discussion on the respective previous
commits that made those explicit.  There may also be some other
third-party scripts we use, outside of this source tree and running
outside our virtualenv, that still run on Python 2.
2017-08-16 17:54:43 -07:00
Greg Price b7ec1b55dd py3: Cut Python 2 codepaths from provision. 2017-08-16 17:54:43 -07:00
Anirudh Jain 28944b6c94 provision: Partially add zesty to supported systems.
We can't fully support it until we fix the tsearch_extras availability
issue, but for now, this is an improvement.

Tweaked by tabbott to cover the outstanding tsearch_extras issue.
2017-08-15 21:51:19 -07:00
Pweaver (Paul Weaver) 1afaa67c7c deps: Change npm to yarn for reliablity, security, and speed. 2017-08-05 12:29:06 -07:00
rht 532bd6eaa6 Ensure virtualenv is run throughout the entire Travis dev provision. 2017-08-01 15:29:13 -07:00
Tim Abbott 5d05cc7294 scripts: Rename NPM_CACHE_PATH to NODE_MODULES_CACHE_PATH. 2017-07-21 17:28:20 -07:00
Tim Abbott 680381c9d0 setup_venv: Add virtualenv as a dependency for Debian.
Apparently, this was missing on Xenial, and we just never noticed
because the package was installed by default on Ubuntu.

It doesn't exist yet in Trusty.
2017-07-14 17:22:20 -07:00
Tim Abbott 4906a47c1a provision: Disable tsearch_extras for Debian. 2017-07-14 17:22:20 -07:00
Tim Abbott 9efc017e8c provision: Add postgres versions for Debian stretch. 2017-07-14 17:22:20 -07:00
Harshit Bansal ebdd8c9e3d emoji: Switch to using `emoji-datasource` npm package.
This commit does the following things:
* Instead of using a manual tool for downloading sprite sheets, use
  `emoji-datasource` npm package.
* Modify the `build_emoji` script to use sprite sheets from the npm
  package.

Bumps PROVISION_VERSION.

Fixes: #4730.
2017-07-13 21:52:01 -07:00
Harshit Bansal 420327b558 provision: Install NPM packages in the beginning of provisioning process.
NPM packages should be installed at the beginning of the provisioning
process so that later in the provisioning process if a script requires
any NPM package it can use it. Earlier, we were installing NPM packages
in the last as the installation process can fail due to network issues
but since we now retry in case the installation fails, they can be
installed safely at the beginning of the process as well just like apt
packages.
2017-07-13 21:52:01 -07:00
Tim Abbott 774216713b provision: Document more clearly the is_production_travis block. 2017-07-06 17:39:49 -07:00
sinwar 49db0855d2 provision: Avoid spending ~3s on RabbitMQ in no-op case.
Tweaked by tabbott to reorganize the logic more clearly.

Fixes #5180.
2017-07-06 17:38:39 -07:00
sinwar 3d0b5b14be provision: Avoid spending ~1.5s on compilemessages in no-op case.
This brings the total no-op provision time down to around 7.3s!

Tweaked by tabbott to clean up the code, close file descriptions, and
avoid issues with a somehow corrupted file.

Fixes #5185.
2017-07-01 11:27:54 -07:00
sinwar 0e24930770 provision: Avoid spending ~3s on apt operations in no-op provision.
Fixes #5186
2017-06-22 09:37:40 -04:00
Aditya Bansal a2b5d6efcb provision: Add wrapper to retry apt-get install if it fails first time. 2017-06-14 21:49:15 -07:00
Tim Abbott cecf7f1740 provision: Don't regenerate dev database unnecessarily.
This reuses the work we did some time ago to avoid regenerating the
test database unnecessarily.

In addition to being a nice convenience for developers (since any
accomulated test data is still available), this also saves about half
the time consumed in a no-op provision.

Fixes #5182.
2017-06-02 13:27:34 -07:00
Steve Howell 5f48b51b60 provision: Add `cd /srv/zulip` to profile.
I didn't bump the provision version on this change, as it
won't break anything if somebody fails to make this change.
2017-05-29 08:40:49 -07:00
Tim Abbott 8aaf9b1426 Revert "Fix emoji cache setup when not using provision.py."
This reverts commit ab2cfadcaa.

This broke production installation.
2017-05-24 10:35:41 -07:00
Carey Metcalfe ab2cfadcaa Fix emoji cache setup when not using provision.py.
Moves creating the emoji folder from the provisioning script to
the build_emoji script.

Fixes the fact that the emoji cache directory wasn't being created
when not using the provision.py script.
2017-05-22 11:08:43 -07:00