Mirror
/
zulip
mirror of https://github.com/zulip/zulip.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
zulip/.circleci/config.yml

177 lines
5.4 KiB
YAML
Raw Normal View History

circle: Draft a config for running our builds in CircleCI. From just my experience attempting this change, I've been impressed with the very fast turnaround in CircleCI for a change -- from `git push` to the build starting is usually just a couple of seconds. It also looks like we get much clearer control over the caching system, and the setup of the environment via a Docker image of our choice, which can help avoid most of the several minutes Travis takes to set up our environment before it actually starts running the tests themselves. Plus it supports fine-grained parallelism, e.g. by test file: https://circleci.com/docs/2.0/parallelism-faster-jobs/ So we can probably get much faster builds by making this switch. Because the environment is much more transparent and even under our control, I'm also hopeful for more reliable builds.
2017-08-30 02:09:09 +02:00
# See CircleCI upstream's docs on this config format:
# https://circleci.com/docs/2.0/language-python/
#
version: 2
circleci: Use alias for create cache directories.
2018-12-05 16:46:25 +01:00
aliases:
- &create_cache_directories
run:
name: create cache directories
command: |
dirs=(/srv/zulip-{npm,venv}-cache)
sudo mkdir -p "${dirs[@]}"
sudo chown -R circleci "${dirs[@]}"
circleci: Use alias for restore cache.
2018-12-05 17:24:51 +01:00
- &restore_cache_package_json
# Workaround until arbitrary env var lookup is implemented by CircleCI
# https://discuss.circleci.com/t/cannot-use-circle-yml-environment-variables-in-cache-keys/10994/8
restore_cache:
keys:
- v1-npm-base.{{ checksum "_dist.txt" }}-{{ checksum "package.json" }}-{{ checksum "yarn.lock" }}
- &restore_cache_requirements
restore_cache:
keys:
- v1-venv-base.{{ checksum "_dist.txt" }}-{{ checksum "requirements/thumbor.txt" }}-{{ checksum "requirements/dev.txt" }}
circleci: Use alias for install dependencies.
2018-12-05 16:53:07 +01:00
- &install_dependencies
run:
name: install dependencies
command: |
sudo apt-get update
sudo apt-get install -y moreutils
rm -f /home/circleci/.gitconfig
tools: Rename tools/travis directory to tools/ci.
2018-12-10 08:05:16 +01:00
mispipe "tools/ci/setup-backend" ts
circleci: Use alias for install dependencies.
2018-12-05 16:53:07 +01:00
circleci: Document the story with hashing the distribution name.
2018-12-05 21:03:06 +01:00
# Note: CircleCI doesn't support any sort of basic variable
# substitution, so we have to do this `_dist.txt` hack where we
# checksum the distro name in the cache path, just because that's
# the only way we can express this without duplicating our cache
# management code.
circleci: Use alias for save cache.
2018-12-05 17:30:08 +01:00
- &save_cache_package_json
save_cache:
paths:
- /srv/zulip-npm-cache
key: v1-npm-base.{{ checksum "_dist.txt" }}-{{ checksum "package.json" }}-{{ checksum "yarn.lock" }}
- &save_cache_requirements
save_cache:
paths:
- /srv/zulip-venv-cache
key: v1-venv-base.{{ checksum "_dist.txt" }}-{{ checksum "requirements/thumbor.txt" }}-{{ checksum "requirements/dev.txt" }}
# TODO: in Travis we also cache ~/zulip-emoji-cache, ~/node, ~/misc
circleci: Use alias for run backend tests.
2018-12-05 16:55:08 +01:00
- &run_backend_tests
run:
name: run backend tests
command: |
. /srv/zulip-py3-venv/bin/activate
tools: Rename tools/travis directory to tools/ci.
2018-12-10 08:05:16 +01:00
mispipe ./tools/ci/backend ts
circleci: Use alias for run backend tests.
2018-12-05 16:55:08 +01:00
circleci: Use alias for run frontend tests.
2018-12-05 16:58:41 +01:00
- &run_frontend_tests
run:
name: run frontend tests
command: |
. /srv/zulip-py3-venv/bin/activate
tools: Rename tools/travis directory to tools/ci.
2018-12-10 08:05:16 +01:00
mispipe ./tools/ci/frontend ts
circleci: Use alias for run frontend tests.
2018-12-05 16:58:41 +01:00
circleci: Use alias for upload coverage report.
2018-12-05 17:00:55 +01:00
- &upload_coverage_report
run:
name: upload coverage report
command: |
. /srv/zulip-py3-venv/bin/activate
pip install codecov && codecov \
|| echo "Error in uploading coverage reports to codecov.io."
circle: Draft a config for running our builds in CircleCI. From just my experience attempting this change, I've been impressed with the very fast turnaround in CircleCI for a change -- from `git push` to the build starting is usually just a couple of seconds. It also looks like we get much clearer control over the caching system, and the setup of the environment via a Docker image of our choice, which can help avoid most of the several minutes Travis takes to set up our environment before it actually starts running the tests themselves. Plus it supports fine-grained parallelism, e.g. by test file: https://circleci.com/docs/2.0/parallelism-faster-jobs/ So we can probably get much faster builds by making this switch. Because the environment is much more transparent and even under our control, I'm also hopeful for more reliable builds.
2017-08-30 02:09:09 +02:00
jobs:
circleci: Run xenial py3.5 backend tests alongside trusty.
2018-01-30 21:20:50 +01:00
"trusty-python-3.4":
circle: Draft a config for running our builds in CircleCI. From just my experience attempting this change, I've been impressed with the very fast turnaround in CircleCI for a change -- from `git push` to the build starting is usually just a couple of seconds. It also looks like we get much clearer control over the caching system, and the setup of the environment via a Docker image of our choice, which can help avoid most of the several minutes Travis takes to set up our environment before it actually starts running the tests themselves. Plus it supports fine-grained parallelism, e.g. by test file: https://circleci.com/docs/2.0/parallelism-faster-jobs/ So we can probably get much faster builds by making this switch. Because the environment is much more transparent and even under our control, I'm also hopeful for more reliable builds.
2017-08-30 02:09:09 +02:00
docker:
circleci: Run xenial py3.5 backend tests alongside trusty.
2018-01-30 21:20:50 +01:00
# This is built from tools/circleci/images/trusty/Dockerfile .
circleci: Bump the Xenial and Trusty base images to fresh builds. This will save us a bit of time running apt upgrades at the start of each build; looks like that's been about 15-20s for recent builds.
2018-06-02 01:24:21 +02:00
- image: gregprice/circleci:trusty-python-5.test
circle: Draft a config for running our builds in CircleCI. From just my experience attempting this change, I've been impressed with the very fast turnaround in CircleCI for a change -- from `git push` to the build starting is usually just a couple of seconds. It also looks like we get much clearer control over the caching system, and the setup of the environment via a Docker image of our choice, which can help avoid most of the several minutes Travis takes to set up our environment before it actually starts running the tests themselves. Plus it supports fine-grained parallelism, e.g. by test file: https://circleci.com/docs/2.0/parallelism-faster-jobs/ So we can probably get much faster builds by making this switch. Because the environment is much more transparent and even under our control, I'm also hopeful for more reliable builds.
2017-08-30 02:09:09 +02:00
circleci: Set working_directory to zulip. Doesn't do much except that when you ssh into circleci you get a directory called zulip instead of repo.
2018-01-30 21:21:29 +01:00
working_directory: ~/zulip
circle: Draft a config for running our builds in CircleCI. From just my experience attempting this change, I've been impressed with the very fast turnaround in CircleCI for a change -- from `git push` to the build starting is usually just a couple of seconds. It also looks like we get much clearer control over the caching system, and the setup of the environment via a Docker image of our choice, which can help avoid most of the several minutes Travis takes to set up our environment before it actually starts running the tests themselves. Plus it supports fine-grained parallelism, e.g. by test file: https://circleci.com/docs/2.0/parallelism-faster-jobs/ So we can probably get much faster builds by making this switch. Because the environment is much more transparent and even under our control, I'm also hopeful for more reliable builds.
2017-08-30 02:09:09 +02:00
steps:
- checkout
circleci: Use alias for create cache directories.
2018-12-05 16:46:25 +01:00
- *create_cache_directories
circleci: Use alias for restore cache.
2018-12-05 17:24:51 +01:00
- run: echo "trusty" > _dist.txt
- *restore_cache_package_json
- *restore_cache_requirements
circle: Draft a config for running our builds in CircleCI. From just my experience attempting this change, I've been impressed with the very fast turnaround in CircleCI for a change -- from `git push` to the build starting is usually just a couple of seconds. It also looks like we get much clearer control over the caching system, and the setup of the environment via a Docker image of our choice, which can help avoid most of the several minutes Travis takes to set up our environment before it actually starts running the tests themselves. Plus it supports fine-grained parallelism, e.g. by test file: https://circleci.com/docs/2.0/parallelism-faster-jobs/ So we can probably get much faster builds by making this switch. Because the environment is much more transparent and even under our control, I'm also hopeful for more reliable builds.
2017-08-30 02:09:09 +02:00
- run:
name: install dependencies
command: |
# Install moreutils so we can use `ts` and `mispipe` in the following.
sudo apt-get install -y moreutils
circleci: Remove Circle-created gitconfig file. [Investigation by hackerkid.]
2017-09-20 02:36:30 +02:00
# CircleCI sets the following in Git config at clone time:
# url.ssh://git@github.com.insteadOf https://github.com
# This breaks the Git clones in the NVM `install.sh` we run
# in `install-node`.
# TODO: figure out why that breaks, and whether we want it.
# (Is it an optimization?)
rm -f /home/circleci/.gitconfig
circle: Draft a config for running our builds in CircleCI. From just my experience attempting this change, I've been impressed with the very fast turnaround in CircleCI for a change -- from `git push` to the build starting is usually just a couple of seconds. It also looks like we get much clearer control over the caching system, and the setup of the environment via a Docker image of our choice, which can help avoid most of the several minutes Travis takes to set up our environment before it actually starts running the tests themselves. Plus it supports fine-grained parallelism, e.g. by test file: https://circleci.com/docs/2.0/parallelism-faster-jobs/ So we can probably get much faster builds by making this switch. Because the environment is much more transparent and even under our control, I'm also hopeful for more reliable builds.
2017-08-30 02:09:09 +02:00
# This is the main setup job for the test suite
tools: Rename tools/travis directory to tools/ci.
2018-12-10 08:05:16 +01:00
mispipe "tools/ci/setup-backend" ts
circle: Draft a config for running our builds in CircleCI. From just my experience attempting this change, I've been impressed with the very fast turnaround in CircleCI for a change -- from `git push` to the build starting is usually just a couple of seconds. It also looks like we get much clearer control over the caching system, and the setup of the environment via a Docker image of our choice, which can help avoid most of the several minutes Travis takes to set up our environment before it actually starts running the tests themselves. Plus it supports fine-grained parallelism, e.g. by test file: https://circleci.com/docs/2.0/parallelism-faster-jobs/ So we can probably get much faster builds by making this switch. Because the environment is much more transparent and even under our control, I'm also hopeful for more reliable builds.
2017-08-30 02:09:09 +02:00
# Cleaning caches is mostly unnecessary in Circle, because
# most builds don't get to write to the cache.
# mispipe "scripts/lib/clean-unused-caches --verbose --threshold 0" ts
circleci: Use alias for save cache.
2018-12-05 17:30:08 +01:00
- *save_cache_package_json
- *save_cache_requirements
circleci: Use alias for run backend tests.
2018-12-05 16:55:08 +01:00
- *run_backend_tests
circleci: Use alias for run frontend tests.
2018-12-05 16:58:41 +01:00
- *run_frontend_tests
circleci: Use alias for upload coverage report.
2018-12-05 17:00:55 +01:00
- *upload_coverage_report
circleci: Upload coverage reports to codecov.io. The Travis config does this by a `pip install codecov` in setup and running the resulting `codecov` command at the end. That risks confusion because it makes the library look like something we might be using *during* the tests to collect coverage information. Instead, install and run codecov together at the end. (The docs also suggest using their bash-based uploader instead; but following those directions seems to result in coverage.xml not getting included (maybe not even built), and no coverage data getting uploaded for Python source files, only JavaScript. Not sure why that is, but using the pip package is fine.) Also, because this step is inherently about talking to a third-party service whose reliability we can't control, and because it's purely a reporting step that happens after our actual test suite has succeeded, don't fail if it fails; just print a message. (I'd prefer a somewhat louder message, like turning that step yellow or something in the Circle web UI, but Circle doesn't offer an option like that. If the coverage reports start consistently failing for some reason, then if we're actually reading them we'll notice anyway.) [greg: Edited step name and error message; wrote this commit message.]
2018-01-26 21:08:13 +01:00
circle: Draft a config for running our builds in CircleCI. From just my experience attempting this change, I've been impressed with the very fast turnaround in CircleCI for a change -- from `git push` to the build starting is usually just a couple of seconds. It also looks like we get much clearer control over the caching system, and the setup of the environment via a Docker image of our choice, which can help avoid most of the several minutes Travis takes to set up our environment before it actually starts running the tests themselves. Plus it supports fine-grained parallelism, e.g. by test file: https://circleci.com/docs/2.0/parallelism-faster-jobs/ So we can probably get much faster builds by making this switch. Because the environment is much more transparent and even under our control, I'm also hopeful for more reliable builds.
2017-08-30 02:09:09 +02:00
# - store_artifacts: # TODO
# path: var/casper/
# # also /tmp/zulip-test-event-log/
# destination: test-reports
circleci: Run xenial py3.5 backend tests alongside trusty.
2018-01-30 21:20:50 +01:00
"xenial-python-3.5":
docker:
# This is built from tools/circleci/images/xenial/Dockerfile .
circleci: Bump the Xenial and Trusty base images to fresh builds. This will save us a bit of time running apt upgrades at the start of each build; looks like that's been about 15-20s for recent builds.
2018-06-02 01:24:21 +02:00
- image: gregprice/circleci:xenial-python-4.test
circleci: Run xenial py3.5 backend tests alongside trusty.
2018-01-30 21:20:50 +01:00
circleci: Set working_directory to zulip. Doesn't do much except that when you ssh into circleci you get a directory called zulip instead of repo.
2018-01-30 21:21:29 +01:00
working_directory: ~/zulip
circleci: Run xenial py3.5 backend tests alongside trusty.
2018-01-30 21:20:50 +01:00
steps:
- checkout
circleci: Use alias for create cache directories.
2018-12-05 16:46:25 +01:00
- *create_cache_directories
circleci: Use alias for restore cache.
2018-12-05 17:24:51 +01:00
- run: echo "xenial" > _dist.txt
- *restore_cache_package_json
- *restore_cache_requirements
circleci: Use alias for install dependencies.
2018-12-05 16:53:07 +01:00
- *install_dependencies
circleci: Use alias for save cache.
2018-12-05 17:30:08 +01:00
- *save_cache_package_json
- *save_cache_requirements
circleci: Use alias for run backend tests.
2018-12-05 16:55:08 +01:00
- *run_backend_tests
circleci: Use alias for upload coverage report.
2018-12-05 17:00:55 +01:00
- *upload_coverage_report
circleci: Upload to codecov on xenial too. Only takes about 11 seconds; and there may be some codepaths that are exercised on one release and not the other.
2018-01-31 18:41:05 +01:00
circleci: Add support for Ubuntu Bionic.
2018-05-30 18:46:08 +02:00
"bionic-python-3.6":
docker:
# This is built from tools/circleci/images/bionic/Dockerfile .
- image: gregprice/circleci:bionic-python-1.test
working_directory: ~/zulip
steps:
- checkout
circleci: Use alias for create cache directories.
2018-12-05 16:46:25 +01:00
- *create_cache_directories
circleci: Add support for Ubuntu Bionic.
2018-05-30 18:46:08 +02:00
- run:
circleci: Use alias for create cache directories.
2018-12-05 16:46:25 +01:00
name: do Bionic hack
circleci: Add support for Ubuntu Bionic.
2018-05-30 18:46:08 +02:00
command: |
# Temporary hack till `sudo service redis-server start` gets fixes in Bionic. See
# https://chat.zulip.org/#narrow/stream/3-backend/topic/Ubuntu.20bionic.20CircleCI
redis-server --daemonize yes
circleci: Use alias for restore cache.
2018-12-05 17:24:51 +01:00
- run: echo "bionic" > _dist.txt
- *restore_cache_package_json
- *restore_cache_requirements
circleci: Use alias for install dependencies.
2018-12-05 16:53:07 +01:00
- *install_dependencies
circleci: Use alias for save cache.
2018-12-05 17:30:08 +01:00
- *save_cache_package_json
- *save_cache_requirements
circleci: Use alias for run backend tests.
2018-12-05 16:55:08 +01:00
- *run_backend_tests
circleci: Use alias for upload coverage report.
2018-12-05 17:00:55 +01:00
- *upload_coverage_report
circleci: Add support for Ubuntu Bionic.
2018-05-30 18:46:08 +02:00
circleci: Run xenial py3.5 backend tests alongside trusty.
2018-01-30 21:20:50 +01:00
workflows:
version: 2
build:
jobs:
- "trusty-python-3.4"
- "xenial-python-3.5"
circleci: Add support for Ubuntu Bionic.
2018-05-30 18:46:08 +02:00
- "bionic-python-3.6"