diff --git a/docs/contributing/code-reviewing.md b/docs/contributing/code-reviewing.md index 2ced64ea80..f3bcb299e5 100644 --- a/docs/contributing/code-reviewing.md +++ b/docs/contributing/code-reviewing.md @@ -72,8 +72,8 @@ this?". Good choices include to the commit hash or the Detail links on a pull request. (Example: in [#10618](https://github.com/zulip/zulip/pull/10618), browse to bottom and click the red X next to `c6044ee` to see the build jobs - for that commit. You can see that there are 4 build jobs in total. - 3 are by CircleCI and 1 is by Travis CI. You can see what caused + for that commit. You can see that there are 5 build jobs in total. + All the 5 jobs run in Circle CI. You can see what caused the job to fail by clicking on the failed job. This will open up a page in the CI that has more details on why the job failed. For example [this](https://circleci.com/gh/zulip/zulip/16617) diff --git a/docs/contributing/zulipbot-usage.md b/docs/contributing/zulipbot-usage.md index 3cb2da0bce..f8bedefdee 100644 --- a/docs/contributing/zulipbot-usage.md +++ b/docs/contributing/zulipbot-usage.md @@ -85,12 +85,6 @@ assignee(s) if they are still working on the issue. issue's current assignee(s) and the "in progress" label to allow others to work on an inactive issue. -* **Receive Travis build status notifications** — If you would like to receive -a notification whenever the build status of your pull request is updated, label -your pull request with the "travis updates" label using the command `@zulipbot -label "travis updates"`, and **@zulipbot** will let you know the build status -(e.g. passed, failed, errored) of your pull request once all tests finish. - ### Contributing If you wish to help develop and contribute to **@zulipbot**, check out the diff --git a/docs/git/cloning.md b/docs/git/cloning.md index f39e9eae8f..0442ed4b6b 100644 --- a/docs/git/cloning.md +++ b/docs/git/cloning.md @@ -104,14 +104,13 @@ first-time contributors][zulip-rtd-dev-first-time]. This step is optional, but recommended. The Zulip Server project is configured to use [Circle CI][circle-ci] -and [Travis CI][travis-ci] to test and create builds upon each new commit -and pull request. CircleCI is the primary CI that runs frontend and backend -tests across a wide range of Ubuntu distributions. Travis CI is used only for -running the end-to-end production installer test. +to test and create builds upon each new commit and pull request. +CircleCI is the primary CI that runs frontend and backend +tests across a wide range of Ubuntu distributions. -CircleCI and Travis CI are free for open source projects and it's easy to -configure for your own fork of Zulip. After doing so, CircleCI and Travis -CI will run tests for new refs you push to GitHub and email you the outcome +CircleCI is free for open source projects and it's easy to +configure for your own fork of Zulip. After doing so, CircleCI +will run tests for new refs you push to GitHub and email you the outcome (you can also view the results in the web interface). Running CI against your fork can help save both your and the @@ -120,8 +119,8 @@ submitting a pull request. We generally recommend a workflow where as you make changes, you use a fast edit-refresh cycle running individual tests locally until your changes work. But then once you've gotten the tests you'd expect to be relevant to your changes working, push a -branch to run the full test suite in CircleCI and Travis CI before -you create a pull request. While you wait for CircleCI and Travis CI +branch to run the full test suite in CircleCI before +you create a pull request. While you wait for Circle CI jobs to run, you can start working on your next task. When the tests finish, you can create a pull request that you already know passes the tests. @@ -133,15 +132,6 @@ in click on **Add Projects** in right sidebar. This will list all your GitHub repositories. Now goto the row of Zulip and click on **Set Up Project**. ![Screencast of CircleCI setup](../images/zulip-circleci.gif) -### Setup Travis CI - -First, sign in to [Travis CI][travis-ci] with your GitHub account and authorize -Travis CI to access your GitHub account and repositories. Once you've done -this, Travis CI will fetch your repository information and display it on your -[profile page][travis-ci-profile]. From there you can enable integration with -Zulip. -![Screencast of Travis CI setup](../_static/zulip-travisci.gif) - [gitbook-rebase]: https://git-scm.com/book/en/v2/Git-Branching-Rebasing [github-help-add-ssh-key]: https://help.github.com/en/articles/adding-a-new-ssh-key-to-your-github-account [github-help-conf-remote]: https://help.github.com/en/articles/configuring-a-remote-for-a-fork @@ -149,9 +139,7 @@ Zulip. [github-help-sync-fork]: https://help.github.com/en/articles/syncing-a-fork [github-zulip]: https://github.com/zulip/ [github-zulip-zulip]: https://github.com/zulip/zulip/ -[travis-ci]: https://travis-ci.org/ [circle-ci]:https://circleci.com/ -[travis-ci-profile]: https://travis-ci.org/profile [zulip-rtd-dev-first-time]: ../development/setup-vagrant.md [zulip-rtd-dev-overview]: ../development/overview.md [zulip-rtd-tools-setup]: ../git/zulip-tools.html#set-up-git-repo-script diff --git a/docs/git/using.md b/docs/git/using.md index ca42f66ff0..d8fd882b83 100644 --- a/docs/git/using.md +++ b/docs/git/using.md @@ -125,7 +125,7 @@ Now you're ready to work on the issue or feature. ## Run linters and tests locally -In addition to having Travis run tests and linters each time you push a new +In addition to having Circle CI run tests and linters each time you push a new commit, you can also run them locally. See [testing](../testing/testing.md) for details. ## Stage changes diff --git a/docs/testing/continuous-integration.md b/docs/testing/continuous-integration.md index 1fb36bca4a..a7e523de80 100644 --- a/docs/testing/continuous-integration.md +++ b/docs/testing/continuous-integration.md @@ -1,12 +1,9 @@ # Continuous integration (CI) -The Zulip server uses [CircleCI](https://circleci.com/) and -[Travis CI](https://travis-ci.org/) for continuous -integration. CircleCI is the primary CI, and runs frontend and backend -tests across a wide range of Ubuntu distributions. Travis CI is -legacy, used only for running the end-to-end production installer -test. This page documents useful tools and tips to know about when -using CircleCI and Travis CI and debugging issues with them. +The Zulip server uses [CircleCI](https://circleci.com/) for continuous +integration. CircleCI runs frontend, backend and end-to-end production +installer tests. This page documents useful tools and tips when using +CircleCI and debugging issues with it. ## Goals @@ -29,9 +26,9 @@ run to iteratively debug something. ### Useful debugging tips and tools -* Zulip uses the `ts` tool to log the current time on every line of the output in -our Travis CI scripts. You can use this output to determine which steps are -actually consuming a lot of time. +* Zulip uses the `ts` tool to log the current time on every line of +the output in our CircleCI scripts. You can use this output to +determine which steps are actually consuming a lot of time. * You can [sign up your personal repo for CircleCI][circleci-setup] so that every remote branch you push will be tested, which can be helpful @@ -50,18 +47,36 @@ uses those SSH keys for authentication. ### Suites -The main CircleCI configuration file is -[./circleci/config.yml](https://github.com/zulip/zulip/blob/master/.circleci/config.yml). -We run multiple jobs during a CircleCI build to test different -supported platforms. They are currently: +The main CircleCI configuration file defining how the tests are run is +[./circleci/config.yml][circleci-config]. Our code for running the +tests in CI lives under `tools/ci`; but they are mostly thin wrappers +around [Zulip's test suites](../testing/testing.md) or production +installer tooling. -* bionic-backend-python3.6 +[circleci-config]: https://github.com/zulip/zulip/blob/master/.circleci/config.yml + +We run multiple jobs during a CircleCI build to run Zulip's test +suites on our supported production platforms. They are currently: + +* bionic-backend-frontend-python3.6 +* focal-backend-python3.8 Each runs the Zulip backend test suites, using the indicated platform/OS and Python version. As suggested by the names, only one suite runs the frontend test suites, since those are not platform-dependent. +Additionally, there a couple jobs designed to do an end-to-end test on +Zulip's production installer: + +* bionic-production-build-python3.6 +* bionic-production-install-python3.6 + +The `production-build` job builds a Zulip release tarball, which is +then installed in a fresh container in the `production-install` job; +various Nagios and other checks are run to confirm the installation +worked. + ### Configuration The remaining details in this section are primarily relevant for doing @@ -98,9 +113,10 @@ generated Dockerfiles. #### Caching -An important element of making CircleCI perform effectively is -caching the provisioning of a Zulip development environment. In -particular, we cache the following.: +An important element of making CircleCI perform effectively is caching +between jobs the various caches that live under `/srv/` in a Zulip +development or production environment. In particular, we cache the +following: * Python virtualenvs * node_modules directories @@ -120,85 +136,3 @@ A consequence of this caching is that test jobs for branches which modify `package.json`, `requirements/`, and other key dependencies will be significantly slower than normal, because they won't get to benefit from the cache. - -## Travis CI - -### Configuration - -The main Travis configuration file is -[.travis.yml](https://github.com/zulip/zulip/blob/master/.travis.yml). -The specific test suites we have are listed in the `matrix` section, -which has a matrix of Python versions and test suites (`$TEST_SUITE`). - -Currently there is only the production test suite in this section as we -have moved the backend and frontend suite to CircleCI. So the value of -the variable `$TEST_SUITE` would be always `production`. - -We've configured it to use a few helper scripts for each job: - -* `tools/ci/setup-$TEST_SUITE`: This script sets up the test - environment for the production suite. This is a complicated process - because of all the packages Travis installs. See the comments in - `tools/ci/setup-production` for details. -* `tools/ci/$TEST_SUITE`: The script that runs the actual test - production test suite. - -The main purpose of the distinction between the two is that if the -`setup-production` job fails, Travis CI will report it as the suite -having "Errored" (grey in their emails), whereas if the `production` job -fails, it'll be reported as "Failed" failure (red in their emails). -Note that Travis CI's web UI seems to make no visual distinction -between these. - -An important detail is that Travis CI will by default hide most phases -other than the actual test; you can see this easily by looking at the -line numbers in the Travis CI output. There are actually a bunch of -phases (e.g. the project's setup job, downloading caches near the -beginning, uploading caches at the end, etc.), and if you're debugging -our configuration, you'll want to look at these closely. - -### Useful debugging tips and tools - -* Zulip uses the `ts` tool to log the current time on every line of - the output in our Travis CI scripts. You can use this output to - determine which steps are actually consuming a lot of time. - -* For performance issues, - [this statistics tool](https://scribu.github.io/travis-stats/#zulip/zulip/master) - can give you test runtime history data that can help with - determining when a performance issue was introduced and whether it - was fixed. Note you need to click the "Run" button for it to do - anything. - -* You can [sign up your personal repo for Travis CI][travis-fork] so - that every remote branch you push will be tested, which can be - helpful when debugging something complicated. - -[travis-fork]: ../git/cloning.html#step-3-configure-continuous-integration-for-your-fork - -### Performance optimizations - -#### Caching - -We cache the following as well apart from what is mentioned in CircleCI -caching section. - -* Built/downloaded emoji sprite sheets and data. - -This is probably worth eventually adding to the CircleCI caches, but -because it only saves ~5s, it hasn't been a priority yet. - -#### Uninstalling packages - -In the production suite, we run `apt-get upgrade` at some point -(effectively, because the Zulip installer does). This carries a huge -performance cost in Travis CI, because (1) they don't keep their test -systems up to date and (2) literally everything is installed in their -build workers (e.g. several copies of Postgres, Java, MySQL, etc.). - -In order to make Zulip's tests performance reasonably well, we -uninstall (or mark with `apt-mark hold`) many of these dependencies -that are irrelevant to Zulip in -[`tools/ci/setup-production`][setup-production]. - -[setup-production]: https://github.com/zulip/zulip/blob/master/tools/ci/setup-production diff --git a/docs/testing/linters.md b/docs/testing/linters.md index 2eee1e9d57..8651ed1417 100644 --- a/docs/testing/linters.md +++ b/docs/testing/linters.md @@ -48,7 +48,7 @@ note are: * `--skip` and `--only`: Only run certain linters. * `-m`: Only check modified files. -Finally, you can rely on our Travis CI setup to run linters for you, but +Finally, you can rely on our Circle CI setup to run linters for you, but it is good practice to run lint checks locally. ```eval_rst