While saving cache we used thumbor.txt in as key but while
restoring cache we used thumbor-dev.txt as the key so we could
never restore zulip-venv-cache in ci builds.
Comment out any Focal jobs in circleci config file.
This is due to a recent bug that is causing some trouble
while installing python-pip in Focal.
So we are pausing Focal tests until we get stability.
The coverage data for the platform where we run the backend+frontend
tests should be complete, so there's no reason to upload coverage data
for the other platforms.
Hopefully this change will decrease the likelihood of codecov
producing flaky results.
Fixes#11079.
Because of "service redis-server start" not working properly, in "do
Bionic hack" another instance of the server was being started. However,
once in a while this would fail for unknown reasons causing flakes in
tests involving redis. The cause of the service not starting was the
error:
::1:6379: bind: Cannot assign requested address
(see https://github.com/antirez/redis/issues/3241)
This is fixed by editing /etc/redis/redis.conf to not attempt binding
on IPv6. We change "do Bionic hack" to a simple sed command,
changing the appropriate line in the file.
By default, the line is "bind 127.0.0.1 ::1".
We change it to "bind 0.0.0.0".
We initially upgraded to 2.1 under the assumption that 2.1 was necessary
to enable pipelines. This turns out to be not true. Pipelines can be enabled
from 2.0 as well. On the other hand upgrading to 2.1 requires enabling
pipelines. This resulted in broken builds across many forks that has enabled
CircleCI and not pipelines. We are also not making use of any 2.1 features
like reusable orbs, commands, and executors currently. Downgrading to 2.0 will
also allow us to use "." in the job names which makes them more readable.
This contains the minimum changes required to move to the CircleCI 2.1
configuration file format. In 2.1 job names should not contain "." in
them.
Moving to 2.1 will allow us to enable enable pipelines from CircleCI
project settings UI. Once pipeline is enabled auto-cancellation of
redundant workflows can be also enabled from the CircleCI UI.
See https://circleci.com/docs/2.0/build-processing/
Caches in Circle are immutable -- even if a path in a cache changes in builds
after the cache was created, the cache is not updated if it already exists. This
was making the zulip-venv-cache and zulip-npm-cache directories useless on
Circle.
This commit changes the cache keys to depend on the checksums of the dependency
manifests (requirements/{dev,thumbor}.txt, package.json and yarn.lock). This
would ensure that the caches are updated when the environments change. It may
result in the occasional build being "uncached" -- when a dependency manifest
changes -- but builds without such changes will be much faster, and such builds
are a majority.
node -> v8.9.4
yarn -> 1.5.1
nvm -> 0.33.8
Also updates a test in timerender.js which depends on time
provided by node which is now changed in newer release.
Some changes have been made in circeci script, we just create ~/.config
directory and chown it to circleci user so installing new version of yarn
does not cause any ci failure on circleci during provision.
We haven't actually been using this zulipbot feature for a long time
(haven't applied the `travis updates` label which it looks for.)
Rather than port the config to CircleCI, just cut it for now; we can
always add something like it back if we start using it.
The Travis config does this by a `pip install codecov` in setup and
running the resulting `codecov` command at the end. That risks
confusion because it makes the library look like something we might
be using *during* the tests to collect coverage information.
Instead, install and run codecov together at the end.
(The docs also suggest using their bash-based uploader instead;
but following those directions seems to result in coverage.xml
not getting included (maybe not even built), and no coverage data
getting uploaded for Python source files, only JavaScript. Not
sure why that is, but using the pip package is fine.)
Also, because this step is inherently about talking to a
third-party service whose reliability we can't control, and because
it's purely a reporting step that happens after our actual test
suite has succeeded, don't fail if it fails; just print a message.
(I'd prefer a somewhat louder message, like turning that step
yellow or something in the Circle web UI, but Circle doesn't offer
an option like that. If the coverage reports start consistently
failing for some reason, then if we're actually reading them we'll
notice anyway.)
[greg: Edited step name and error message; wrote this commit message.]
This new base image just comes from me doing a fresh `docker build`
and uploading the result. That gives it fresher versions of our
APT dependencies than the image I built a couple of months ago,
speeding up provision by giving it less to install.
From just my experience attempting this change, I've been impressed
with the very fast turnaround in CircleCI for a change -- from `git
push` to the build starting is usually just a couple of seconds.
It also looks like we get much clearer control over the caching
system, and the setup of the environment via a Docker image of our
choice, which can help avoid most of the several minutes Travis takes
to set up our environment before it actually starts running the tests
themselves.
Plus it supports fine-grained parallelism, e.g. by test file:
https://circleci.com/docs/2.0/parallelism-faster-jobs/
So we can probably get much faster builds by making this switch.
Because the environment is much more transparent and even under
our control, I'm also hopeful for more reliable builds.
arpit551
Aman Agrawal
Anders Kaseorg
Vishnu Ks
Wyatt Hoodes
Tim Abbott
Mateusz Mandera
rht
Greg Price
Puneeth Chaganti
Priyank
Greg Price