Commit Graph

84 Commits

Author SHA1 Message Date
arpit551 e19aef569a ci: Update Circle CI images.
Removed postgresql and python 2 packages when creating docker images
for Circle CI. Updated the images.
2020-06-24 11:19:47 -07:00
Alex Vandiver 2283a16476
ci: Only attempt to post the failure if we have a key.
Only some branches define the `ZULIP_BOT_KEY`; others, including PRs,
thus currently fail to POST:

    {"result":"error","msg":"Malformed API key","code":"INVALID_API_KEY"}

Simply abort early, without attempting to report the result, if the
key is not present.
2020-06-23 16:43:18 -07:00
Aman Agrawal 6950d8d769 provision: Don't run migrations on `zulip` db in CircleCI.
The automated tests running in CircleCI don't actually use the `zulip`
db, so we can skip running migrations on it in some CircleCI shards to
save time.

NOTE: This only effects build jobs that run provision, except the
`production-build` job where we skip building the dbs altogether.
Migrations still run on `focal-backend` build job to ensure
we are testing all our development setup code.
2020-06-16 15:37:32 +05:30
Anders Kaseorg 69a2ad02cd circleci: Send stderr through ts too.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 16:49:32 -07:00
arpit551 662f4902f8 ci: Setup production job for Focal.
Run production suites on Ubuntu Focal.
Added separate success-http-headers files for Focal and Bionic.
Also excluded them from whitespace rules in lint.

memcached 1.5.22 in Ubuntu 20.04 has a bug where it looks for its SASL
configuration at /etc/sasl2/memcached.conf/memcached.conf instead of
/etc/sasl2/memcached.conf.
We already use a workaround for this while applying puppet configurations in
99e71f3786 but for docker builds we used
do memcached hack since we can not use systemd in docker containers.
2020-06-08 21:59:57 -07:00
Anders Kaseorg ad24eef6ee circleci: Drop -python3.X suffixes from job names.
The workflow names should be sufficient.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 11:04:40 -07:00
Anders Kaseorg fcfe8eb49b circleci: Split into four workflows.
This should hopefully give better status output with GitHub Checks
integration enabled.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 11:04:40 -07:00
Priyank Patel 99bed7b702 circleci: Store puppeteer screenshots in artifacts. 2020-06-03 17:39:35 -07:00
Aman Agrawal c9bff71410 ci: Don't remove .coverage file & set empty suffix.
Fixes #14962
* codecov needs `.coverage` file in the pwd to upload coverage
results.
* concurrency='multiprocessing' forces `.coverage` file to have a
data_suffix, we explicity set it to "" for the file to have no suffix.
Filed issue as https://github.com/nedbat/coveragepy/issues/989
2020-05-27 17:54:54 -07:00
Aman Agrawal f4e70714b4 ci: Pin codecov version to avoid nondeterministic behaviour.
* This fixes the gov find error when dicovering files.
From v2.0.16, codecov-python uses "find" to discover files which
is buggy on some platforms.
Filed issue as https://github.com/codecov/codecov-python/issues/250
2020-05-20 09:47:51 -07:00
arpit551 1427270df4 ci: Correct notify_failure_status in circleci config. 2020-05-14 11:58:26 -07:00
arpit551 f3ab8e66dc ci: add configuration for posting to czo whenever CI fails in master.
Since notify is mostly depreciated  in the latest versions of CircleCI.
Although we can use use notify in CircleCI 2.0 but currently
there is no documentation regarding it.

We could have use notify here rather than this hacky solution but
if we use notify it was not possible to trigger CircleCI webhook only
for the main Zulip repository.

Also corrected the circle ci webhook for the case where we don't receive
previous in post request
2020-05-11 13:45:22 -07:00
Anders Kaseorg 80caaaae65 circleci: Remove legacy check_xenial_puppet step.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-08 16:42:43 -07:00
Aman Agrawal 7728611bf4 ci: bugfix: Don't restore emoji cache on prod.
The emoji cache files are not available on production jobs. Also,
copying the files, creating and restoring caches doesn't add
much optimization over just rebuilding the caches here.
2020-04-30 10:52:50 -07:00
Aman Agrawal 6069d04696 ci: Save and restore zulip-emoji-caches on all jobs. 2020-04-29 10:54:53 -07:00
Aman Agrawal c2b8f8f7ed ci: Add run scripts to save and restore emoji cache. 2020-04-29 10:54:53 -07:00
Aman Agrawal 01b267dc0a ci: Remove old TODO statement. 2020-04-29 10:54:53 -07:00
Aman Agrawal 7e8ba97d85 ci: Add focal production install job. 2020-04-29 10:54:53 -07:00
Aman Agrawal 1324e7798c ci: Directly run production install script. 2020-04-28 11:11:00 -07:00
Aman Agrawal b5e31c04c9 ci: Rename production install script. 2020-04-28 11:11:00 -07:00
Aman Agrawal d22e26ce26 ci: Rename production build job. 2020-04-28 11:11:00 -07:00
Aman Agrawal 70f7d04bf1 ci: Rename production builder script file. 2020-04-28 11:11:00 -07:00
Aman Agrawal 8a841a5634 ci: Update bionic docker images to save upgrade time in builds. 2020-04-25 16:10:55 -07:00
Aman Agrawal cf14cc211f ci: Separate prod tarball extraction and install process.
This allows to run scripts between extraction and install
process.

It will be used to restore npm caches for production install jobs.

We extract the tarball in the working directory so that yarn.lock and
package.json are available to restore cache.

(And also so the path is deterministic).
2020-04-25 16:10:41 -07:00
Aman Agrawal 0aa18e88a5 ci: Stop cloning repo as it is not required for prod install.
Move prod installer and test files to persistent dir so that
they can be utilized.
2020-04-25 16:07:55 -07:00
Aman Agrawal a015160ec0 ci: Fix restoring and and saving npm caches. 2020-04-25 16:07:48 -07:00
Aman Agrawal ad07814fa4 ci: Run production build and install jobs in sequence. 2020-04-24 12:40:31 -07:00
Aman Agrawal 0e544ec40b ci: Add job to run production script to tests production install.
Only restore npm cache, venv cache not required.
Restart rabbitmq-server for production install.
2020-04-24 12:40:31 -07:00
Aman Agrawal 7c99555107 ci: Copy https headers to persistent dir for comparison. 2020-04-24 12:40:31 -07:00
Aman Agrawal 962af445e9 ci: Add job to build release tarball and save it to be reused.
The tarball will be persistent across jobs in /tmp directory.
We will use this tarball to install production server in a separate
job. This makes sure we are running the production installation
as a normal user would run it and stop any package leaks between
provision and production installation scripts.
2020-04-24 12:40:31 -07:00
Aman Agrawal 77b24d056a ci: Remove old bionic production build job. 2020-04-24 12:40:10 -07:00
Anders Kaseorg 7fd4ba8cbe circleci: Test parsing PostgreSQL scripts on Ubuntu 16.04.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-23 17:27:27 -07:00
Anders Kaseorg 9d9ae735c0 circleci: Test zulip-puppet-apply on Ubuntu 16.04.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-22 16:16:56 -07:00
arpit551 12c83141ed ci: Run Focal CI job.
Uncomment Focal CI job and fix its name.
2020-04-22 16:10:59 -07:00
arpit551 aabad1c0bb test: run test-locked-requirements only for Bionic.
Currently our compiled requirements files will differ for different
python versions. So we will run test-locked-requirements only for Bionic.
2020-04-22 16:10:59 -07:00
Anders Kaseorg 3ac9982411 circleci: Test provision and upgrade error messages on Ubuntu 16.04.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-22 14:39:32 -07:00
arpit551 88d253e8f2 ci: Add bionic production job in Circle CI.
We now replaced our production suites to run on circle ci
that too on bionic docker image.
2020-04-21 14:46:40 -07:00
arpit551 0121e82d2c ci: Fixed saving cache bug in Circle CI jobs.
While saving cache we used thumbor.txt in as key but while
restoring cache we used thumbor-dev.txt as the key so we could
never restore zulip-venv-cache in ci builds.
2020-04-21 14:46:40 -07:00
Aman Agrawal b13beb5ea8 circleci: Remove Xenial build as we are deprecating Xenial.
Docs updated to reflect the change.
2020-04-19 11:38:49 -07:00
Aman Agrawal 227762c6ed circleci: Run frontend tests on bionic and upload coverage.
This is preparation for dropping production support for Ubuntu Xenial.
2020-04-19 11:35:49 -07:00
Anders Kaseorg ba16b37f8d circleci: Update venv cache prefix based on thumbor-dev.txt.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-05 13:45:25 -07:00
arpit551 eb294238c9 circleci: Comment out focal backend-frontend job.
Comment out any Focal jobs in circleci config file.
This is due to a recent bug that is causing some trouble
while installing python-pip in Focal.
So we are pausing Focal tests until we get stability.
2020-03-26 13:11:48 -07:00
arpit551 f23a435fcb circleci: Add base image for Ubuntu Focal.
Added base image for Focal and new job in config.yml
to run backend and frontend tests in circleci for
python3.8 using Focal docker image.
2020-03-25 16:09:59 -07:00
arpit551 6c1f1e12fe yml: Fix trailing whitespace in .circleci/config.yml.
Removed trailing whitespace in .circleci/config.yml.
2020-03-24 15:31:59 -07:00
Vishnu Ks d01ca8a390 circleci: Store XUnit test results.
Fixes part of #8593.
2019-07-07 22:31:11 -07:00
Wyatt Hoodes 2f8e1c0393 circleci: Store var/casper/ and zulip-test-event-log as artifacts. 2019-07-02 15:19:30 -07:00
Anders Kaseorg bfa82a70c8 ci: Remove Ubuntu 14.04 (trusty) from CI.
Tweaked by tabbott to improve our CI docs discussion of these.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-06-26 11:01:13 -07:00
Anders Kaseorg ae524b677d ci: Move backend and production tests to Ubuntu 16.04 (xenial).
This is preparation for dropping support for Trusty in CI.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-24 17:07:15 -07:00
Tim Abbott c2baca2c78 ci: Don't run codecov on additional platforms.
The coverage data for the platform where we run the backend+frontend
tests should be complete, so there's no reason to upload coverage data
for the other platforms.

Hopefully this change will decrease the likelihood of codecov
producing flaky results.
2019-05-20 11:02:58 -07:00
Mateusz Mandera 63c170c390 circleci: Fix redis on Bionic.
Fixes #11079.
Because of "service redis-server start" not working properly, in "do
Bionic hack" another instance of the server was being started. However,
once in a while this would fail for unknown reasons causing flakes in
tests involving redis. The cause of the service not starting was the
error:
::1:6379: bind: Cannot assign requested address
(see https://github.com/antirez/redis/issues/3241)
This is fixed by editing /etc/redis/redis.conf to not attempt binding
on IPv6. We change "do Bionic hack" to a simple sed command,
changing the appropriate line in the file.
By default, the line is "bind 127.0.0.1 ::1".
We change it to "bind 0.0.0.0".
2019-05-03 11:33:39 -07:00