Commit Graph

70 Commits

Author SHA1 Message Date
Alex Vandiver 150d782d6b production-verify: Fix puppet path. 2020-10-06 17:26:14 -07:00
Alex Vandiver 8cf37a0d4b queue: Add a tool to profile no-op enqueue and dequeue actions. 2020-10-06 17:26:14 -07:00
Anders Kaseorg d751e0cece puppet: Don’t install netcat.
It’s been unused since commit 0af22dad18
(#13239).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 10:33:47 -07:00
Dinesh 2d22d88700 casper: Remove few traces of casper.
Now that all casper tests have been migrated to
puppeteer, there's no need for having casper
related things.

Removed the casperjs package and removed/replaced
casper in few places with puppeteer.

Only removed few of them which I'm confident
about. Also didn't make any changes in docs
as it would be easier to remove them while
adding puppeteer docs.
2020-08-30 17:16:02 -07:00
Tim Abbott dbf3894c49 test-backend: Enable --ban-console-output and document. 2020-08-27 11:39:53 -07:00
Anders Kaseorg dbdf67301b memcached: Switch from pylibmc to python-binary-memcached.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-06 12:51:14 -07:00
Priyank Patel 9504d403b3 github-actions: Fix incorrect hash key in production install job.
The hash keys were missing hash for package.json and yarn.lock
because they were not present since we don't do a full checkout
in this job. We fix this by sending over those files and generating
hashes from them.

I usally verify these cache keys by clicking the Restore <cache>
step dropdown menu and then clicking the Run ... dropdown menu again
to see the generated hash.
2020-08-03 12:37:20 -07:00
Priyank Patel dda7e938f4 puppeteer: Fix flake with compose box input.
There were a lots of flakes in CI recently because typeahead didn't
appear when Enter was pressed and real emails are not accepted as
valid inputs. To fix this we wait for typeahead to appear and then
click that instead of Enter. We also use delay option to type the
email (100ms delay between keypresses) since without we'd also get
flakes.

Re-enable puppeteer test in CI after this fix too.
2020-07-30 11:32:32 -07:00
Tim Abbott cee7e8e89d ci: Disable puppeteer tests. 2020-07-28 17:12:49 -07:00
Steve Howell da8691d7d7 ci: Add new check-node-fixtures tool to CI.
This tool only takes a couple seconds to run,
so it's pretty low risk to add to CI.
2020-07-26 16:07:40 -07:00
Priyank Patel 51199878ed tools/ci: Fix KeyError when running tools/ci/generate-dockerfiles.
This KeyError occurs because we remove extra_packages from
images.yml because it was no longer needed in a486872a8e.
2020-07-26 16:03:35 -07:00
Priyank Patel 6a4792e844 tools/ci: Adjust ZULIP_PATH so it works for both CIs.
Until, we remove the circleci.
2020-07-24 10:36:55 -07:00
Priyank Patel 75ab26771b ci: Store production built assets to /tmp/production-build.
Previously, we copied them to /tmp and from there we specified those
assets we copied in circleci config in presist_to_workspace step.
Copying it to a directory allows us to get rid of list in circleci
config and GitHub Actions's upload artifact (their version of
presist to workspace) doesn't allow us to specify indivivual files
so only is this cleaner but required.
2020-07-24 10:36:55 -07:00
Mohit Gupta 308643cace tests: Rename test_narrow.py to test_message_fetch.py.
This commit renames test_narrow.py and also all its references to
test_message_fetch.py.
2020-07-07 17:14:10 -07:00
arpit551 4f6cd6577c http_headers: Substitute Nginx version based on OS.
success-http-headers-bionic.txt and success-http-headers-focal.txt
differ only in the nginx version so this substitution will allow
us to have single file for both of them. Also this change helps
to avoid CI failure if Nginx version is updated in the OS.
2020-07-07 11:20:05 -07:00
Priyank Patel 29dd0c485a ci: Move upgrading git for bionic to docker image.
Just reduces the clutter in configuration file for GitHub Actions.
2020-07-06 18:09:15 -07:00
Priyank Patel c908278237 ci: Install moreutils in the docker image. 2020-07-06 18:09:15 -07:00
Aman Agrawal a486872a8e requirements: Upgrade Thumbor to 7.0.0a5 on Python 3.
Co-authored-by: Anders Kaseorg <anders@zulip.com>
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-06 16:09:53 -07:00
arpit551 834abefdb9 http_headers: Update Nginx version in success-http-headers-focal. 2020-07-02 22:42:22 +05:30
Anders Kaseorg fa89d1b266 generate-self-signed-cert: Correct subjectAltName for an IP address.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-29 22:19:47 -07:00
Alex Vandiver 2540b678b0 ci: Test upgrading PostgreSQL 10 -> 12. 2020-06-29 17:18:47 -07:00
Alex Vandiver f621e1b7e4 ci: Split out installation testing. 2020-06-29 17:18:47 -07:00
Alex Vandiver 36335ecd39 ci: Simplify tool-copying in production-build. 2020-06-29 17:18:47 -07:00
Priyank Patel 92f98292c1 tools/ci: Create a github user instead of a circleci user.
While we are making this change, we create a ARG USERNAME variable
that behaves like ENV but just for the lifetime of the Dockerfile.
2020-06-29 16:31:43 -07:00
Priyank Patel a2c2c6ceff tools: Move CI docker images files into tools/ci. 2020-06-29 16:31:43 -07:00
Aman Agrawal 6950d8d769 provision: Don't run migrations on `zulip` db in CircleCI.
The automated tests running in CircleCI don't actually use the `zulip`
db, so we can skip running migrations on it in some CircleCI shards to
save time.

NOTE: This only effects build jobs that run provision, except the
`production-build` job where we skip building the dbs altogether.
Migrations still run on `focal-backend` build job to ensure
we are testing all our development setup code.
2020-06-16 15:37:32 +05:30
Anders Kaseorg 80b991fc44 production-helper: Clarify log output on check_rabbitmq_consumers failure.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 16:49:32 -07:00
arpit551 662f4902f8 ci: Setup production job for Focal.
Run production suites on Ubuntu Focal.
Added separate success-http-headers files for Focal and Bionic.
Also excluded them from whitespace rules in lint.

memcached 1.5.22 in Ubuntu 20.04 has a bug where it looks for its SASL
configuration at /etc/sasl2/memcached.conf/memcached.conf instead of
/etc/sasl2/memcached.conf.
We already use a workaround for this while applying puppet configurations in
99e71f3786 but for docker builds we used
do memcached hack since we can not use systemd in docker containers.
2020-06-08 21:59:57 -07:00
arpit551 291bce93d0 provision: Rename --production-test-suite option in provision.
Since we use this option in our docker-zulip project also
so rather than using it as a test suite option we made it
more specific i.e. --build-release-tarball-only.
2020-06-04 14:26:23 -07:00
Dinesh d429b7145e tests: Make puppeteer tests run in full suites and CircleCI. 2020-06-03 17:39:35 -07:00
Aman Agrawal c9bff71410 ci: Don't remove .coverage file & set empty suffix.
Fixes #14962
* codecov needs `.coverage` file in the pwd to upload coverage
results.
* concurrency='multiprocessing' forces `.coverage` file to have a
data_suffix, we explicity set it to "" for the file to have no suffix.
Filed issue as https://github.com/nedbat/coveragepy/issues/989
2020-05-27 17:54:54 -07:00
Aman Agrawal 1a8798d9c5 ci: Disable casper tests in ci due to flaky behaviour. 2020-05-13 14:02:06 -07:00
Aman Agrawal 3442182d7b ci: Use absolute path to zulip folder.
Strangely focal resolves `~/zulip` as `/root/zulip`, this makes
sure we are in the correct folder.
2020-04-29 10:54:53 -07:00
Tim Abbott 28cb0aa81a docs: Use consistent spelling of CircleCI. 2020-04-28 11:26:58 -07:00
Aman Agrawal 1324e7798c ci: Directly run production install script. 2020-04-28 11:11:00 -07:00
Aman Agrawal b5e31c04c9 ci: Rename production install script. 2020-04-28 11:11:00 -07:00
Aman Agrawal 70f7d04bf1 ci: Rename production builder script file. 2020-04-28 11:11:00 -07:00
Aman Agrawal cf14cc211f ci: Separate prod tarball extraction and install process.
This allows to run scripts between extraction and install
process.

It will be used to restore npm caches for production install jobs.

We extract the tarball in the working directory so that yarn.lock and
package.json are available to restore cache.

(And also so the path is deterministic).
2020-04-25 16:10:41 -07:00
Aman Agrawal 0aa18e88a5 ci: Stop cloning repo as it is not required for prod install.
Move prod installer and test files to persistent dir so that
they can be utilized.
2020-04-25 16:07:55 -07:00
Aman Agrawal 0e544ec40b ci: Add job to run production script to tests production install.
Only restore npm cache, venv cache not required.
Restart rabbitmq-server for production install.
2020-04-24 12:40:31 -07:00
Aman Agrawal 7c99555107 ci: Copy https headers to persistent dir for comparison. 2020-04-24 12:40:31 -07:00
Aman Agrawal b7c50034b8 ci: Remove unnecessary setup-production steps.
This steps won't be used since production install will run in
a separate job.
2020-04-24 12:40:26 -07:00
Aman Agrawal 1a096a9a10 ci: Move built production tarball to /tmp/ directory.
This directory will persist across build jobs.
2020-04-24 12:40:20 -07:00
Pranav 93b09d44d6 ci: Add test to check for untracked files.
Runs a test at the end of tools/ci/backend to check if any untracked
files have been created during this ci tests. Exits with error code 1
if untracked files are found, otherwise exits successfully.

Fixes #14691.
2020-04-24 12:35:47 -07:00
Puneeth Chaganti d5f5a99b07 tools: Verify that all integrations have bot avatars. 2020-04-22 17:45:30 -07:00
arpit551 aabad1c0bb test: run test-locked-requirements only for Bionic.
Currently our compiled requirements files will differ for different
python versions. So we will run test-locked-requirements only for Bionic.
2020-04-22 16:10:59 -07:00
arpit551 56a5beddc0 ci: Fixed errors related to running services in ci production build.
Restart postgres service if provision is called in production test suite.
This is required because terminate-psql-sessions script (used
in tools/ci/setup-production) throws error if postgres service is not running.

Restart rabbitmq service if provision is called in production test suite.
This is done to start the node as Circle CI don't start services on installation.

Removed memcached restart as flush-memcached script (which is furthur
used in tools/ci/production) throws UNKNOWN READ FAILURE if memcached is restarted
in development.
2020-04-21 14:46:40 -07:00
arpit551 e6edf469ee ci: Remove the need of using TRAVIS in env.
Since now we want to use production suites on Circle CI so there
is no need to set TRAVIS in env while running scripts.

CIRCLECI is set default in the enviroment of Circle CI builds
so we can use it directly.

Also Travis CI had rabbitmq-server installed so we had to add workaround
in install script to avoid the error. That workaround is removed.
2020-04-21 14:46:40 -07:00
arpit551 4946405e3a ci: Change production setup in ci according to bionic and Circle CI.
Used postgres 10 inplace of postgres 9.5 as it is used in Bionic.

Upgraded nginx version in success-http-headers which is in CircleCI
Bionic enviroment.
2020-04-21 14:46:40 -07:00
arpit551 dee8de3b88 ci: Replaced travis with test suite while setting production setup on ci. 2020-04-21 14:46:40 -07:00