Commit Graph

77 Commits

Author SHA1 Message Date
Alex Vandiver 2b0bbbb882 tools: Rename postgres to postgresql in tool names. 2020-10-28 11:57:02 -07:00
Alex Vandiver 5eb8064a1a install: Rename postgres options to postgresql. 2020-10-28 11:55:32 -07:00
Alex Vandiver e124324050 puppet: Rename postgres_appdb in nagios to postgresql. 2020-10-28 11:51:52 -07:00
Alex Vandiver c2185a81d6 puppet: Move top-level zulip deployments into "profile" directory.
This moves the puppet configuration closer to the "roles and profiles
method"[1] which is suggested for organizing puppet classes.  Notably,
here it makes clear which classes are meant to be able to stand alone
as deployments.

Shims are left behind at the previous names, for compatibility with
existing `zulip.conf` files when upgrading.

[1] https://puppet.com/docs/pe/2019.8/the_roles_and_profiles_method
2020-10-27 13:29:19 -07:00
Anders Kaseorg 72d6ff3c3b docs: Fix more capitalization issues.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-23 11:46:55 -07:00
Mateusz Mandera 716df658fa queue_processors: Don't run test queues with run-dev.py. 2020-10-18 14:07:31 -07:00
Anders Kaseorg dfaea9df65 shfmt: Reformat shell scripts with shfmt.
https://github.com/mvdan/sh

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-15 15:16:00 -07:00
Alex Vandiver 150d782d6b production-verify: Fix puppet path. 2020-10-06 17:26:14 -07:00
Alex Vandiver 8cf37a0d4b queue: Add a tool to profile no-op enqueue and dequeue actions. 2020-10-06 17:26:14 -07:00
Anders Kaseorg d751e0cece puppet: Don’t install netcat.
It’s been unused since commit 0af22dad18
(#13239).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 10:33:47 -07:00
Dinesh 2d22d88700 casper: Remove few traces of casper.
Now that all casper tests have been migrated to
puppeteer, there's no need for having casper
related things.

Removed the casperjs package and removed/replaced
casper in few places with puppeteer.

Only removed few of them which I'm confident
about. Also didn't make any changes in docs
as it would be easier to remove them while
adding puppeteer docs.
2020-08-30 17:16:02 -07:00
Tim Abbott dbf3894c49 test-backend: Enable --ban-console-output and document. 2020-08-27 11:39:53 -07:00
Anders Kaseorg dbdf67301b memcached: Switch from pylibmc to python-binary-memcached.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-06 12:51:14 -07:00
Priyank Patel 9504d403b3 github-actions: Fix incorrect hash key in production install job.
The hash keys were missing hash for package.json and yarn.lock
because they were not present since we don't do a full checkout
in this job. We fix this by sending over those files and generating
hashes from them.

I usally verify these cache keys by clicking the Restore <cache>
step dropdown menu and then clicking the Run ... dropdown menu again
to see the generated hash.
2020-08-03 12:37:20 -07:00
Priyank Patel dda7e938f4 puppeteer: Fix flake with compose box input.
There were a lots of flakes in CI recently because typeahead didn't
appear when Enter was pressed and real emails are not accepted as
valid inputs. To fix this we wait for typeahead to appear and then
click that instead of Enter. We also use delay option to type the
email (100ms delay between keypresses) since without we'd also get
flakes.

Re-enable puppeteer test in CI after this fix too.
2020-07-30 11:32:32 -07:00
Tim Abbott cee7e8e89d ci: Disable puppeteer tests. 2020-07-28 17:12:49 -07:00
Steve Howell da8691d7d7 ci: Add new check-node-fixtures tool to CI.
This tool only takes a couple seconds to run,
so it's pretty low risk to add to CI.
2020-07-26 16:07:40 -07:00
Priyank Patel 51199878ed tools/ci: Fix KeyError when running tools/ci/generate-dockerfiles.
This KeyError occurs because we remove extra_packages from
images.yml because it was no longer needed in a486872a8e.
2020-07-26 16:03:35 -07:00
Priyank Patel 6a4792e844 tools/ci: Adjust ZULIP_PATH so it works for both CIs.
Until, we remove the circleci.
2020-07-24 10:36:55 -07:00
Priyank Patel 75ab26771b ci: Store production built assets to /tmp/production-build.
Previously, we copied them to /tmp and from there we specified those
assets we copied in circleci config in presist_to_workspace step.
Copying it to a directory allows us to get rid of list in circleci
config and GitHub Actions's upload artifact (their version of
presist to workspace) doesn't allow us to specify indivivual files
so only is this cleaner but required.
2020-07-24 10:36:55 -07:00
Mohit Gupta 308643cace tests: Rename test_narrow.py to test_message_fetch.py.
This commit renames test_narrow.py and also all its references to
test_message_fetch.py.
2020-07-07 17:14:10 -07:00
arpit551 4f6cd6577c http_headers: Substitute Nginx version based on OS.
success-http-headers-bionic.txt and success-http-headers-focal.txt
differ only in the nginx version so this substitution will allow
us to have single file for both of them. Also this change helps
to avoid CI failure if Nginx version is updated in the OS.
2020-07-07 11:20:05 -07:00
Priyank Patel 29dd0c485a ci: Move upgrading git for bionic to docker image.
Just reduces the clutter in configuration file for GitHub Actions.
2020-07-06 18:09:15 -07:00
Priyank Patel c908278237 ci: Install moreutils in the docker image. 2020-07-06 18:09:15 -07:00
Aman Agrawal a486872a8e requirements: Upgrade Thumbor to 7.0.0a5 on Python 3.
Co-authored-by: Anders Kaseorg <anders@zulip.com>
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-06 16:09:53 -07:00
arpit551 834abefdb9 http_headers: Update Nginx version in success-http-headers-focal. 2020-07-02 22:42:22 +05:30
Anders Kaseorg fa89d1b266 generate-self-signed-cert: Correct subjectAltName for an IP address.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-29 22:19:47 -07:00
Alex Vandiver 2540b678b0 ci: Test upgrading PostgreSQL 10 -> 12. 2020-06-29 17:18:47 -07:00
Alex Vandiver f621e1b7e4 ci: Split out installation testing. 2020-06-29 17:18:47 -07:00
Alex Vandiver 36335ecd39 ci: Simplify tool-copying in production-build. 2020-06-29 17:18:47 -07:00
Priyank Patel 92f98292c1 tools/ci: Create a github user instead of a circleci user.
While we are making this change, we create a ARG USERNAME variable
that behaves like ENV but just for the lifetime of the Dockerfile.
2020-06-29 16:31:43 -07:00
Priyank Patel a2c2c6ceff tools: Move CI docker images files into tools/ci. 2020-06-29 16:31:43 -07:00
Aman Agrawal 6950d8d769 provision: Don't run migrations on `zulip` db in CircleCI.
The automated tests running in CircleCI don't actually use the `zulip`
db, so we can skip running migrations on it in some CircleCI shards to
save time.

NOTE: This only effects build jobs that run provision, except the
`production-build` job where we skip building the dbs altogether.
Migrations still run on `focal-backend` build job to ensure
we are testing all our development setup code.
2020-06-16 15:37:32 +05:30
Anders Kaseorg 80b991fc44 production-helper: Clarify log output on check_rabbitmq_consumers failure.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 16:49:32 -07:00
arpit551 662f4902f8 ci: Setup production job for Focal.
Run production suites on Ubuntu Focal.
Added separate success-http-headers files for Focal and Bionic.
Also excluded them from whitespace rules in lint.

memcached 1.5.22 in Ubuntu 20.04 has a bug where it looks for its SASL
configuration at /etc/sasl2/memcached.conf/memcached.conf instead of
/etc/sasl2/memcached.conf.
We already use a workaround for this while applying puppet configurations in
99e71f3786 but for docker builds we used
do memcached hack since we can not use systemd in docker containers.
2020-06-08 21:59:57 -07:00
arpit551 291bce93d0 provision: Rename --production-test-suite option in provision.
Since we use this option in our docker-zulip project also
so rather than using it as a test suite option we made it
more specific i.e. --build-release-tarball-only.
2020-06-04 14:26:23 -07:00
Dinesh d429b7145e tests: Make puppeteer tests run in full suites and CircleCI. 2020-06-03 17:39:35 -07:00
Aman Agrawal c9bff71410 ci: Don't remove .coverage file & set empty suffix.
Fixes #14962
* codecov needs `.coverage` file in the pwd to upload coverage
results.
* concurrency='multiprocessing' forces `.coverage` file to have a
data_suffix, we explicity set it to "" for the file to have no suffix.
Filed issue as https://github.com/nedbat/coveragepy/issues/989
2020-05-27 17:54:54 -07:00
Aman Agrawal 1a8798d9c5 ci: Disable casper tests in ci due to flaky behaviour. 2020-05-13 14:02:06 -07:00
Aman Agrawal 3442182d7b ci: Use absolute path to zulip folder.
Strangely focal resolves `~/zulip` as `/root/zulip`, this makes
sure we are in the correct folder.
2020-04-29 10:54:53 -07:00
Tim Abbott 28cb0aa81a docs: Use consistent spelling of CircleCI. 2020-04-28 11:26:58 -07:00
Aman Agrawal 1324e7798c ci: Directly run production install script. 2020-04-28 11:11:00 -07:00
Aman Agrawal b5e31c04c9 ci: Rename production install script. 2020-04-28 11:11:00 -07:00
Aman Agrawal 70f7d04bf1 ci: Rename production builder script file. 2020-04-28 11:11:00 -07:00
Aman Agrawal cf14cc211f ci: Separate prod tarball extraction and install process.
This allows to run scripts between extraction and install
process.

It will be used to restore npm caches for production install jobs.

We extract the tarball in the working directory so that yarn.lock and
package.json are available to restore cache.

(And also so the path is deterministic).
2020-04-25 16:10:41 -07:00
Aman Agrawal 0aa18e88a5 ci: Stop cloning repo as it is not required for prod install.
Move prod installer and test files to persistent dir so that
they can be utilized.
2020-04-25 16:07:55 -07:00
Aman Agrawal 0e544ec40b ci: Add job to run production script to tests production install.
Only restore npm cache, venv cache not required.
Restart rabbitmq-server for production install.
2020-04-24 12:40:31 -07:00
Aman Agrawal 7c99555107 ci: Copy https headers to persistent dir for comparison. 2020-04-24 12:40:31 -07:00
Aman Agrawal b7c50034b8 ci: Remove unnecessary setup-production steps.
This steps won't be used since production install will run in
a separate job.
2020-04-24 12:40:26 -07:00
Aman Agrawal 1a096a9a10 ci: Move built production tarball to /tmp/ directory.
This directory will persist across build jobs.
2020-04-24 12:40:20 -07:00