Commit Graph

75 Commits

Author SHA1 Message Date
Alex Vandiver 71b56f7c1c puppet: process_fts_updates connects as nagios (or provided username).
It should not use the configured zulip username, but should instead
pull from the login user (likely `nagios`), or an explicit alternate
provided PostgreSQL username.  Failure to do so results in Nagios
failures because the `nagios` login does not have permissions to
authenticated the `zulip` PostgreSQL user.

This requires CI changes, as the install tests install as the `zulip`
login username, which allowed Nagios tests to pass previously; with
the custom database and username, however, they must be passed to
process_fts_updates explicitly when validating the install.
2021-12-14 14:48:53 -08:00
Alex Vandiver a751d3a6ff ci: Refactor --test-custom-db to be an argument, not a command. 2021-12-14 14:48:53 -08:00
Alex Vandiver 0ae375e0f9 ci: Test upgrades from the latest minor release. 2021-11-25 08:00:34 -08:00
Alex Vandiver 4f1f8d8f6e ci: Remove an unnecessary and incorrect is_focal tag. 2021-11-25 08:00:34 -08:00
Alex Vandiver 73055255bb ci: Test pgroonga installation. 2021-11-20 07:13:50 -08:00
Alex Vandiver c9ebccdf70 ci: Update outdated comments to generated Dockerfiles.
These were missed in the doc updates in d78723b6e8, which were for
behavior changes in 16067bc4fc.
2021-11-05 15:45:46 -07:00
rht a3a71487b0 CI: Add Codespell linter.
This tool helps catch common typos in code and documentation, which is
particularly useful for our many contributors who are not native
English speakers.

The config is based on the codespell that I ran in
https://github.com/zulip/zulip/pull/18535.
2021-10-27 16:49:30 -07:00
Alex Vandiver a1d22cc7fa ci: Run production tests when files mentioning zilencer are changed.
Production installs do not use the zilencer application, but the tests
do include it; as such, changes to any files which reference zilencer
are more likely to pass tests but fail production installs.

Run production tests when those files are changed.
2021-10-20 16:53:11 -07:00
Anders Kaseorg 9cf5a03f2a ci: Migrate to new Codecov uploader.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-10-14 17:56:52 -07:00
Anders Kaseorg a88ace988a ci: Remove /__w permission twiddling.
Commit 9f2ac49fb3 (#19963) should make
this unnecessary.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-10-14 13:24:30 -07:00
Alex Vandiver acbe7ae7a8 github: Ignore CodeQL analysis in private repos.
CodeQL only runs in public repos; private forks will otherwise error
their CI runs.
2021-09-30 18:07:20 -07:00
Anders Kaseorg e96abc3c5a ci: Remove legacy-os test.
As of yesterday, the GitHub Actions ubuntu-16.04 environment has been
removed.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-22 13:58:46 -07:00
Ganesh Pawar dee20f0dbf node_tests: Run node tests in parallel.
Fixes #9471.
2021-09-10 14:18:31 -07:00
Anders Kaseorg a4dbc1edd4 docs: Format Markdown with Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-08 12:06:24 -07:00
Anders Kaseorg a9501772c1 ci: Do not run CodeQL on Dependabot push events.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-04 02:34:21 -07:00
Anders Kaseorg d464b3c805 ci: Run CodeQL on merge commit.
As per https://github.com/github/codeql-action/pull/297.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-03 13:51:18 -07:00
Tim Abbott aa651f75c1 ci: Adjust when production suite is run.
We make a few adjustments:
* We now run full CI whenever pushing to master. It's cheap enough
  that it's worth getting accurate signal.
* We now don't run production tests on PRs for changes to JavaScript/CSS
  in static/ that don't also affect the webpack configuration.
* We sort the list of paths that trigger tests.
2021-07-30 15:32:23 -07:00
Alex Vandiver 2daad58afa ci: Use an init process to reap defunct processes.
When Github Actions run in Docker, the default pid 1 entrypoint is
`tail -f /dev/null`.  PID 1 is responsible for propagating signals to
its children, and calling `waitpid()` on defunct processes; `tail`
does not do these things.  This results in zombie processes piling up
inside the container, which is not an issue in most contexts.

However, it affects `start-stop-daemon`, which hangs when stopping
daemon processes, as they are never reaped.  This appears in CI as
`/etc/init.d/supervisor restart` never being able to succeed.

Run the docker container with `--init`, which spawns a
`/sbin/docker-init` PID 1 to handle the job of an init process.
2021-07-22 14:09:01 -07:00
Alex Vandiver d78723b6e8 ci: Update outdated comments, documentation and gitignore.
Use of `Dockerfile.template` and generated `tools/ci/images/` was
removed in 16067bc4fc.
2021-07-22 14:09:01 -07:00
Anders Kaseorg 47897c76a2 scripts: Use curl -f (--fail).
This makes curl exit with nonzero status on HTTP 4xx/5xx errors.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-13 16:47:49 -07:00
Gaurav Pandey 9282956f19 upgrade-check: Add create cache directory step.
Create cache directories for the upgrade check
in the production-suite.
2021-07-05 09:55:38 -07:00
Gaurav Pandey e648ad3477 ci: Add prod upgrade step to prod suite.
This adds a check in the current production suite of
CI that upgrades a previous release of zulip server
with a newer one.

Fixes #18346.
2021-06-21 10:15:24 -07:00
Priyank Patel 05510a8c04 github-actions: Upgrade styfle/cancel-workflow-action. 2021-06-20 16:53:45 -07:00
Priyank Patel 607110ca33 github-actions: Ensure cancel previous run job never fails. 2021-06-20 16:53:45 -07:00
Riken Shah 4f54e15993 refactor: Convert `clean-unused-caches` to`clean_unused_caches.py`.
We convert the `clean-unused-caches` script to a
python file so we can run it in provision by importing it
instead of running the script, hence saving some time.
2021-06-12 07:28:16 -07:00
Adam Birds 9aab03654e ci: Update CI to test once with custom db name and user.
Update CI to test once with a custom db name and user so we can test
both scenarios of a custom dbname/user and the default "zulip".
2021-05-25 13:56:05 -07:00
Tim Abbott 3af0485d84 ci: Unpack installation tarball under /root/.
This ensures that we exercise the fact that the Zulip installer may be
unpacked to a directory that may not be world-readable.

bc45525369 fixed a recent regression in
this behavior that would have been caught by this commit.
2021-05-14 14:31:00 -07:00
Anders Kaseorg 405bc8dabf requirements: Remove Thumbor.
Thumbor and tc-aws have been dragging their feet on Python 3 support
for years, and even the alphas and unofficial forks we’ve been running
don’t seem to be maintained anymore.  Depending on these projects is
no longer viable for us.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-06 20:07:32 -07:00
Adam Birds e27268837b tools: Have `optimize-svg` do the optimization automatiically.
I have made `tools/setup/optimize-svg` do the SVG optimization
automatically rather than just telling you the command to run if they
need optimizing. This included adding a `--check` parameter to use in
CI to only check as we previously did rather than actually running the
optimization.

I have also made `tools/setup/optimize-svg` execute
`tools/setup/generate_integration_bots_avatars.py` once it has run the
optimization to ensure it is always ran.

This makes it one less command to run when creating an integration,
but also means that we catch instances where a PNG has just been
copied into the `static/images/integrations/bot_avatars` folder as the
only instance where this won't be run is if `optimize-svg` has not
been run which would be caught in CI.

Fixes #18183. Fixes #18184.
2021-04-19 10:16:54 -07:00
Gaurav Pandey 1bdcb11543 ci: Run zulip backend test suite for Debian bullseye.
This also verifies the Zulip codebase's Python 3.9 support.
2021-04-15 21:38:31 -07:00
Gaurav Pandey 303e7b9701 ci: Add Debian bullseye to production test suite. 2021-04-15 21:38:31 -07:00
Vishnu KS 8362865c8d actions: Rename kandra-ops to kandra ops in oneclick config file.
This bug was introduced in f00c13d303
when we started to hardcode stream name in config file instead of
storing it as one of the GitHub secrets.
2021-04-15 10:41:20 -07:00
Tim Abbott ad2e7dcba3 ci: Add comments documenting building base images. 2021-04-13 10:33:47 -07:00
Tim Abbott 7910a42586
docs: Create FUNDING.yml. 2021-04-07 20:15:29 -07:00
Vishnu KS f00c13d303 actions: Improve the secrets of one click release action. 2021-04-05 14:40:45 -07:00
Alex Vandiver 0023d561dd ci: Switch to hosting the CI images under Zulip on Dockerhub. 2021-03-31 16:54:34 -07:00
Aman Agrawal 837736001d ci: Use latest containers for running production tests.
This should have been a part of
da80895249 where the containers
were originally created.
2021-03-30 07:54:41 -07:00
Anders Kaseorg 056b715765 ci: Remove 2>&1 redirection.
We had used 2>&1 to redirect stderr to stdout so it could be piped
into ts, but commit dd3cdd6ec5 (#17611)
removed ts, so we no longer need the redirection.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-29 14:52:06 -07:00
Aman Agrawal 9c4d9dbaf1 ci: Regenerate bionic and focal containers.
This helps us reduce time to update dependencies on every CI
build since the previous containers used to take about 1 minute.

`sudo` had a bug due to which we were not able to create directories.
See https://github.com/sudo-project/sudo/issues/42.
We used these directories to restore caches.
Upgrading the focal dependencies via this commit naturally fixes that
bug.

Fixes #17854
2021-03-29 12:26:56 +05:30
Gaurav Pandey 371cdf9dc1 production_suite: Add debian production install to production suite.
We support Debian as an OS for setting up the Zulip server. But the CI
does not run on pull request to test the setting up of the server on
Debian. Hence, add the check to CI.
2021-03-28 16:04:03 -07:00
Vishnu KS 79586cc466 github: Create action for generating DigitalOcean one click app image. 2021-03-26 16:01:18 -07:00
Aman Agrawal e0ed9cc605 ci: Remove CircleCI workaround for buggy CPU count allocation.
GitHub Actions gives us 2 cpus (probably shared) to run the
jobs. Specifying 6 processes here doesn't make a difference
since both jobs run in around 5 minutes right now.
2021-03-16 15:11:26 -07:00
Aman Agrawal f2a137f863 github_actions: Remove Codecov workaround.
Codecov has released the new version which fixes the find error.
Followup from 6a357ea114
2021-03-16 15:11:21 -07:00
Aman Agrawal 76c69b943c github_actions: Explode backend and frontend tests.
We basically move all the tests from backend and frontend test
files to zulip-ci workflow. This results in GitHub Actions
nicely displaying all the tests separately.
2021-03-16 15:11:21 -07:00
Aman Agrawal dd3cdd6ec5 github_actions: Stop logging timestamp.
Timestamps are logged automatically by GitHub Actions and can be
made visible using log settings easily. Hence we remove the
unnecessary timestamps here to make the logs look much cleaner.
2021-03-16 15:11:21 -07:00
Alex Vandiver 2dc0662a50 ci: Upload puppeteer artifacts on failure.
Storing the puppeteer artifacts is useful for debugging failures in
CI.

Confusingly, `if: ${{ something }}` does not work out to be true like
`if: ${{ always() && something }}` does; the former has a silent
`success()` built into it[1]:

> If your if expression does not contain any of the status functions
> it will automatically result with success().

[1] https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#job-status-check-functions
2021-03-10 12:00:47 -08:00
Aman Agrawal 80268c52df ci: Notify in zulip when a build fails in GitHub Actions.
We use the circleci integration which already has a nice setup
for sending messages when triggered to send the build failure
notification.
2021-02-26 08:29:56 -08:00
Anders Kaseorg 892ef2749f ci: Port xenial-legacy check to GitHub Actions.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-08 18:01:29 -08:00
Tim Abbott b6ec66e972 github: Enable retention periods for uploaded artifacts.
This prevents Zulip CI from eventually consuming large amounts of
storage on one's GitHub account.

I picked a longer retention period for the Puppeteer artifacts because
humans look at those; the production tarballs are unlikely to be used
10 minutes after the run completes as they are just for the next stage
fo the build; certainly 14 days seems ample for any debugging.
2020-11-03 16:36:26 -08:00
Alex Vandiver 2b0bbbb882 tools: Rename postgres to postgresql in tool names. 2020-10-28 11:57:02 -07:00