Commit Graph

236 Commits

Author SHA1 Message Date
Anders Kaseorg 513848a9d2 requirements: Install Gitlint from PyPI again.
In https://github.com/jorisroovers/gitlint/pull/246 I split the
gitlint package into gitlint and gitlint-core, where the latter avoids
pinning exact versions of its requirements so we can use it again.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-11-29 13:35:58 -08:00
Alex Vandiver 2b025608d0 provision: Remove unsupported Ubuntu versions, add Impish. 2021-11-08 18:21:46 -08:00
Anders Kaseorg 5ab90c541c requirements: Install gitlint from its Debian package.
gitlint has a bunch of pinned requirements that hold back important
upgrades and conflict with other packages’ requirements.  The gitlint
author has rejected proposals to unpin them because it might increase
the amount of maintenance he needs to do
(https://github.com/jorisroovers/gitlint/pull/133).  That decision is
his to make, but _somebody_ needs to do the maintenance, so we
delegate it to Debian and Ubuntu.  If that means using a significantly
older version of gitlint, that’s a tradeoff we need to make to keep
the rest of our requirements current.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-17 17:05:18 -07:00
Anders Kaseorg 4206e5f00b python: Remove locally dead code.
These changes are all independent of each other; I just didn’t feel
like making dozens of commits for them.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg ad5f0c05b5 python: Remove default "utf8" argument for encode(), decode().
Partially generated by pyupgrade.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-02 15:53:52 -07:00
manavdesai27 572cef9a0f provision: Add support for Fedora 34. 2021-07-20 12:10:41 -07:00
Anders Kaseorg 0ba9114c22 install-yarn: Rewrite Yarn installer.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-27 16:58:17 -07:00
Anders Kaseorg 91bfebca7d install: Replace wget with curl.
curl uses Happy Eyeballs to avoid long timeouts on systems with broken
IPv6.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-25 09:05:07 -07:00
Anders Kaseorg bf361e9951 ci: Remove uses of VERSION_CODENAME.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-24 14:35:51 -07:00
Gilbert Bishop-White 11257933b3 provision: Add support for aarch64 ubuntu/debian.
Install prerequisite needed on aarch64, as pip package has to build
from scratch.Tested by running on M1 Mac.

Fixes part of #18614
2021-05-27 16:01:42 -07:00
Anders Kaseorg 405bc8dabf requirements: Remove Thumbor.
Thumbor and tc-aws have been dragging their feet on Python 3 support
for years, and even the alphas and unofficial forks we’ve been running
don’t seem to be maintained anymore.  Depending on these projects is
no longer viable for us.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-06 20:07:32 -07:00
Anders Kaseorg 178736c8eb docs: Fix spelling errors caught by codespell.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-26 09:31:08 -07:00
Gaurav Pandey 78524d4f87 provision: Add support for debian bullseye.
Fixes part of #17863.
2021-04-15 21:38:31 -07:00
Ganesh Pawar c1628e7605 provision: Upgrade support for Fedora to version 33.
Note that the `overwrite_symlink` changes fix a bug introduced in
5c20ee998c, that we need root
permissions to do those operations.
2021-03-22 19:34:18 -07:00
Aman Agrawal 2b23609f9a ci: Remove unused circleci config file and update codebase.
We have disabled CircleCI and are using GitHub Actions for automated
testing.

docs: Changed context from CircleCI to Github Actions and wrote
some documentation specific to GH Actions.

tools: Replaced env checks for CIRCLECI with GITHUB_ACTION.

README: Use GitHub Actions build status badge.
2021-03-16 15:11:21 -07:00
Anders Kaseorg ccad00b7e9 provision: Suppress exception chaining for CalledProcessError retries.
When exception is raised inside an exception handler, Python 3
helpfully prints both tracebacks separated by “During handling of the
above exception, another exception occurred:”.  But when we’re using
an exception handler to retry the same operation, multiple tracebacks
are just noise.  Suppress the earlier one using PEP 409 syntax.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-03 16:25:03 -08:00
Anders Kaseorg 6e4c3e41dc python: Normalize quotes with Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 11741543da python: Reformat with Black, except quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 5028c081cb python: Merge concatenated string literals that Black would uglify.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Ganesh Pawar 7eeca9da46 provision: Add provision support for Ubuntu 20.10(Groovy).
PostgreSQL 13 is used when os_version is 20.10.
2021-02-05 09:30:34 -08:00
aditya-mitra b8f57fa6bb provision: Add provision support for KDE Neon 20.04.
PostgreSQL 12 is used for Neon 20.04, which is a fork of Ubuntu 20.04.
2021-02-03 11:51:05 -08:00
Sutou Kouhei 0d3f9fc855 install: Use PGroonga packages built for PostgreSQL packages by PGDG
Because we always use PostgreSQL packages by PGDG since Zulip 3.0.

Fixes #16058.
2020-12-18 15:38:21 -08:00
Alex Vandiver c1201bd69c provision: Rename variables to postgresql. 2020-10-28 11:57:03 -07:00
Alex Vandiver 1f7132f50d docs: Standardize on PostgreSQL, not Postgres. 2020-10-28 11:55:16 -07:00
Anders Kaseorg 31d0141a30 python: Close opened files.
Fixes various instances of ‘ResourceWarning: unclosed file’ with
python -Wd.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-26 12:31:30 -07:00
Anders Kaseorg 72d6ff3c3b docs: Fix more capitalization issues.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-23 11:46:55 -07:00
Anders Kaseorg 6099612a15 lint: Add shfmt as a linter.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-15 15:16:00 -07:00
Anders Kaseorg dd48dbd912 docs: Add spaces to “check out”, “log in”, “set up”, “sign up” as verbs.
“Checkout”, “login”, “setup”, and “signup” are nouns, not verbs.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-13 15:47:13 -07:00
Anders Kaseorg b4597a8ca8 python: Elide default for store_{true,false} argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg d751e0cece puppet: Don’t install netcat.
It’s been unused since commit 0af22dad18
(#13239).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 10:33:47 -07:00
Anders Kaseorg 0682d5c0f3 provision: Convert "".format to Python 3.6 f-strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 17:27:14 -07:00
Anders Kaseorg 424b441bb1 provision: Check for old Ubuntu or Python before starting Python.
This unblocks us from being able to use Python 3.6 syntax in
provision.py and its dependencies.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 17:27:14 -07:00
Anders Kaseorg 1ded51aa9d python: Replace list literal concatenation with * unpacking.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:15:41 -07:00
Anders Kaseorg a5dbab8fb0 python: Remove redundant dest for argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:04:10 -07:00
Priyank Patel c1ba648b4f dependencies: Add xvfb dependency.
The node package allow use to control xvfb apt package in puppeteer
tests. This help us create a fake display so we can run puppeteer in
headful (headless: false) mode, which is required to use the chrome
extension desktop capture API.
2020-08-30 23:38:52 -07:00
Tim Abbott 2f5f5d7749 dependencies: Remove PhantomJS from development environment. 2020-08-30 17:16:02 -07:00
Anders Kaseorg 18dcb86de5 Revert "provision: Fix missing <sasl/sasl.h> headers during provision."
This reverts commit cc9dc07b07.  It was
merged with no review and has various issues (see #16086).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-10 12:09:37 -07:00
Aman cc9dc07b07 provision: Fix missing <sasl/sasl.h> headers during provision.
This fixes the error of missing sasl.h when compiling python-ldap
from source during provision.
2020-08-09 23:21:13 -07:00
Anders Kaseorg d35b5027bb semgrep: Upgrade semgrep to 0.17.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-30 15:12:22 -07:00
Priyank Patel e834439d82 provision: Do service <service-name> start instead of restart.
Doing service memcached start instead of restart fixed an issue on
focal build in GitHub actions, where it exits with code 1 when it
is done twice.It is done first in Install Dependencies step and then
again in last step where we call tools/ci/setup-backend again which
runs provision.

Furthermore, I don't belive there is a technical reason we use
restart over start; rather I think it was just a random choice with
the intend to just start the services in CI. I traced the code back
to commit 1f2f497cab if it helps.
Looking at the source code of memcached, the step that's failing is:
  start-stop-daemon --stop --quiet --oknodo --pidfile $PIDFILE
which is equivilent to: service memcached stop, we can rule out the
service memcache start since it works. Ideally, we do figure out and
solve the issue of why memcached fails when executing service
memcached stop but I am not equipped with debugging it. And this
workaround seems reasonable rather than a "hacky" solution.

For the relevant code in memcached see:
https://github.com/memcached/memcached/blob/master/scripts/memcached-init.

Finally, the change to the rest of services is for consistency.
2020-07-06 18:09:15 -07:00
Priyank Patel b1c0c13008 provision: Include check for GitHub Action along with CircleCI. 2020-06-29 16:31:43 -07:00
Aman Agrawal 6950d8d769 provision: Don't run migrations on `zulip` db in CircleCI.
The automated tests running in CircleCI don't actually use the `zulip`
db, so we can skip running migrations on it in some CircleCI shards to
save time.

NOTE: This only effects build jobs that run provision, except the
`production-build` job where we skip building the dbs altogether.
Migrations still run on `focal-backend` build job to ensure
we are testing all our development setup code.
2020-06-16 15:37:32 +05:30
Alex Vandiver 7fe1780088 provision: Install `crudini` for zulip.conf shell scripting.
A few tools already use this, and it is part of server installation;
install it in development environments as well.
2020-06-15 17:58:39 -07:00
Anders Kaseorg 5dc9b55c43 python: Manually convert more percent-formatting to f-strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
Anders Kaseorg 3461db7ef5 python: Convert percent formatting to "".format in certain files.
These files can’t use f-strings yet because they need to run in Python
2 or Python 3.5.

Generated by pyupgrade.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
Anders Kaseorg 365fe0b3d5 python: Sort imports with isort.
Fixes #2665.

Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.

Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start.  I expect this change will increase pressure for us to split
those files, which isn't a bad thing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:45:32 -07:00
Anders Kaseorg 69730a78cc python: Use trailing commas consistently.
Automatically generated by the following script, based on the output
of lint with flake8-comma:

import re
import sys

last_filename = None
last_row = None
lines = []

for msg in sys.stdin:
    m = re.match(
        r"\x1b\[35mflake8    \|\x1b\[0m \x1b\[1;31m(.+):(\d+):(\d+): (\w+)", msg
    )
    if m:
        filename, row_str, col_str, err = m.groups()
        row, col = int(row_str), int(col_str)

        if filename == last_filename:
            assert last_row != row
        else:
            if last_filename is not None:
                with open(last_filename, "w") as f:
                    f.writelines(lines)

            with open(filename) as f:
                lines = f.readlines()
            last_filename = filename
        last_row = row

        line = lines[row - 1]
        if err in ["C812", "C815"]:
            lines[row - 1] = line[: col - 1] + "," + line[col - 1 :]
        elif err in ["C819"]:
            assert line[col - 2] == ","
            lines[row - 1] = line[: col - 2] + line[col - 1 :].lstrip(" ")

if last_filename is not None:
    with open(last_filename, "w") as f:
        f.writelines(lines)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 16:04:12 -07:00
Anders Kaseorg ca4357fd64 python: Use standard NoReturn (Python ≥ 3.6).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 12:56:52 -07:00
arpit551 569731da50 provision: Removed hack for running production suites in ci.
Since now, we use different jobs for getting release-tarball
and running production suites so we don't require this hack
2020-06-04 14:26:23 -07:00
arpit551 291bce93d0 provision: Rename --production-test-suite option in provision.
Since we use this option in our docker-zulip project also
so rather than using it as a test suite option we made it
more specific i.e. --build-release-tarball-only.
2020-06-04 14:26:23 -07:00