Commit Graph

141 Commits

Author SHA1 Message Date
Aman Agrawal 0504c61bfd semgrep: Use pattern-where-python operator to filter patterns.
See https://github.com/returntocorp/semgrep/blob/experimental/docs/config/advanced.md#pattern-where-python for usage.

This helps us minimize duplication of similar patterns.
2020-05-20 09:37:26 -07:00
Anders Kaseorg baab041acd lint: Upgrade semgrep from 0.5.0-dev.2 to 0.5.0-dev.4.
The major PROVISION_VERSION bump would not be needed, but it was
missing in commit 5ab62a3514 (#14834),
so I’m doing it here.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-02 10:19:40 -07:00
Anders Kaseorg 82f629091a lint: Check docs/THIRDPARTY for format errors.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-02 10:10:22 -07:00
Aman Agrawal 5ab62a3514 sgrep/semgrep: Updgrade and rename sgrep to semgrep.
With its new upgrade, sgrep has been renamed to semgrep.

Rename sgrep.yml to semgrep.yml
2020-05-01 11:01:14 -07:00
Anders Kaseorg 5901e7ba7e python: Convert function type annotations to Python 3 style.
Generated by com2ann (slightly patched to avoid also converting
assignment type annotations, which require Python 3.6), followed by
some manual whitespace adjustment, and six fixes for runtime issues:

-    def __init__(self, token: Token, parent: Optional[Node]) -> None:
+    def __init__(self, token: Token, parent: "Optional[Node]") -> None:

-def main(options: argparse.Namespace) -> NoReturn:
+def main(options: argparse.Namespace) -> "NoReturn":

-def fetch_request(url: str, callback: Any, **kwargs: Any) -> Generator[Callable[..., Any], Any, None]:
+def fetch_request(url: str, callback: Any, **kwargs: Any) -> "Generator[Callable[..., Any], Any, None]":

-def assert_server_running(server: subprocess.Popen[bytes], log_file: Optional[str]) -> None:
+def assert_server_running(server: "subprocess.Popen[bytes]", log_file: Optional[str]) -> None:

-def server_is_up(server: subprocess.Popen[bytes], log_file: Optional[str]) -> bool:
+def server_is_up(server: "subprocess.Popen[bytes]", log_file: Optional[str]) -> bool:

-    method_kwarg_pairs: List[FuncKwargPair],
+    method_kwarg_pairs: "List[FuncKwargPair]",

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-18 20:42:48 -07:00
Anders Kaseorg c734bbd95d python: Modernize legacy Python 2 syntax with pyupgrade.
Generated by `pyupgrade --py3-plus --keep-percent-format` on all our
Python code except `zthumbor` and `zulip-ec2-configure-interfaces`,
followed by manual indentation fixes.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-09 16:43:22 -07:00
brendon f65e6d0d94 sgrep: Install syntactic code search tool as an external linter.
Add sgrep (sgrep.dev) to tooling and include simple rule as
proof of concept. Included rule detects use of old django render
function.

Also added a rule that looks for if-else statements where both
code paths are identical.
2020-04-01 15:08:34 -07:00
arpit551 0c821424cd lint: Run whitespace linter against .yml files.
Lint now checks for .yml files with whitespace_rules.
2020-03-24 15:31:59 -07:00
Anders Kaseorg 9d598d95a6 puppet: Fix puppet-lint warning.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-20 19:51:48 -08:00
Steve Howell 752d6dc6df tools: Remove find-add-class tool.
I added this tool a few years ago, and I did have
a vision for how it would improve our codebase, but
I can't remember exactly where I was going with it.

At this point the tool is just a little too noisy
to be helpful.  An example of it creating confusion
was a recent PR where somebody was patching
user_circle_class in the PM list, and we already
had similar code in the buddy list, because they
use the same CSS.  I mean, there was possibly a way
that the code could have been structured to remove
some of the duplication, but it probably would have
just moved the complexity around.

I just don't think it's worth maintaining the tool
at this point.
2020-01-14 15:45:49 -08:00
Anders Kaseorg ab211c7acf lint: Tell ShellCheck to look for sourced files at relative paths.
This uses the new -P option of ShellCheck 0.7.0.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-12-18 03:48:02 -08:00
Anders Kaseorg 0ce4de0421 lint: Run mypy with --no-error-summary.
This suppresses the mypy message “Success: no issues found in 1085
source files” or “Found 1 error in 1 file (checked 1085 source files)”
in the output of lint.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 13:26:02 -08:00
Anders Kaseorg 0584a7938f tsconfig: Move to top level.
This way, webpack.config.ts is type checked.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-04 18:12:11 -08:00
Anders Kaseorg e57ede9421 yarn.lock: Share duplicate packages with yarn-deduplicate.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-02 19:30:09 -07:00
Hemanth V. Alluri c90056bdb2 tools: Move check-swagger to check-openapi and make it executable. 2019-08-05 21:06:19 -07:00
Anders Kaseorg ecfb7c6a7f lint: Add TypeScript compiler as a linter.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-22 17:55:18 -07:00
Wyatt Hoodes 89df6e9425 check-templates: Add fix option to tools/check_templates.
`valid_indent_html` allows for replacing the incorrectly
indented file with the correct pretty-printed version
if `--fix` is passed to `tools/lint`.

Fixes #12641.
2019-07-19 11:12:11 -07:00
Anders Kaseorg 3c3471b720 templates: Rename *.handlebars ↦ *.hbs and - ↦ _.
Tweaked by tabbott to avoid accidentally disabling the linter for
handlebars templates.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-12 21:11:03 -07:00
Aman Agrawal 05aa3bcbc1 lint: Make `--fix` a default arg for zulint.
Since the check for `--fix` is in `zulint/command.py`,
we should keep it as an arg for zulint.
2019-07-03 14:57:31 -07:00
Thomas Ip a2de8ab44f lint: Add --fix option to ./tools/lint.
Linters with the fix option are stylelint, eslint and puppet-lint.
2019-06-24 14:13:04 -07:00
Aman Agrawal 8fc7c89874 zulint: Add `--verbose` arg to print good/bad lines in error.
This argument probably still needs some iteration, but it's already
useful.
2019-06-23 22:24:46 -07:00
Aman Agrawal 8eee19a160 lint: Add descriptions for all the linters. 2019-06-23 22:23:15 -07:00
Aman Agrawal e9ff9e34b3 lint: Use --groups to specify specific groups to run.
This helps generalize the use of groups inside zulint.
Introduce list_files to return `by_lang` files dict.
Add feature to create custom groups.
Make custom groups for backend and frontend files.
2019-06-23 22:23:15 -07:00
Aman Agrawal 7b7e887386 lint: Pass args to LinterConfig when initializing it.
This makes it easy to use args inside LinterConfig.
2019-06-23 21:55:02 -07:00
Aman Agrawal 0b12ec1417 lint: Initialize logger inside zulint.
This also helps move verbose_timing logic inside zulint.
2019-06-23 21:55:02 -07:00
Aman Agrawal b6d4438726 lint: Pass `args` instead of passing individual attributes to do_lint.
Pass the args to linter_config.do_lint.
2019-06-23 21:55:02 -07:00
Aman Agrawal b2b49089fd tools: Extract get_provisioning_status check logic.
Move get_provisioning_status check logic into
assert_provisioning_status_ok and use it instead of duplicating the
check code.
2019-06-23 21:55:02 -07:00
Aman Agrawal 6d040d330c lint: Add --list arg which lists all the registered linters. 2019-06-18 11:32:04 -07:00
Aman Agrawal 6457c1f7c6 lint: Generalize --skip arg and move logic to zulint/command. 2019-06-18 11:32:04 -07:00
Aman Agrawal 995b357cde lint: Add --only agr to run only the specified linters. 2019-06-18 11:32:04 -07:00
Aman Agrawal cf8653945f lint: Rename commit_messages to gitlint.
Use `gitlint` name to register gitlint linter since it is the commonly
used name for it.
2019-06-18 11:32:04 -07:00
Aman Agrawal 6b73926e93 lint: Add --skip arg to replace --no-gitlint/mypy.
Use --skip=gitlint,mypy instead of --no-gitlint/mypy.
2019-06-18 11:32:04 -07:00
Aman Agrawal 426a222b7e lint: Rename --verbose to --verbose-timing. 2019-06-18 11:32:04 -07:00
Aman Agrawal db25c0c2ca lint: Combine functions in custom_rules into RuleList class.
This makes linting rules in zulint more general. Make necessary
changes in tools/lint and tools/custom_check.py to run with the new
RuleList class.

Modify tests for `RuleList` class. Tests only include minor changes to
test with the new class.
2019-06-18 11:27:50 -07:00
Raymond Akornor e01d3be1ba tools: Remove check-urls linter.
This commit removes `tools/check-urls`. It was added as
a useful tool in preparation for the Django 1.10 migration.
Since we completed that migration, it is no longer needed.

Fixes #12180.
2019-05-05 17:47:41 -07:00
Tim Abbott 31f6ace084 lint: Fix typo in eslint command line. 2019-04-13 22:45:07 -07:00
Thomas Ip 02cb85a8a9 typescript: Use ESLint instead of TSLint. 2019-04-13 11:42:47 -07:00
Thomas Ip a2872c107e typescript: Move TS files into JS directory.
This is just a code reorganization to avoid making it difficult to
find things as we migrate more file to TypeScript.
2019-03-25 12:11:37 -07:00
Anders Kaseorg ee8ff4df66 tools: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:10:31 -08:00
Harshit Bansal 2944075f70 lint: Respect `--force` flag while running mypy. 2019-01-17 10:34:43 -08:00
Tim Abbott 6b69cc0b39 lint: Run mypy as part of main linter.
To support this, we add a pass_targets option to the main linter
library, because with current mypy, it's generally counterproductive
to pass the list of files in (can produce spurious errors; isn't
faster).
2018-12-16 21:52:53 -08:00
Anders Kaseorg c3bd293eaf Add shellcheck as a linter.
Fixes #10162.

Tweaked by tabbott to cover documentation as well.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-18 15:32:26 -07:00
Aditya Bansal a3b8ab46b3 puppet-lint: Include in linter tool chain to run by default.
We start to use puppet-lint to lint puppet modules by default by
adding it to tools/lint (which controls our linter tool chain).

We also define a few puppet-lint rules to exclude.

Fixes: #9185.
2018-08-13 12:30:57 -07:00
Tim Abbott e67fa36803 lint: Update config to split files more evenly. 2018-08-07 09:42:37 -07:00
Tim Abbott 78a93b8d9e lint: Refactor pyflakes to use more consistent style.
In particular, we no longer pass in the full `by_lang` object and
expect it to pull out the Python piece.
2018-08-07 09:42:37 -07:00
Tim Abbott 55ba434ecb zulint: Extract LinterConfig class to zulint. 2018-08-04 19:53:53 -07:00
Tim Abbott f74bb8a9b2 zulint: Extract run_parallel and do_lint. 2018-08-04 19:53:53 -07:00
Tim Abbott acf8ec492d zulint: Extract common linter argument parsing logic.
The remaining code is Zulip-specific business logic.
2018-08-04 19:53:53 -07:00
Tim Abbott 1fcc2a6ea4 zulint: Move lister.py to tools/zulint.
This is preparatory refactoring work for being able to extract the
linter as an external project called "zulint".
2018-08-04 19:53:53 -07:00
Aditya Bansal 0042694e24 stylelint: Start using stylelint to lint our css.
Tweaked by tabbott to document it.

Fixes #8894.
2018-07-25 15:03:35 -07:00
Steve Howell b9c9dd9b5a lint: Turn off the CSS checker (officially).
Our CSS checker globs for .css files.  Since the
SCSS cutover, it has been a no-op, so there's no
sense launching it.  See #8894 for details on
future plans.
2018-07-20 10:03:28 -04:00
Joshua Pan a67ba617d6 minor: Fix typo in tools/lint. 2018-05-16 13:19:28 -04:00
Tommy Ip 7243a91e47 linter: Re-enable gitlint. 2017-12-18 09:36:47 -05:00
Greg Price 137c0e65bb tools: Revert to Python 2 typing syntax for now.
This reverts commit 66261f1cc.  See parent commit for reason; here,
provision worked but `tools/run-dev.py` would give errors.

We need to figure out a test that reproduces these issues, then make a
version of these changes that keeps that test working, before we
re-merge them.
2017-12-13 10:38:15 -08:00
rht 66261f1cc3 tools: Use Python 3 syntax for typing in many files. 2017-12-12 17:42:57 -08:00
cPhost c6985e2eee eslint: add --cache flag for faster linting 2017-12-07 15:14:51 -08:00
Tim Abbott cb4acf2aa4 docs: Lint rst files for trailing whitespace. 2017-11-16 13:20:50 -08:00
Tim Abbott b1d98c1d0f lint: Only run find_add_class when JS files change. 2017-08-27 13:51:06 -07:00
Tim Abbott 79ae262463 lint: Only run check-urls after python files change. 2017-08-27 13:51:06 -07:00
Robert Hönig c5da0ada31 linter: Replace --pep8 with new --frontend/--backend options.
Adds the --frontend and --backend options to replace
--pep8.

Significantly modified by tabbott to use a cleaner framework.
2017-08-27 13:50:53 -07:00
Robert Hönig 2186d53cf5 linter: Migrate lint from optparse to argparse. 2017-08-27 13:18:55 -07:00
Greg Price a099e698e2 py3: Switch almost all shebang lines to use `python3`.
This causes `upgrade-zulip-from-git`, as well as a no-option run of
`tools/build-release-tarball`, to produce a Zulip install running
Python 3, rather than Python 2.  In particular this means that the
virtualenv we create, in which all application code runs, is Python 3.

One shebang line, on `zulip-ec2-configure-interfaces`, explicitly
keeps Python 2, and at least one external ops script, `wal-e`, also
still runs on Python 2.  See discussion on the respective previous
commits that made those explicit.  There may also be some other
third-party scripts we use, outside of this source tree and running
outside our virtualenv, that still run on Python 2.
2017-08-16 17:54:43 -07:00
neiljp (Neil Pilgrim) f96f71e439 mypy: assert Popen.stdout based on API, in tools/lint. 2017-08-03 11:09:50 -07:00
Tommy Ip 61e24cfbea tools: Remove unused `bright_red_output` context manager.
This context manager is no longer in use since the linter visual
updates.
2017-07-05 23:31:09 -07:00
Tommy Ip 766abaa1a3 linter: Update external_linter visual output. 2017-07-05 23:31:08 -07:00
Steve Howell a606628b16 Extract tools/linter_lib/pep8.py 2017-06-05 09:20:21 -07:00
Steve Howell 9fab41aebb Extract tools/linter_lib/pyflakes.py 2017-06-05 09:20:21 -07:00
Steve Howell 29affda956 Extract tools/linter_lib/exclude.py 2017-06-05 09:20:21 -07:00
Steve Howell 40a2ffad98 Extract tools/linter_lib/custom_check.py.
As part of extracting this, we exempt the library from all custom
checks on itself.  This is expedient, since a lot of our
custom checks are naive about whether things are in strings, and
it is also a pain to configure individual rules.
2017-06-05 09:20:21 -07:00
Steve Howell 8ee50f019b lint: Add "nolint" handling for custom checks. 2017-06-05 09:20:21 -07:00
Aditya Bansal acefc67f2f lint: Start enforcing PEP-E261 by default in our linters.
In this commit we start to check for violations to PEP-E261 in our
codebase by default in our linters. Also we have introduced a
ignore list which has around 20 files still in violation to
PEP-E261 which we intend to clear up soon.
2017-06-04 09:18:23 -07:00
Aditya Bansal a9e6106d54 pep8: Add compliance with rule E261 lint. 2017-06-04 09:18:22 -07:00
Tim Abbott ba74d74dca lint: Fix incorrectly placed docstring. 2017-06-03 16:02:24 -07:00
Yago González 9602f73d05 lint: Add validation for the Swagger file. 2017-06-03 15:48:04 -07:00
Yago González 99b004942f lint: Explain the purpose of target_langs.
Tweaked by tabbott to be a longer docstring.
2017-06-03 15:48:00 -07:00
Eeshan Garg 829491fde7 webhooks/papertrail: Migrate docs to Markdown. 2017-06-01 22:16:12 -07:00
Eeshan Garg 5d868b1612 webhooks/helloworld: Migrate docs to Markdown. 2017-06-01 22:16:12 -07:00
Eeshan Garg ccd079b425 webhooks/solano: Migrate docs to Markdown. 2017-06-01 22:16:12 -07:00
Umair Khan bf2bc8d44b lint: Allow empty string in alt arg. 2017-06-01 22:07:43 -07:00
Rohitt Vashishtha db9918f3d6 bots: Move contrib_bots to api/bots*.
This will make it convenient to include these bots in Zulip API
releases on pypi.

Fix #5009.
2017-06-01 12:31:54 -07:00
Pweaver (Paul Weaver) bbb3aad611 Add typescript support to asset compilation. 2017-05-28 17:28:28 -07:00
Umair Khan 7f43ec05f2 lint: Add rule to check alt arg for translation. 2017-05-26 15:28:46 -07:00
Eeshan Garg e9b3b56105 webhooks/appfollow: Use parametric Markdown macros. 2017-05-24 17:00:19 -07:00
Eklavya Sharma b805e0ca10 mypy: Assert non-Noneness.
Convince mypy that an object cannot be None by using an assert.
2017-05-23 21:56:50 -07:00
Eeshan Garg ed8c6943f2 webhooks/airbrake: Use Markdown macros for documentation. 2017-05-22 18:19:14 -07:00
Aditya Bansal cd7fda993a pep8: Add compliance with rule E261 to lint. 2017-05-18 03:00:32 +05:30
Eeshan Garg 21e0db03e1 webhooks/appfollow: Migrate docs to Markdown. 2017-05-15 20:14:23 -07:00
Eeshan Garg 0b5711aa7e webhooks/airbrake: Migrate docs to Markdown. 2017-05-15 20:14:23 -07:00
Rishi Gupta 21a2c7b9d9 email: Remove translation tags from email templates.
The only thing being translated in any email was the title of the Zulip
header image in the html file, probably because not doing so breaks a linter
rule.

he name "Zulip" doesn't make sense to translate anyway.
2017-05-03 23:25:33 -07:00
Tim Abbott ec90f0c342 Disable gitlint commit message linter for now.
We've found a couple major issues that we need to fix:
* TRAVIS_COMMIT_RANGE being computed incorrectly in some cases (?!)
* The imperative linter could use some work.
2017-04-28 12:26:26 -07:00
Joshua Pan 64984f5bf4 Rename tools/lint-all to tools/lint.
Fixes #4574.
2017-04-21 14:57:47 -07:00