Mirror
/
zulip
mirror of https://github.com/zulip/zulip.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
zulip/tools/linter_lib/custom_check.py

819 lines
40 KiB
Python
Raw Normal View History

portico: Update registration form placeholders.
2017-06-23 02:16:10 +02:00
# -*- coding: utf-8 -*-
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
from __future__ import print_function
from __future__ import absolute_import
import os
import re
import traceback
linter: Update custom linters' visual output.
2017-07-06 06:54:30 +02:00
from .printer import print_err, colors
mypy: Improve typing in custom_check_file() in custom_check.py.
2017-08-08 04:13:25 +02:00
from typing import cast, Any, Callable, Dict, List, Optional, Tuple, Iterable
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
mypy: Remove un-necessary comment re mypy issue #3145. RuleList is required at global scope now, for later functions.
2017-11-05 00:48:08 +01:00
RuleList = List[Dict[str, Any]]
mypy: Add note re mypy supporting only global type aliases. https://github.com/python/mypy/issues/3145
2017-08-04 02:23:09 +02:00
linter: Move custom_check_file to global namespace. This is needed in order to mock the method when testing `custom_check.py`. The diff for this commit is a bit broken; all it really does is moving the method out of `build_custom_checkers`.
2017-09-11 19:56:19 +02:00
def custom_check_file(fn, identifier, rules, color, skip_rules=None, max_length=None):
mypy: Improve typing in custom_check_file() in custom_check.py.
2017-08-08 04:13:25 +02:00
# type: (str, str, RuleList, str, Optional[Iterable[str]], Optional[int]) -> bool
linter: Move custom_check_file to global namespace. This is needed in order to mock the method when testing `custom_check.py`. The diff for this commit is a bit broken; all it really does is moving the method out of `build_custom_checkers`.
2017-09-11 19:56:19 +02:00
failed = False
line_tups = []
for i, line in enumerate(open(fn)):
line_newline_stripped = line.strip('\n')
line_fully_stripped = line_newline_stripped.strip()
skip = False
mypy: Improve typing in custom_check_file() in custom_check.py.
2017-08-08 04:13:25 +02:00
for skip_rule in skip_rules or []:
if re.match(skip_rule, line):
linter: Move custom_check_file to global namespace. This is needed in order to mock the method when testing `custom_check.py`. The diff for this commit is a bit broken; all it really does is moving the method out of `build_custom_checkers`.
2017-09-11 19:56:19 +02:00
skip = True
if line_fully_stripped.endswith(' # nolint'):
continue
if skip:
continue
tup = (i, line, line_newline_stripped, line_fully_stripped)
line_tups.append(tup)
rules_to_apply = []
for rule in rules:
lint: Fix logic for exclude to handle directory trees. The previous exclude rules only allowed excluding a directory (and things in subdirectories would silently still be linted). Anyone using this would expect it to exclude a directory tree, so we make it do that.
2017-11-23 20:53:17 +01:00
excluded = False
for item in rule.get('exclude', set()):
if fn.startswith(item):
excluded = True
break
if excluded:
linter: Move custom_check_file to global namespace. This is needed in order to mock the method when testing `custom_check.py`. The diff for this commit is a bit broken; all it really does is moving the method out of `build_custom_checkers`.
2017-09-11 19:56:19 +02:00
continue
if rule.get("include_only"):
found = False
for item in rule.get("include_only", set()):
if item in fn:
found = True
if not found:
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
continue
linter: Move custom_check_file to global namespace. This is needed in order to mock the method when testing `custom_check.py`. The diff for this commit is a bit broken; all it really does is moving the method out of `build_custom_checkers`.
2017-09-11 19:56:19 +02:00
rules_to_apply.append(rule)
for rule in rules_to_apply:
exclude_lines = {
line for
(exclude_fn, line) in rule.get('exclude_line', set())
if exclude_fn == fn
}
pattern = rule['pattern']
Speed up custom checkers in `./tools/lint`. The main thing here is to make looping over lines be the inner loop, instead of looping over rules. This keeps regexes in cache, and it also avoids some O(N) checks. This is a significant speedup for me, reducing time from 16s to 11s.
2017-06-21 01:17:00 +02:00
for (i, line, line_newline_stripped, line_fully_stripped) in line_tups:
linter: Move custom_check_file to global namespace. This is needed in order to mock the method when testing `custom_check.py`. The diff for this commit is a bit broken; all it really does is moving the method out of `build_custom_checkers`.
2017-09-11 19:56:19 +02:00
if line_fully_stripped in exclude_lines:
exclude_lines.remove(line_fully_stripped)
continue
try:
line_to_check = line_fully_stripped
if rule.get('strip') is not None:
if rule['strip'] == '\n':
line_to_check = line_newline_stripped
else:
raise Exception("Invalid strip rule")
if re.search(pattern, line_to_check):
lint: Use new exclude_pattern system to clean up inline style lint.
2017-10-06 22:07:47 +02:00
if rule.get("exclude_pattern"):
if re.search(rule['exclude_pattern'], line_to_check):
continue
linter: Move custom_check_file to global namespace. This is needed in order to mock the method when testing `custom_check.py`. The diff for this commit is a bit broken; all it really does is moving the method out of `build_custom_checkers`.
2017-09-11 19:56:19 +02:00
print_err(identifier, color, '{} at {} line {}:'.format(
rule['description'], fn, i+1))
print_err(identifier, color, line)
failed = True
except Exception:
print("Exception with %s at %s line %s" % (rule['pattern'], fn, i+1))
traceback.print_exc()
if exclude_lines:
print('Please remove exclusions for file %s: %s' % (fn, exclude_lines))
linter: Add custom shebang rules section. Tweaked by tabbott for clarity and to use print_err.
2017-10-12 18:40:45 +02:00
# TODO: Move the below into more of a framework.
firstline = None
if line_tups:
firstline = line_tups[0][3] # line_fully_stripped for the first line.
linter: Move custom_check_file to global namespace. This is needed in order to mock the method when testing `custom_check.py`. The diff for this commit is a bit broken; all it really does is moving the method out of `build_custom_checkers`.
2017-09-11 19:56:19 +02:00
lastLine = None
for (i, line, line_newline_stripped, line_fully_stripped) in line_tups:
if isinstance(line, bytes):
line_length = len(line.decode("utf-8"))
else:
line_length = len(line)
if (max_length is not None and line_length > max_length and
'# type' not in line and 'test' not in fn and 'example' not in fn and
lint: Add exclude rules for URLs in code comments.
2017-11-22 06:53:35 +01:00
# Don't throw errors for markdown format URLs
lint: Fix use of re.match in judging long lines. The `re.match` function in the Python stdlib is a trap for the unwary, with surprising and asymmetrical semantics; we should probably add a lint rule to ban it entirely. The docstring says: > Try to apply the pattern at the start of the string, [...] In other words, it effectively adds a `^` at the start (or `\A`, where the distinction matters.) It's bad enough that this differs from what grep, sed, perl, less, and every other tool I can think of do when looking for matches to a regex; on top of that, it treats the beginning of the string differently from the end, for no obvious reason. The function that does what the rest of the world understands by "match against this regex" is `re.search`. In this case, it's unlikely that anyone intended for comments with URLs, or `api_url` references, to miss out on their respective exceptions to the long-line rule if they happen to start after the first column. So fix those rules by just switching to `re.search` with the same pattern. I think Markdown URL references may have to start at the beginning of the line, so I've left a `^` there to preserve -- but now make explicit -- the `re.match` behavior.
2017-11-28 05:39:24 +01:00
not re.search("^\[[ A-Za-z0-9_:,&()-]*\]: http.*", line) and
lint: Add exclude rules for URLs in code comments.
2017-11-22 06:53:35 +01:00
# Don't throw errors for URLs in code comments
lint: Fix use of re.match in judging long lines. The `re.match` function in the Python stdlib is a trap for the unwary, with surprising and asymmetrical semantics; we should probably add a lint rule to ban it entirely. The docstring says: > Try to apply the pattern at the start of the string, [...] In other words, it effectively adds a `^` at the start (or `\A`, where the distinction matters.) It's bad enough that this differs from what grep, sed, perl, less, and every other tool I can think of do when looking for matches to a regex; on top of that, it treats the beginning of the string differently from the end, for no obvious reason. The function that does what the rest of the world understands by "match against this regex" is `re.search`. In this case, it's unlikely that anyone intended for comments with URLs, or `api_url` references, to miss out on their respective exceptions to the long-line rule if they happen to start after the first column. So fix those rules by just switching to `re.search` with the same pattern. I think Markdown URL references may have to start at the beginning of the line, so I've left a `^` there to preserve -- but now make explicit -- the `re.match` behavior.
2017-11-28 05:39:24 +01:00
not re.search("[#].*http.*", line) and
not re.search("`\{\{ api_url \}\}[^`]+`", line) and
custom_check.py: Fix the comment for ignoring long lines. Change `#ignorelongline` to `# ignorelongline` as comments must a have whitespace after `#`.
2017-11-15 21:00:27 +01:00
"# ignorelongline" not in line and 'migrations' not in fn):
linter: Move custom_check_file to global namespace. This is needed in order to mock the method when testing `custom_check.py`. The diff for this commit is a bit broken; all it really does is moving the method out of `build_custom_checkers`.
2017-09-11 19:56:19 +02:00
print("Line too long (%s) at %s line %s: %s" % (len(line), fn, i+1, line_newline_stripped))
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
failed = True
linter: Move custom_check_file to global namespace. This is needed in order to mock the method when testing `custom_check.py`. The diff for this commit is a bit broken; all it really does is moving the method out of `build_custom_checkers`.
2017-09-11 19:56:19 +02:00
lastLine = line
linter: Add custom shebang rules section. Tweaked by tabbott for clarity and to use print_err.
2017-10-12 18:40:45 +02:00
if firstline:
if os.path.splitext(fn)[1] and 'zerver/' in fn:
shebang_rules = [{'pattern': '^#!',
'description': "zerver library code shouldn't have a shebang line."}]
else:
shebang_rules = [{'pattern': '#!/usr/bin/python',
'description': "Use `#!/usr/bin/env python3` instead of `#!/usr/bin/python`"},
{'pattern': '#!/usr/bin/env python$',
'description': "Use `#!/usr/bin/env python3` instead of `#!/usr/bin/env python`."}]
for rule in shebang_rules:
if re.search(rule['pattern'], firstline):
print_err(identifier, color,
'{} at {} line 1:'.format(rule['description'], fn))
print_err(identifier, color, firstline)
failed = True
linter: Move custom_check_file to global namespace. This is needed in order to mock the method when testing `custom_check.py`. The diff for this commit is a bit broken; all it really does is moving the method out of `build_custom_checkers`.
2017-09-11 19:56:19 +02:00
if lastLine and ('\n' not in lastLine):
print("No newline at the end of file. Fix with `sed -i '$a\\' %s`" % (fn,))
failed = True
Speed up custom checkers in `./tools/lint`. The main thing here is to make looping over lines be the inner loop, instead of looping over rules. This keeps regexes in cache, and it also avoids some O(N) checks. This is a significant speedup for me, reducing time from 16s to 11s.
2017-06-21 01:17:00 +02:00
linter: Move custom_check_file to global namespace. This is needed in order to mock the method when testing `custom_check.py`. The diff for this commit is a bit broken; all it really does is moving the method out of `build_custom_checkers`.
2017-09-11 19:56:19 +02:00
return failed
def build_custom_checkers(by_lang):
# type: (Dict[str, List[str]]) -> Tuple[Callable[[], bool], Callable[[], bool]]
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
linter: Add explanation for custom rule file delimitation operators.
2017-09-11 19:48:19 +02:00
# By default, a rule applies to all files within the extension for which it is specified (e.g. all .py files)
# There are three operators we can use to manually include or exclude files from linting for a rule:
# 'exclude': 'set([<path>, ...])' - if <path> is a filename, excludes that file.
# if <path> is a directory, excludes all files directly below the directory <path>.
# 'exclude_line': 'set([(<path>, <line>), ...])' - excludes all lines matching <line> in the file <path> from linting.
# 'include_only': 'set([<path>, ...])' - includes only those files where <path> is a substring of the filepath.
linter: Add trailing whitespace check for json files.
2017-08-15 17:54:29 +02:00
trailing_whitespace_rule = {
'pattern': '\s+$',
'strip': '\n',
'description': 'Fix trailing whitespace'
}
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
whitespace_rules = [
# This linter should be first since bash_rules depends on it.
linter: Add trailing whitespace check for json files.
2017-08-15 17:54:29 +02:00
trailing_whitespace_rule,
docs: Update links to ReadTheDocs to always use https. This is better security practice. We also add a lint rule to enforce this for the future.
2017-11-16 19:54:24 +01:00
{'pattern': 'http://zulip.readthedocs.io',
'description': 'Use HTTPS when linking to ReadTheDocs',
},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': '\t',
'strip': '\n',
linter: Remove redundant custom rule file delimitors.
2017-09-11 19:51:47 +02:00
'exclude': set(['tools/travis/success-http-headers.txt']),
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
'description': 'Fix tab-based whitespace'},
] # type: RuleList
lint: Ban two spaces after comma in CSS.
2017-10-18 19:11:33 +02:00
comma_whitespace_rule = [
lint: Ban two spaces after comma in JS code. We exclude the frontend tests, mostly because the lint rule isn't that precise, and the test code has some sample user input that's a bit funny.
2017-10-18 19:16:41 +02:00
{'pattern': ', {2,}[^#/ ]',
'exclude': set(['zerver/tests', 'frontend_tests/node_tests']),
lint: Ban two spaces after comma in CSS.
2017-10-18 19:11:33 +02:00
'description': "Remove multiple whitespaces after ','",
'good_lines': ['foo(1, 2, 3)', 'foo = bar # some inline comment'],
'bad_lines': ['foo(1, 2, 3)', 'foo(1, 2, 3)']},
] # type: RuleList
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
markdown_whitespace_rules = list([rule for rule in whitespace_rules if rule['pattern'] != '\s+$']) + [
# Two spaces trailing a line with other content is okay--it's a markdown line break.
# This rule finds one space trailing a non-space, three or more trailing spaces, and
# spaces on an empty line.
{'pattern': '((?<!\s)\s$)|(\s\s\s+$)|(^\s+$)',
'strip': '\n',
'description': 'Fix trailing whitespace'},
{'pattern': '^#+[A-Za-z0-9]',
'strip': '\n',
custom linter rules test: Add pattern test. The pattern test method `test_rule_patterns` tests each rule by fetching two strings from it: `test_good` and `test_bad`. Each string is then presented as an input file to `custom_check_file`, which should return True or False. All lines in a string need to end with `\n`. Since the linter expects an additional newline at the end of a file, the test case adds `\n` to each string on top of that. Fixes #6320.
2017-09-13 18:02:58 +02:00
'description': 'Missing space after # in heading',
'good_lines': ['### some heading', '# another heading'],
'bad_lines': ['###some heading', '#another heading']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
] # type: RuleList
js_rules = cast(RuleList, [
{'pattern': '[^_]function\(',
'description': 'The keyword "function" should be followed by a space'},
{'pattern': '.*blueslip.warning\(.*',
'description': 'The module blueslip has no function warning, try using blueslip.warn'},
{'pattern': '[)]{$',
'description': 'Missing space between ) and {'},
linter: Improve i18n related regexes. There are four regexes which try to ensure that the i18n strings are properly captured. 1) The one which disallows multiline strings. ``` i18n\.t\([^)]+[^,\{\)]$ // Disallows: i18n.t('some ' + 'text'); ``` 2) The one which disallows concatenation within argument to i18n.t(): ``` i18n\.t\([\'\"].+?[\'\"]\s*\+ // Disallows: i18n.t("some " + "text"); ``` 3) There are two which disallow concatenation with i18n.t(): ``` i18n\.t\(.+\).*\+ // Disallows: i18n.t('some text') + \+.*i18n\.t\(.+\) // Disallows: + i18n.t('some text') ``` The ideal case is that you try to bring the string argument to the i18n.t() on one line. In case this is not possible, you can do the following: ``` var1 = i18n.t("Some text to be translated"); var2 = i18n.t("Some more text to be translated"); complete = var1 + var2;
2017-09-08 09:35:11 +02:00
{'pattern': 'i18n\.t\([^)]+[^,\{\)]$',
lint: Add regex to disallow i18n string concatenation. Fixes #6398
2017-09-05 08:37:25 +02:00
'description': 'i18n string should not be a multiline string'},
linter: Improve i18n related regexes. There are four regexes which try to ensure that the i18n strings are properly captured. 1) The one which disallows multiline strings. ``` i18n\.t\([^)]+[^,\{\)]$ // Disallows: i18n.t('some ' + 'text'); ``` 2) The one which disallows concatenation within argument to i18n.t(): ``` i18n\.t\([\'\"].+?[\'\"]\s*\+ // Disallows: i18n.t("some " + "text"); ``` 3) There are two which disallow concatenation with i18n.t(): ``` i18n\.t\(.+\).*\+ // Disallows: i18n.t('some text') + \+.*i18n\.t\(.+\) // Disallows: + i18n.t('some text') ``` The ideal case is that you try to bring the string argument to the i18n.t() on one line. In case this is not possible, you can do the following: ``` var1 = i18n.t("Some text to be translated"); var2 = i18n.t("Some more text to be translated"); complete = var1 + var2;
2017-09-08 09:35:11 +02:00
{'pattern': 'i18n\.t\([\'\"].+?[\'\"]\s*\+',
'description': 'Do not concatenate arguments within i18n.t()'},
{'pattern': 'i18n\.t\(.+\).*\+',
lint: Add regex to disallow i18n string concatenation. Fixes #6398
2017-09-05 08:37:25 +02:00
'description': 'Do not concatenate i18n strings'},
linter: Improve i18n related regexes. There are four regexes which try to ensure that the i18n strings are properly captured. 1) The one which disallows multiline strings. ``` i18n\.t\([^)]+[^,\{\)]$ // Disallows: i18n.t('some ' + 'text'); ``` 2) The one which disallows concatenation within argument to i18n.t(): ``` i18n\.t\([\'\"].+?[\'\"]\s*\+ // Disallows: i18n.t("some " + "text"); ``` 3) There are two which disallow concatenation with i18n.t(): ``` i18n\.t\(.+\).*\+ // Disallows: i18n.t('some text') + \+.*i18n\.t\(.+\) // Disallows: + i18n.t('some text') ``` The ideal case is that you try to bring the string argument to the i18n.t() on one line. In case this is not possible, you can do the following: ``` var1 = i18n.t("Some text to be translated"); var2 = i18n.t("Some more text to be translated"); complete = var1 + var2;
2017-09-08 09:35:11 +02:00
{'pattern': '\+.*i18n\.t\(.+\)',
lint: Add regex to disallow i18n string concatenation. Fixes #6398
2017-09-05 08:37:25 +02:00
'description': 'Do not concatenate i18n strings'},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': '["\']json/',
'description': 'Relative URL for JSON route not supported by i18n'},
# This rule is constructed with + to avoid triggering on itself
{'pattern': " =" + '[^ =>~"]',
'description': 'Missing whitespace after "="'},
{'pattern': '^[ ]*//[A-Za-z0-9]',
'description': 'Missing space after // in comment'},
{'pattern': 'if[(]',
'description': 'Missing space between if and ('},
{'pattern': 'else{$',
'description': 'Missing space between else and {'},
{'pattern': '^else {$',
'description': 'Write JS else statements on same line as }'},
{'pattern': '^else if',
'description': 'Write JS else statements on same line as }'},
{'pattern': 'console[.][a-z]',
'exclude': set(['static/js/blueslip.js',
'frontend_tests/zjsunit',
'frontend_tests/casper_lib/common.js',
'frontend_tests/node_tests',
icons: Create framework for custom icons and add new bot icon.
2017-10-12 05:39:46 +02:00
'static/js/debug.js',
'tools/generate-custom-icon-webfont']),
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
'description': 'console.log and similar should not be used in webapp'},
{'pattern': '[.]text\(["\'][a-zA-Z]',
'description': 'Strings passed to $().text should be wrapped in i18n.t() for internationalization'},
{'pattern': 'compose_error\(["\']',
'description': 'Argument to compose_error should be a literal string enclosed '
'by i18n.t()'},
{'pattern': 'ui.report_success\(',
'description': 'Deprecated function, use ui_report.success.'},
{'pattern': 'report.success\(["\']',
'description': 'Argument to report_success should be a literal string enclosed '
'by i18n.t()'},
{'pattern': 'ui.report_error\(',
'description': 'Deprecated function, use ui_report.error.'},
{'pattern': 'report.error\(["\']',
'description': 'Argument to report_error should be a literal string enclosed '
'by i18n.t()'},
linter: Add rule against verbose $(document).ready() calls.
2017-10-05 16:03:58 +02:00
{'pattern': '\$\(document\)\.ready\(',
'description': "`Use $(f) rather than `$(document).ready(f)`",
'good_lines': ['$(function () {foo();}'],
'bad_lines': ['$(document).ready(function () {foo();}']},
lint: Add linter rule banning $.get and friends. These are not allowed in our style guide.
2018-02-14 01:47:09 +01:00
{'pattern': '[$][.](get|post|patch|delete|ajax)[(]',
'description': "Use channel module for AJAX calls",
'exclude': set([
# Internal modules can do direct network calls
'static/js/blueslip.js',
'static/js/channel.js',
# External modules that don't include channel.js
'static/js/stats/',
'static/js/portico/',
]),
'good_lines': ['channel.get(...)'],
'bad_lines': ['$.get()', '$.post()', '$.ajax()']},
linter: Add rule against using the `style` attribute. This has a ton of exclude rules, for two reasons: (1) We haven't been particularly systematic about avoiding unnecessary inline style in the past, so there's a lot of code we need to fix. (2) There are cases where one wants to dynamically compute style rules. For the latter category, ideally we'd figure out a way to exclude these automatically (e.g. checking for mustache tags in the style tag).
2017-10-06 10:12:47 +02:00
{'pattern': 'style ?=',
'description': "Avoid using the `style=` attribute; we prefer styling in CSS files",
'exclude': set([
copy_and_paste: Add a simple extensible test suite. This commit is mostly about adding the test suite; some of the results are things we'd like to change.
2017-11-23 21:17:08 +01:00
'frontend_tests/node_tests/copy_and_paste.js',
refactor: Extract tests for upload mechanics. This temporarily removes the tests for clear_out_file_list since fixing that test proved to be difficult.
2017-11-25 22:22:36 +01:00
'frontend_tests/node_tests/upload.js',
linter: Add rule against using the `style` attribute. This has a ton of exclude rules, for two reasons: (1) We haven't been particularly systematic about avoiding unnecessary inline style in the past, so there's a lot of code we need to fix. (2) There are cases where one wants to dynamically compute style rules. For the latter category, ideally we'd figure out a way to exclude these automatically (e.g. checking for mustache tags in the style tag).
2017-10-06 10:12:47 +02:00
'frontend_tests/node_tests/templates.js',
refactor: Extract upload mechanics to new JS module. Tweaked by tabbott to move changes from the next commit that are required for this to pass tests into this commit. Note that this exports a few items that were not previously exported.
2017-11-23 15:00:05 +01:00
'static/js/upload.js',
linter: Add rule against using the `style` attribute. This has a ton of exclude rules, for two reasons: (1) We haven't been particularly systematic about avoiding unnecessary inline style in the past, so there's a lot of code we need to fix. (2) There are cases where one wants to dynamically compute style rules. For the latter category, ideally we'd figure out a way to exclude these automatically (e.g. checking for mustache tags in the style tag).
2017-10-06 10:12:47 +02:00
'static/js/dynamic_text.js',
'static/js/stream_color.js',
]),
'good_lines': ['#my-style {color: blue;}'],
'bad_lines': ['<p style="color: blue;">Foo</p>', 'style = "color: blue;"']},
lint: Ban two spaces after comma in JS code. We exclude the frontend tests, mostly because the lint rule isn't that precise, and the test code has some sample user input that's a bit funny.
2017-10-18 19:16:41 +02:00
]) + whitespace_rules + comma_whitespace_rule
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
python_rules = cast(RuleList, [
{'pattern': '^(?!#)@login_required',
linter: Add tests for first 10 python custom rules.
2017-09-26 15:54:14 +02:00
'description': '@login_required is unsupported; use @zulip_login_required',
'good_lines': ['@zulip_login_required', '# foo @login_required'],
'bad_lines': ['@login_required', ' @login_required']},
lint: Ban general use of user_profile.save(). This often can cause minor caching problems. Obviously, it'd be better if we had access to the AST and thus could do this rule for UserProfile objects in general.
2017-11-20 19:52:10 +01:00
{'pattern': '^user_profile[.]save[(][)]',
'description': 'Always pass update_fields when saving user_profile objects',
'exclude_line': set([
('zerver/lib/actions.py', "user_profile.save() # Can't use update_fields because of how the foreign key works."),
]),
'exclude': set(['zerver/tests', 'zerver/lib/create_user.py']),
'good_lines': ['user_profile.save(update_fields=["pointer"])'],
'bad_lines': ['user_profile.save()']},
lint: Narrow the space-around-% patterns, remove exclusions. This lint rule has bitten me a couple of times in working on logging. These regex rules will inevitably be heuristic, but we can make it a bit more specific so that the heuristic mainly means it could occasionally miss something, rather than get in the way with an obviously wrong complaint.
2017-10-07 03:33:20 +02:00
{'pattern': '^[^"]*"[^"]*"%\(',
linter: Add tests for first 10 python custom rules.
2017-09-26 15:54:14 +02:00
'description': 'Missing space around "%"',
'good_lines': ['"%s" % ("foo")', '"%s" % (foo)'],
'bad_lines': ['"%s"%("foo")', '"%s"%(foo)']},
lint: Narrow the space-around-% patterns, remove exclusions. This lint rule has bitten me a couple of times in working on logging. These regex rules will inevitably be heuristic, but we can make it a bit more specific so that the heuristic mainly means it could occasionally miss something, rather than get in the way with an obviously wrong complaint.
2017-10-07 03:33:20 +02:00
{'pattern': "^[^']*'[^']*'%\(",
linter: Add tests for first 10 python custom rules.
2017-09-26 15:54:14 +02:00
'description': 'Missing space around "%"',
'good_lines': ["'%s' % ('foo')", "'%s' % (foo)"],
'bad_lines': ["'%s'%('foo')", "'%s'%(foo)"]},
lint: Prevent `self: Any` annotations. We should omit these for mypy. For most class definitions, mypy doesn't need `Any`, and it provides no real useful info. For clever monkeypatches, you should provide a more specific type than `Any`.
2017-12-08 17:03:58 +01:00
{'pattern': 'self: Any',
'description': 'you can omit Any annotation for self',
'good_lines': ['def foo (self):'],
'bad_lines': ['def foo(self: Any):']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
# This rule is constructed with + to avoid triggering on itself
{'pattern': " =" + '[^ =>~"]',
linter: Add tests for first 10 python custom rules.
2017-09-26 15:54:14 +02:00
'description': 'Missing whitespace after "="',
'good_lines': ['a = b', '5 == 6'],
'bad_lines': ['a =b', 'asdf =42']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': '":\w[^"]*$',
linter: Add tests for first 10 python custom rules.
2017-09-26 15:54:14 +02:00
'description': 'Missing whitespace after ":"',
'good_lines': ['"foo": bar', '"some:string:with:colons"'],
'bad_lines': ['"foo":bar', '"foo":1']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': "':\w[^']*$",
linter: Add tests for first 10 python custom rules.
2017-09-26 15:54:14 +02:00
'description': 'Missing whitespace after ":"',
'good_lines': ["'foo': bar", "'some:string:with:colons'"],
'bad_lines': ["'foo':bar", "'foo':1"]},
linter: Remove unnecessary escape sequences in custom rules.
2017-09-27 09:47:46 +02:00
{'pattern': "^\s+#\w",
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
'strip': '\n',
remote dev: Add code and instructions for creating digital ocean droplets. Mostly copied from the zulip/zulip-gci repository, but with some changes to wordings and code cleanup for linters.
2017-10-29 02:02:58 +01:00
'exclude': set(['tools/droplets/create.py']),
linter: Add tests for first 10 python custom rules.
2017-09-26 15:54:14 +02:00
'description': 'Missing whitespace after "#"',
'good_lines': ['a = b # some operation', '1+2 # 3 is the result'],
'bad_lines': [' #some operation', ' #not valid!!!']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': "assertEquals[(]",
linter: Add tests for first 10 python custom rules.
2017-09-26 15:54:14 +02:00
'description': 'Use assertEqual, not assertEquals (which is deprecated).',
'good_lines': ['assertEqual(1, 2)'],
'bad_lines': ['assertEquals(1, 2)']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': "== None",
linter: Add tests for first 10 python custom rules.
2017-09-26 15:54:14 +02:00
'description': 'Use `is None` to check whether something is None',
'good_lines': ['if foo is None'],
'bad_lines': ['foo == None']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': "type:[(]",
linter: Add tests for first 10 python custom rules.
2017-09-26 15:54:14 +02:00
'description': 'Missing whitespace after ":" in type annotation',
'good_lines': ['# type: (Any, Any)', 'colon:separated:string:containing:type:as:keyword'],
'bad_lines': ['# type:(Any, Any)']},
lint: Add a lint check for bare `type: ignore`s.
2017-07-28 00:59:36 +02:00
{'pattern': "type: ignore$",
'exclude': set(['tools/tests',
'zerver/lib/test_runner.py',
'zerver/tests']),
linter: Add example lines for several python rules.
2017-11-08 19:35:35 +01:00
'description': '"type: ignore" should always end with "# type: ignore # explanation for why"',
'good_lines': ['foo = bar # type: ignore # explanation'],
'bad_lines': ['foo = bar # type: ignore']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': "# type [(]",
linter: Add example lines for several python rules.
2017-11-08 19:35:35 +01:00
'description': 'Missing : after type in type annotation',
'good_lines': ['foo = 42 # type: int', '# type: (str, int) -> None'],
'bad_lines': ['# type (str, int) -> None']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': "#type",
linter: Add example lines for several python rules.
2017-11-08 19:35:35 +01:00
'description': 'Missing whitespace after "#" in type annotation',
'good_lines': ['foo = 42 # type: int'],
'bad_lines': ['foo = 42 #type: int']},
linter: Extend no-space-before-bracket linting rule.
2017-10-05 15:09:46 +02:00
{'pattern': r'\b(if|else|while)[(]',
'description': 'Put a space between statements like if, else, etc. and (.',
'good_lines': ['if (1 == 2):', 'while (foo == bar):'],
'bad_lines': ['if(1 == 2):', 'while(foo == bar):']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': ", [)]",
linter: Add example lines for several python rules.
2017-11-08 19:35:35 +01:00
'description': 'Unnecessary whitespace between "," and ")"',
'good_lines': ['foo = (1, 2, 3,)', 'foo(bar, 42)'],
'bad_lines': ['foo = (1, 2, 3, )']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': "% [(]",
linter: Add example lines for several python rules.
2017-11-08 19:35:35 +01:00
'description': 'Unnecessary whitespace between "%" and "("',
'good_lines': ['"foo %s bar" % ("baz",)'],
'bad_lines': ['"foo %s bar" % ("baz",)']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
# This next check could have false positives, but it seems pretty
# rare; if we find any, they can be added to the exclude list for
# this rule.
linter: Make string formatting custom rule more complete.
2017-11-08 19:34:49 +01:00
{'pattern': ' % [a-zA-Z0-9_."\']*\)?$',
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
'exclude_line': set([
('tools/tests/test_template_parser.py', '{% foo'),
]),
linter: Add example lines for several python rules.
2017-11-08 19:35:35 +01:00
'description': 'Used % comprehension without a tuple',
'good_lines': ['"foo %s bar" % ("baz",)'],
'bad_lines': ['"foo %s bar" % "baz"']},
linter: Make string formatting custom rule more complete.
2017-11-08 19:34:49 +01:00
{'pattern': '.*%s.* % \([a-zA-Z0-9_."\']*\)$',
linter: Add example lines for several python rules.
2017-11-08 19:35:35 +01:00
'description': 'Used % comprehension without a tuple',
'good_lines': ['"foo %s bar" % ("baz",)"'],
'bad_lines': ['"foo %s bar" % ("baz")']},
lint: Add a linter rule banning use of sudo in most scripts. This should help prevent against bugs where we accidentally introduce use of sudo somewhere in the production installer or upgrade code path (these used to happen all the time), which doesn't work on production systems that don't have sudo setup.
2018-03-02 01:27:08 +01:00
{'pattern': 'sudo',
'include_only': set(['scripts/']),
'exclude': set(['scripts/lib/setup_venv.py']),
'exclude_line': set([
('scripts/lib/zulip_tools.py', '# We need sudo here, since the path will be under /srv/ in the'),
('scripts/lib/zulip_tools.py', 'subprocess.check_call(["sudo", "/bin/bash", "-c",'),
('scripts/lib/zulip_tools.py', 'subprocess.check_call(["sudo", "rm", "-rf", directory])'),
]),
'description': 'Most scripts are intended to run on systems without sudo.',
'good_lines': ['subprocess.check_call(["ls"])'],
'bad_lines': ['subprocess.check_call(["sudo", "ls"])']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': 'django.utils.translation',
linter: Fix `include_only` rule path comparison. In order for the `include_only` linter rule to not have any side effects, we need to explicitly add a trailing '/' after every directory we want to include.
2017-06-20 22:27:49 +02:00
'include_only': set(['test/']),
linter: Add example lines for several python rules.
2017-11-08 19:35:35 +01:00
'description': 'Test strings should not be tagged for translation',
'good_lines': [''],
'bad_lines': ['django.utils.translation']},
lint: Prevent "userid" in code. We should always use user_id for consistency sake.
2017-10-07 17:55:36 +02:00
{'pattern': 'userid',
linter: Add example lines for several python rules.
2017-11-08 19:35:35 +01:00
'description': 'We prefer user_id over userid.',
'good_lines': ['id = alice.user_id'],
'bad_lines': ['id = alice.userid']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': 'json_success\({}\)',
linter: Add example lines for several python rules.
2017-11-08 19:35:35 +01:00
'description': 'Use json_success() to return nothing',
'good_lines': ['return json_success()'],
'bad_lines': ['return json_success({})']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': '\Wjson_error\(_\(?\w+\)',
'exclude': set(['zerver/tests']),
linter: Add example lines for several python rules.
2017-11-08 19:35:35 +01:00
'description': 'Argument to json_error should be a literal string enclosed by _()',
'good_lines': ['return json_error(_("string"))'],
'bad_lines': ['return json_error(_variable)', 'return json_error(_(variable))']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': '\Wjson_error\([\'"].+[),]$',
'exclude': set(['zerver/tests']),
'exclude_line': set([
# We don't want this string tagged for translation.
('zerver/views/compatibility.py', 'return json_error("Client is too old")'),
]),
'description': 'Argument to json_error should a literal string enclosed by _()'},
# To avoid JsonableError(_variable) and JsonableError(_(variable))
{'pattern': '\WJsonableError\(_\(?\w.+\)',
'exclude': set(['zerver/tests']),
'description': 'Argument to JsonableError should be a literal string enclosed by _()'},
{'pattern': '\WJsonableError\(["\'].+\)',
'exclude': set(['zerver/tests']),
'description': 'Argument to JsonableError should be a literal string enclosed by _()'},
{'pattern': '([a-zA-Z0-9_]+)=REQ\([\'"]\\1[\'"]',
'description': 'REQ\'s first argument already defaults to parameter name'},
{'pattern': 'self\.client\.(get|post|patch|put|delete)',
'description': \
'''Do not call self.client directly for put/patch/post/get.
See WRAPPER_COMMENT in test_helpers.py for details.
'''},
# Directly fetching Message objects in e.g. views code is often a security bug.
linter: Remove unnecessary escape sequences in custom rules.
2017-09-27 09:47:46 +02:00
{'pattern': '[^r]Message.objects.get',
onboarding: Add turtle reaction to an initial message.
2017-08-02 06:02:06 +02:00
'exclude': set(["zerver/tests",
"zerver/lib/onboarding.py",
features: Update screenshot and add tool to generate. This adds a new management command that should make generating or updating screenshots a bit easier in the future.
2017-08-06 09:56:29 +02:00
"zilencer/management/commands/add_mock_conversation.py",
onboarding: Add turtle reaction to an initial message.
2017-08-02 06:02:06 +02:00
"zerver/worker/queue_processors.py"]),
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
'description': 'Please use access_message() to fetch Message objects',
},
linter: Remove unnecessary escape sequences in custom rules.
2017-09-27 09:47:46 +02:00
{'pattern': 'Stream.objects.get',
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
'include_only': set(["zerver/views/"]),
'description': 'Please use access_stream_by_*() to fetch Stream objects',
},
{'pattern': 'get_stream[(]',
'include_only': set(["zerver/views/", "zerver/lib/actions.py"]),
'exclude_line': set([
# This one in check_message is kinda terrible, since it's
# how most instances are written, but better to exclude something than nothing
('zerver/lib/actions.py', 'stream = get_stream(stream_name, realm)'),
actions: Send new user messages to signup_notifications_stream.
2017-10-04 02:01:22 +02:00
('zerver/lib/actions.py', 'get_stream(admin_realm_signup_notifications_stream, admin_realm)'),
lint: Ban use of get_stream in the rest of messages.py. This will help avoid potential future security bugs.
2017-08-15 18:58:29 +02:00
# Here we need get_stream to access streams you've since unsubscribed from.
('zerver/views/messages.py', 'stream = get_stream(operand, self.user_profile.realm)'),
Have exclude_topic_mutes() accept a stream id. We want to convert stream names to stream ids as close to the "edges" of our system as possible, so we let our caller do the work of finding the stream id for a stream narrow.
2017-09-17 20:19:12 +02:00
# Use stream_id to exclude mutes.
('zerver/views/messages.py', 'stream_id = get_stream(stream_name, user_profile.realm).id'),
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
]),
'description': 'Please use access_stream_by_*() to fetch Stream objects',
},
linter: Remove unnecessary escape sequences in custom rules.
2017-09-27 09:47:46 +02:00
{'pattern': 'Stream.objects.filter',
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
'include_only': set(["zerver/views/"]),
'description': 'Please use access_stream_by_*() to fetch Stream objects',
},
{'pattern': '^from (zerver|analytics|confirmation)',
'include_only': set(["/migrations/"]),
Add migration to fix unread messages.
2017-09-01 14:22:18 +02:00
'exclude': set([
'zerver/migrations/0032_verify_all_medium_avatar_images.py',
'zerver/migrations/0041_create_attachments_for_old_messages.py',
'zerver/migrations/0060_move_avatars_to_be_uid_based.py',
'zerver/migrations/0104_fix_unreads.py',
]),
docs: Reorganize developer docs to improve navigation. This commit helps reduce clutter on the navigation sidebar. Creates new directories and moves relevant files into them. Modifies index.rst, symlinks, and image paths accordingly. This commit also enables expandable/collapsible navigation items, renames files in docs/development and docs/production, modifies /tools/test-documentation so that it overrides a theme setting, Also updates links to other docs, file paths in the codebase that point to developer documents, and files that should be excluded from lint tests. Note that this commit does not update direct links to zulip.readthedocs.io in the codebase; those will be resolved in an upcoming follow-up commit (it'll be easier to verify all the links once this is merged and ReadTheDocs is updated). Fixes #5265.
2017-11-08 17:55:36 +01:00
'description': "Don't import models or other code in migrations; see docs/subsystems/schema-migrations.md",
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
},
{'pattern': 'datetime[.](now|utcnow)',
'include_only': set(["zerver/", "analytics/"]),
'description': "Don't use datetime in backend code.\n"
docs: Reorganize developer docs to improve navigation. This commit helps reduce clutter on the navigation sidebar. Creates new directories and moves relevant files into them. Modifies index.rst, symlinks, and image paths accordingly. This commit also enables expandable/collapsible navigation items, renames files in docs/development and docs/production, modifies /tools/test-documentation so that it overrides a theme setting, Also updates links to other docs, file paths in the codebase that point to developer documents, and files that should be excluded from lint tests. Note that this commit does not update direct links to zulip.readthedocs.io in the codebase; those will be resolved in an upcoming follow-up commit (it'll be easier to verify all the links once this is merged and ReadTheDocs is updated). Fixes #5265.
2017-11-08 17:55:36 +01:00
"See https://zulip.readthedocs.io/en/latest/contributing/code-style.html#naive-datetime-objects",
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
},
{'pattern': 'render_to_response\(',
'description': "Use render() instead of render_to_response().",
},
python: Add lint rule banning 'from os.path import'. This enforces our use of a consistent style in how we access Python modules; "from os.path import dirname" is a particularly popular abbreviation inconsistent with our style, and so it deserves a lint rule. Commit message and error text tweaked by tabbott. Fixes #6543.
2017-09-22 08:15:01 +02:00
{'pattern': 'from os.path',
'description': "Don't use from when importing from the standard library",
},
linter: Add lint rule banning 'import os.path'
2017-09-22 18:30:18 +02:00
{'pattern': 'import os.path',
'description': "Use import os instead of import os.path",
},
linter: Add rule against using the deprecated Logging.warn. Tweaked by tabbott to improve the test cases.
2017-10-02 11:12:52 +02:00
{'pattern': '(logging|logger)\.warn\W',
'description': "Logger.warn is a deprecated alias for Logger.warning; Use 'warning' instead of 'warn'.",
'good_lines': ["logging.warning('I am a warning.')", "logger.warning('warning')"],
'bad_lines': ["logging.warn('I am a warning.')", "logger.warn('warning')"]},
linter: Add rule against using `pk` instead of `id`. There's one exception here, for model._meta.pk. To support this nicely, we added this exclude-pattern feature.
2017-10-06 13:17:23 +02:00
{'pattern': '\.pk',
'exclude_pattern': '[.]_meta[.]pk',
'description': "Use `id` instead of `pk`.",
'good_lines': ['if my_django_model.id == 42', 'self.user_profile._meta.pk'],
linter: Add rule against multiple whitespaces after comma.
2017-10-18 11:12:55 +02:00
'bad_lines': ['if my_django_model.pk == 42']},
linter: Require Py3 function annotations, with some exceptions. Main exceptions are scripts/tools/puppet & tests. Other current exclusions: - api_test_helpers.py (avoid changing test code); - cache.py, due to comments in file; - models.py, due to failure on QuerySet[Message]; - stream_subscription.py, due to failure on QuerySet[Subscription]; - tornado/descriptors.py; - views/streams.py, due to remaining FuncKwargPair issue; - zthumbor/, since thumbor is in python2. Tweaked by tabbott to partially document the stauts in comments.
2018-03-12 00:09:33 +01:00
{'pattern': '^[ ]*# type: \(',
'exclude': set([
# These directories, especially scripts/ and puppet/,
# have tools that need to run before a Zulip environment
# is provisioned; in some of those, the `typing` module
# might not be available yet, so care is required.
'scripts/',
'tools/',
'puppet/',
# Zerver files that we should just clean.
'zerver/tests',
'zerver/lib/api_test_helpers.py',
'zerver/lib/request.py',
'zerver/views/streams.py',
# thumbor is (currently) python2 only
'zthumbor/',
]),
'description': 'Comment-style function type annotation. Use Python3 style annotations instead.',
},
lint: Ban two spaces after comma in CSS.
2017-10-18 19:11:33 +02:00
]) + whitespace_rules + comma_whitespace_rule
mypy: Fix errors in linter. We should figure out a better way to manage this issue.
2018-03-02 05:31:46 +01:00
bash_rules = cast(RuleList, [
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': '#!.*sh [-xe]',
'description': 'Fix shebang line with proper call to /usr/bin/env for Bash path, change -x|-e switches'
' to set -x|set -e'},
lint: Add a linter rule banning use of sudo in most scripts. This should help prevent against bugs where we accidentally introduce use of sudo somewhere in the production installer or upgrade code path (these used to happen all the time), which doesn't work on production systems that don't have sudo setup.
2018-03-02 01:27:08 +01:00
{'pattern': 'sudo',
'description': 'Most scripts are intended to work on systems without sudo',
'include_only': set(['scripts/']),
'exclude': set([
'scripts/lib/install',
'scripts/lib/create-zulip-admin',
'scripts/setup/terminate-psql-sessions',
'scripts/setup/configure-rabbitmq'
]), },
mypy: Fix errors in linter. We should figure out a better way to manage this issue.
2018-03-02 05:31:46 +01:00
]) + whitespace_rules[0:1]
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
css_rules = cast(RuleList, [
css-minifier: Lint for '+' operator in calc(). Fixes: #8403.
2018-02-25 19:49:13 +01:00
{'pattern': 'calc\([^+]+\+[^+]+\)',
'description': "Avoid using calc with '+' operator. See #8403 : in CSS.",
'good_lines': ["width: calc(20% - -14px);"],
'bad_lines': ["width: calc(20% + 14px);"]},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': '^[^:]*:\S[^:]*;$',
linter: Add good/bad lines for CSS and HTML.
2017-12-01 17:11:55 +01:00
'description': "Missing whitespace after : in CSS",
'good_lines': ["background-color: white;", "text-size: 16px;"],
'bad_lines': ["background-color:white;", "text-size:16px;"]},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': '[a-z]{',
linter: Add good/bad lines for CSS and HTML.
2017-12-01 17:11:55 +01:00
'description': "Missing whitespace before '{' in CSS.",
'good_lines': ["input {", "body {"],
'bad_lines': ["input{", "body{"]},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': 'https://',
linter: Add good/bad lines for CSS and HTML.
2017-12-01 17:11:55 +01:00
'description': "Zulip CSS should have no dependencies on external resources",
'good_lines': ['background: url(/static/images/landing-page/pycon.jpg);'],
'bad_lines': ['background: url(https://example.com/image.png);']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': '^[ ][ ][a-zA-Z0-9]',
'description': "Incorrect 2-space indentation in CSS",
'exclude': set(['static/third/thirdparty-fonts.css']),
linter: Add good/bad lines for CSS and HTML.
2017-12-01 17:11:55 +01:00
'strip': '\n',
'good_lines': [" color: white;", "color: white;"],
'bad_lines': [" color: white;"]},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': '{\w',
linter: Add good/bad lines for CSS and HTML.
2017-12-01 17:11:55 +01:00
'description': "Missing whitespace after '{' in CSS (should be newline).",
'good_lines': ["{\n"],
'bad_lines': ["{color: LightGoldenRodYellow;"]},
{'pattern': ' thin[ ;]',
'description': "thin CSS attribute is under-specified, please use 1px.",
'good_lines': ["border-width: 1px;"],
'bad_lines': ["border-width: thin;", "border-width: thin solid black;"]},
{'pattern': ' medium[ ;]',
'description': "medium CSS attribute is under-specified, please use pixels.",
'good_lines': ["border-width: 3px;"],
'bad_lines': ["border-width: medium;", "border: medium solid black;"]},
{'pattern': ' thick[ ;]',
'description': "thick CSS attribute is under-specified, please use pixels.",
'good_lines': ["border-width: 5px;"],
'bad_lines': ["border-width: thick;", "border: thick solid black;"]},
lint: Ban two spaces after comma in CSS.
2017-10-18 19:11:33 +02:00
]) + whitespace_rules + comma_whitespace_rule
mypy: Fix annotations for linter.
2017-10-18 19:50:25 +02:00
prose_style_rules = cast(RuleList, [
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': '[^\/\#\-\"]([jJ]avascript)', # exclude usage in hrefs/divs
'description': "javascript should be spelled JavaScript"},
{'pattern': '[^\/\-\.\"\'\_\=\>]([gG]ithub)[^\.\-\_\"\<]', # exclude usage in hrefs/divs
'description': "github should be spelled GitHub"},
{'pattern': '[oO]rganisation', # exclude usage in hrefs/divs
docs: Expand the french translation style guide. As the french word organisation is used, the related line is excluded from the linter spell check.
2017-09-13 18:45:06 +02:00
'description': "Organization is spelled with a z",
docs: Reorganize developer docs to improve navigation. This commit helps reduce clutter on the navigation sidebar. Creates new directories and moves relevant files into them. Modifies index.rst, symlinks, and image paths accordingly. This commit also enables expandable/collapsible navigation items, renames files in docs/development and docs/production, modifies /tools/test-documentation so that it overrides a theme setting, Also updates links to other docs, file paths in the codebase that point to developer documents, and files that should be excluded from lint tests. Note that this commit does not update direct links to zulip.readthedocs.io in the codebase; those will be resolved in an upcoming follow-up commit (it'll be easier to verify all the links once this is merged and ReadTheDocs is updated). Fixes #5265.
2017-11-08 17:55:36 +01:00
'exclude_line': [('docs/translating/french.md', '* organization - **organisation**')]},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': '!!! warning',
'description': "!!! warning is invalid; it's spelled '!!! warn'"},
linter: Add rule against misspelled "Terms of Service".
2017-09-07 11:30:43 +02:00
{'pattern': 'Terms of service',
'description': "The S in Terms of Service is capitalized"},
mypy: Fix annotations for linter.
2017-10-18 19:50:25 +02:00
]) + comma_whitespace_rule
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
html_rules = whitespace_rules + prose_style_rules + [
lint: Clean up translation tag lint for filter settings.
2018-03-02 18:30:42 +01:00
{'pattern': 'placeholder="[^{#](?:(?!\.com).)+$',
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
'description': "`placeholder` value should be translatable.",
'exclude_line': [('templates/zerver/register.html', 'placeholder="acme"'),
lint: Add a translation tag for marketing.
2018-03-02 18:28:57 +01:00
('templates/zerver/register.html', 'placeholder="Acme or Aκμή"')],
linter: Add good/bad lines for CSS and HTML.
2017-12-01 17:11:55 +01:00
'good_lines': ['<input class="stream-list-filter" type="text" placeholder="{{ _(\'Search streams\') }}" />'],
template_parser: Add parsing support for self closing tags as per HTML5. In this commit we add support for some tags which are also called void-elements according to http://w3c.github.io/html/syntax.html#void-elements to be parsed by our template parser and get tagged as singleton_html_tags. Fixes: #8387.
2018-02-15 22:16:40 +01:00
'bad_lines': ['<input placeholder="foo">']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': "placeholder='[^{]",
linter: Add good/bad lines for CSS and HTML.
2017-12-01 17:11:55 +01:00
'description': "`placeholder` value should be translatable.",
'good_lines': ['<input class="stream-list-filter" type="text" placeholder="{{ _(\'Search streams\') }}" />'],
template_parser: Add parsing support for self closing tags as per HTML5. In this commit we add support for some tags which are also called void-elements according to http://w3c.github.io/html/syntax.html#void-elements to be parsed by our template parser and get tagged as singleton_html_tags. Fixes: #8387.
2018-02-15 22:16:40 +01:00
'bad_lines': ["<input placeholder='foo'>"]},
lint: Require translation tags on aria-label tags. Since these are, by their nature, directly user-facing strings, they should always be tagged for translation.
2017-07-14 23:48:17 +02:00
{'pattern': "aria-label='[^{]",
linter: Add good/bad lines for CSS and HTML.
2017-12-01 17:11:55 +01:00
'description': "`aria-label` value should be translatable.",
'good_lines': ['<button type="button" class="close close-alert-word-status" aria-label="{{t \'Close\' }}">'],
'bad_lines': ["<button aria-label='foo'></button>"]},
lint: Require translation tags on aria-label tags. Since these are, by their nature, directly user-facing strings, they should always be tagged for translation.
2017-07-14 23:48:17 +02:00
{'pattern': 'aria-label="[^{]',
docs: Add rtd layout template. The readthedocs theme overrides a few settings in their layout template. We might want to change some settings back to their default values. This commit copies the original readthedocs layout file from https://github.com/rtfd/sphinx_rtd_theme/blob/master/sphinx_rtd_theme/layout.html to _templates/layout.html, and excludes it from lint and template checks. Addresses #7417.
2017-11-21 18:49:45 +01:00
'description': "`aria-label` value should be translatable.",
linter: Add good/bad lines for CSS and HTML.
2017-12-01 17:11:55 +01:00
'good_lines': ['<button type="button" class="close close-alert-word-status" aria-label="{{t \'Close\' }}">'],
'bad_lines': ['<button aria-label="foo"></button>']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': 'script src="http',
linter: Add good/bad lines for CSS and HTML.
2017-12-01 17:11:55 +01:00
'description': "Don't directly load dependencies from CDNs. See docs/subsystems/front-end-build-process.md",
/billing/: Rename "payment.html" => "billing.html". This matches the URL path /billing/ to the filename "billing.html".
2018-01-30 01:30:02 +01:00
'exclude': set(["templates/zilencer/billing.html"]),
linter: Add good/bad lines for CSS and HTML.
2017-12-01 17:11:55 +01:00
'good_lines': ["{{ render_bundle('landing-page') }}"],
'bad_lines': ['<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script>']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': "title='[^{]",
linter: Add good/bad lines for CSS and HTML.
2017-12-01 17:11:55 +01:00
'description': "`title` value should be translatable.",
'good_lines': ['<link rel="author" title="{{ _(\'About these documents\') }}" />'],
'bad_lines': ["<p title='foo'></p>"]},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': 'title="[^{\:]',
'exclude_line': set([
('templates/zerver/markdown_help.html',
'<td><img alt=":heart:" class="emoji" src="/static/generated/emoji/images/emoji/heart.png" title=":heart:" /></td>')
]),
'exclude': set(["templates/zerver/emails"]),
'description': "`title` value should be translatable."},
{'pattern': '\Walt=["\'][^{"\']',
'description': "alt argument should be enclosed by _() or it should be an empty string.",
'exclude': set(['static/templates/settings/display-settings.handlebars',
'templates/zerver/keyboard_shortcuts.html',
'templates/zerver/markdown_help.html']),
linter: Add good/bad lines for CSS and HTML.
2017-12-01 17:11:55 +01:00
'good_lines': ['<img src="{{source_url}}" alt="{{ _(name) }}" />', '<img alg="" />'],
'bad_lines': ['<img alt="Foo Image" />']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': '\Walt=["\']{{ ?["\']',
'description': "alt argument should be enclosed by _().",
linter: Add good/bad lines for CSS and HTML.
2017-12-01 17:11:55 +01:00
'good_lines': ['<img src="{{source_url}}" alt="{{ _(name) }}" />'],
'bad_lines': ['<img alt="{{ " />']},
linter: Add rule against using inline event handlers.
2017-10-05 16:42:08 +02:00
{'pattern': r'\bon\w+ ?=',
'description': "Don't use inline event handlers (onclick=, etc. attributes) in HTML. Instead,"
"attach a jQuery event handler ($('#foo').on('click', function () {...})) when "
"the DOM is ready (inside a $(function () {...}) block).",
'exclude': set(['templates/zerver/dev_login.html']),
'good_lines': ["($('#foo').on('click', function () {}"],
'bad_lines': ["<button id='foo' onclick='myFunction()'>Foo</button>", "<input onchange='myFunction()'>"]},
linter: Add rule against using the `style` attribute. This has a ton of exclude rules, for two reasons: (1) We haven't been particularly systematic about avoiding unnecessary inline style in the past, so there's a lot of code we need to fix. (2) There are cases where one wants to dynamically compute style rules. For the latter category, ideally we'd figure out a way to exclude these automatically (e.g. checking for mustache tags in the style tag).
2017-10-06 10:12:47 +02:00
{'pattern': 'style ?=',
'description': "Avoid using the `style=` attribute; we prefer styling in CSS files",
lint: Use new exclude_pattern system to clean up inline style lint.
2017-10-06 22:07:47 +02:00
'exclude_pattern': r'.*style ?=["' + "'" + '](display: ?none|background: {{|color: {{|background-color: {{).*',
linter: Add rule against using the `style` attribute. This has a ton of exclude rules, for two reasons: (1) We haven't been particularly systematic about avoiding unnecessary inline style in the past, so there's a lot of code we need to fix. (2) There are cases where one wants to dynamically compute style rules. For the latter category, ideally we'd figure out a way to exclude these automatically (e.g. checking for mustache tags in the style tag).
2017-10-06 10:12:47 +02:00
'exclude': set([
# KaTeX output uses style attribute
'templates/zerver/markdown_help.html',
# 5xx page doesn't have external CSS
'static/html/5xx.html',
# Group PMs color is dynamically calculated
'static/templates/group_pms.handlebars',
lint: Use new exclude_pattern system to clean up inline style lint.
2017-10-06 22:07:47 +02:00
# exclude_pattern above handles color, but have other issues:
linter: Add rule against using the `style` attribute. This has a ton of exclude rules, for two reasons: (1) We haven't been particularly systematic about avoiding unnecessary inline style in the past, so there's a lot of code we need to fix. (2) There are cases where one wants to dynamically compute style rules. For the latter category, ideally we'd figure out a way to exclude these automatically (e.g. checking for mustache tags in the style tag).
2017-10-06 10:12:47 +02:00
'static/templates/draft.handlebars',
'static/templates/subscription.handlebars',
lint: Use new exclude_pattern system to clean up inline style lint.
2017-10-06 22:07:47 +02:00
'static/templates/single_message.handlebars',
linter: Add rule against using the `style` attribute. This has a ton of exclude rules, for two reasons: (1) We haven't been particularly systematic about avoiding unnecessary inline style in the past, so there's a lot of code we need to fix. (2) There are cases where one wants to dynamically compute style rules. For the latter category, ideally we'd figure out a way to exclude these automatically (e.g. checking for mustache tags in the style tag).
2017-10-06 10:12:47 +02:00
# Old-style email templates need to use inline style
# attributes; it should be possible to clean these up
# when we convert these templates to use premailer.
'templates/zerver/emails/digest.html',
'templates/zerver/emails/missed_message.html',
'templates/zerver/emails/email_base_messages.html',
lint: Exclude style= rules for email log. This is temporary, because this PR was written before these rules were put in place.
2017-10-11 19:41:40 +02:00
# Email log templates; should clean up.
'templates/zerver/email.html',
'templates/zerver/email_log.html',
lint: Use new exclude_pattern system to clean up inline style lint.
2017-10-06 22:07:47 +02:00
# Probably just needs to be changed to display: none so the exclude works
'templates/zerver/navbar.html',
# Needs the width cleaned up; display: none is fine
'static/templates/settings/account-settings.handlebars',
linter: Add rule against using the `style` attribute. This has a ton of exclude rules, for two reasons: (1) We haven't been particularly systematic about avoiding unnecessary inline style in the past, so there's a lot of code we need to fix. (2) There are cases where one wants to dynamically compute style rules. For the latter category, ideally we'd figure out a way to exclude these automatically (e.g. checking for mustache tags in the style tag).
2017-10-06 10:12:47 +02:00
# Inline styling for an svg; could be moved to CSS files?
'templates/zerver/landing_nav.html',
'templates/zerver/home.html',
'templates/zerver/features.html',
'templates/zerver/portico-header.html',
# Miscellaneous violations to be cleaned up
'static/templates/user_info_popover_title.handlebars',
'static/templates/subscription_invites_warning_modal.handlebars',
'templates/zerver/reset_confirm.html',
'templates/zerver/config_error.html',
'templates/zerver/dev_env_email_access_details.html',
'templates/zerver/confirm_continue_registration.html',
'templates/zerver/register.html',
'templates/zerver/accounts_send_confirm.html',
'templates/zerver/integrations/index.html',
'templates/zerver/help/main.html',
/api/: Add Markdown-based /api-new documentation. This commit allows for the /api-new/ page to rendered similarly to our /help pages. It's based on the old content for /api, but we're not replacing the old content yet, to give a bit of time to restructure things reasonably. Tweaked by eeshangarg and tabbott.
2017-07-25 02:37:04 +02:00
'templates/zerver/api/main.html',
linter: Add rule against using the `style` attribute. This has a ton of exclude rules, for two reasons: (1) We haven't been particularly systematic about avoiding unnecessary inline style in the past, so there's a lot of code we need to fix. (2) There are cases where one wants to dynamically compute style rules. For the latter category, ideally we'd figure out a way to exclude these automatically (e.g. checking for mustache tags in the style tag).
2017-10-06 10:12:47 +02:00
'templates/analytics/realm_summary_table.html',
'templates/corporate/zephyr.html',
'templates/corporate/zephyr-mirror.html',
]),
lint: Use new exclude_pattern system to clean up inline style lint.
2017-10-06 22:07:47 +02:00
'good_lines': ['#my-style {color: blue;}', 'style="display: none"', "style='display: none"],
linter: Add rule against using the `style` attribute. This has a ton of exclude rules, for two reasons: (1) We haven't been particularly systematic about avoiding unnecessary inline style in the past, so there's a lot of code we need to fix. (2) There are cases where one wants to dynamically compute style rules. For the latter category, ideally we'd figure out a way to exclude these automatically (e.g. checking for mustache tags in the style tag).
2017-10-06 10:12:47 +02:00
'bad_lines': ['<p style="color: blue;">Foo</p>', 'style = "color: blue;"']},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
] # type: RuleList
handlebars_rules = html_rules + [
{'pattern': "[<]script",
'description': "Do not use inline <script> tags here; put JavaScript in static/js instead."},
lint: Add rule for spacing inside Handlebars translation tags. This commit also includes the fixes to make all Handlebars templates compliant with the new rule.
2017-06-10 12:32:06 +02:00
{'pattern': '{{ t ("|\')',
'description': 'There should be no spaces before the "t" in a translation tag.'},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
{'pattern': "{{t '.*' }}[\.\?!]",
'description': "Period should be part of the translatable string."},
{'pattern': '{{t ".*" }}[\.\?!]',
'description': "Period should be part of the translatable string."},
{'pattern': "{{/tr}}[\.\?!]",
'description': "Period should be part of the translatable string."},
lint: Ban leading spaces in Handlebars translation tags.
2017-06-10 11:56:36 +02:00
{'pattern': '{{t ("|\') ',
'description': 'Translatable strings should not have leading spaces.'},
lint: Ban trailing spaces in Handlebars translation tags.
2017-06-10 12:11:33 +02:00
{'pattern': "{{t '[^']+ ' }}",
'description': 'Translatable strings should not have trailing spaces.'},
{'pattern': '{{t "[^"]+ " }}',
'description': 'Translatable strings should not have trailing spaces.'},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
]
jinja2_rules = html_rules + [
{'pattern': "{% endtrans %}[\.\?!]",
'description': "Period should be part of the translatable string."},
{'pattern': "{{ _(.+) }}[\.\?!]",
'description': "Period should be part of the translatable string."},
]
linter: Add trailing whitespace check for json files.
2017-08-15 17:54:29 +02:00
json_rules = [
lint: Clean up json_rules logic for tab-based whitespace. This both improves the comment to be more readable, and also uses the new and improved exclude feature to limit the exclusion to just the webhook fixtures (where it's needed). Also fixes a mypy error.
2017-11-23 20:56:05 +01:00
# Here, we don't use `whitespace_rules`, because the tab-based
# whitespace rule flags a lot of third-party JSON fixtures
# under zerver/webhooks that we want preserved verbatim. So
# we just include the trailing whitespace rule and a modified
# version of the tab-based whitespace rule (we can't just use
# exclude in whitespace_rules, since we only want to ignore
# JSON files with tab-based whitespace, not webhook code).
linter: Add trailing whitespace check for json files.
2017-08-15 17:54:29 +02:00
trailing_whitespace_rule,
lint: Ban tab-based whitespace in our our text fixtures. We don't want to do this for the webhook fixtures since those come from third-party code, but it makes sense to do this for hand-edited code.
2017-11-23 19:39:58 +01:00
{'pattern': '\t',
'strip': '\n',
lint: Clean up json_rules logic for tab-based whitespace. This both improves the comment to be more readable, and also uses the new and improved exclude feature to limit the exclusion to just the webhook fixtures (where it's needed). Also fixes a mypy error.
2017-11-23 20:56:05 +01:00
'exclude': set(['zerver/webhooks/']),
lint: Ban tab-based whitespace in our our text fixtures. We don't want to do this for the webhook fixtures since those come from third-party code, but it makes sense to do this for hand-edited code.
2017-11-23 19:39:58 +01:00
'description': 'Fix tab-based whitespace'},
tools/lint: Require a space after every `:` in JSON files.
2018-03-02 22:06:21 +01:00
{'pattern': ':[\"\[\{]',
'exclude': set(['zerver/webhooks/', 'zerver/fixtures/']),
'description': 'Require space after : in JSON'},
lint: Clean up json_rules logic for tab-based whitespace. This both improves the comment to be more readable, and also uses the new and improved exclude feature to limit the exclusion to just the webhook fixtures (where it's needed). Also fixes a mypy error.
2017-11-23 20:56:05 +01:00
] # type: RuleList
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
markdown_rules = markdown_whitespace_rules + prose_style_rules + [
{'pattern': '\[(?P<url>[^\]]+)\]\((?P=url)\)',
lint: Check for absolute/relative link style in developer docs. This should help avoid a pretty wide range of link bugs we've had in the past.
2017-11-16 21:44:16 +01:00
'description': 'Linkified markdown URLs should use cleaner <http://example.com> syntax.'},
{'pattern': 'https://zulip.readthedocs.io/en/latest/[a-zA-Z0-9]',
docs: Move readme-symlink symlink to be just called readme. This name was always a little confusing.
2017-11-17 06:43:01 +01:00
'exclude': ['docs/overview/contributing.md', 'docs/overview/readme.md'],
lint: Fix tools tests. These were broken when Travis wasn't running properly.
2017-11-16 22:59:21 +01:00
'include_only': set(['docs/']),
lint: Check for absolute/relative link style in developer docs. This should help avoid a pretty wide range of link bugs we've had in the past.
2017-11-16 21:44:16 +01:00
'description': "Use relatve links (../foo/bar.html) to other documents in docs/",
},
{'pattern': '\][(][^#h]',
'include_only': set(['README.md', 'CONTRIBUTING.md']),
'description': "Use absolute links from docs served by GitHub",
},
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
]
help_markdown_rules = markdown_rules + [
{'pattern': '[a-z][.][A-Z]',
'description': "Likely missing space after end of sentence"},
{'pattern': '[rR]ealm',
'description': "Realms are referred to as Organizations in user-facing docs."},
]
txt_rules = whitespace_rules
def check_custom_checks_py():
# type: () -> bool
failed = False
lint: Fix colors being randomized on every file with custom checks. Now, each custom check language gets a consistent color.
2017-07-28 00:59:53 +02:00
color = next(colors)
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
for fn in by_lang['py']:
if 'custom_check.py' in fn:
continue
Reduce python files line length limit to 110. License: Apache-2.0 Signed-off-by: rht <rhtbot@protonmail.com>
2017-11-04 13:21:48 +01:00
if custom_check_file(fn, 'py', python_rules, color, max_length=110):
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
failed = True
return failed
def check_custom_checks_nonpy():
# type: () -> bool
failed = False
lint: Fix colors being randomized on every file with custom checks. Now, each custom check language gets a consistent color.
2017-07-28 00:59:53 +02:00
color = next(colors)
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
for fn in by_lang['js']:
lint: Fix colors being randomized on every file with custom checks. Now, each custom check language gets a consistent color.
2017-07-28 00:59:53 +02:00
if custom_check_file(fn, 'js', js_rules, color):
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
failed = True
lint: Fix colors being randomized on every file with custom checks. Now, each custom check language gets a consistent color.
2017-07-28 00:59:53 +02:00
color = next(colors)
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
for fn in by_lang['sh']:
lint: Fix colors being randomized on every file with custom checks. Now, each custom check language gets a consistent color.
2017-07-28 00:59:53 +02:00
if custom_check_file(fn, 'sh', bash_rules, color):
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
failed = True
lint: Fix colors being randomized on every file with custom checks. Now, each custom check language gets a consistent color.
2017-07-28 00:59:53 +02:00
color = next(colors)
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
for fn in by_lang['css']:
lint: Fix colors being randomized on every file with custom checks. Now, each custom check language gets a consistent color.
2017-07-28 00:59:53 +02:00
if custom_check_file(fn, 'css', css_rules, color):
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
failed = True
lint: Fix colors being randomized on every file with custom checks. Now, each custom check language gets a consistent color.
2017-07-28 00:59:53 +02:00
color = next(colors)
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
for fn in by_lang['handlebars']:
lint: Fix colors being randomized on every file with custom checks. Now, each custom check language gets a consistent color.
2017-07-28 00:59:53 +02:00
if custom_check_file(fn, 'handlebars', handlebars_rules, color):
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
failed = True
lint: Fix colors being randomized on every file with custom checks. Now, each custom check language gets a consistent color.
2017-07-28 00:59:53 +02:00
color = next(colors)
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
for fn in by_lang['html']:
lint: Fix colors being randomized on every file with custom checks. Now, each custom check language gets a consistent color.
2017-07-28 00:59:53 +02:00
if custom_check_file(fn, 'html', jinja2_rules, color):
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
failed = True
lint: Fix colors being randomized on every file with custom checks. Now, each custom check language gets a consistent color.
2017-07-28 00:59:53 +02:00
color = next(colors)
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
for fn in by_lang['json']:
lint: Fix colors being randomized on every file with custom checks. Now, each custom check language gets a consistent color.
2017-07-28 00:59:53 +02:00
if custom_check_file(fn, 'json', json_rules, color):
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
failed = True
lint: Fix colors being randomized on every file with custom checks. Now, each custom check language gets a consistent color.
2017-07-28 00:59:53 +02:00
color = next(colors)
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
markdown_docs_length_exclude = {
docs: Clean a few files of too-long lines. We also document why the rest of them are still on the list.
2017-11-16 20:44:00 +01:00
# Has some example Vagrant output that's very long
docs: Reorganize developer docs to improve navigation. This commit helps reduce clutter on the navigation sidebar. Creates new directories and moves relevant files into them. Modifies index.rst, symlinks, and image paths accordingly. This commit also enables expandable/collapsible navigation items, renames files in docs/development and docs/production, modifies /tools/test-documentation so that it overrides a theme setting, Also updates links to other docs, file paths in the codebase that point to developer documents, and files that should be excluded from lint tests. Note that this commit does not update direct links to zulip.readthedocs.io in the codebase; those will be resolved in an upcoming follow-up commit (it'll be easier to verify all the links once this is merged and ReadTheDocs is updated). Fixes #5265.
2017-11-08 17:55:36 +01:00
"docs/development/setup-vagrant.md",
docs: Clean a few files of too-long lines. We also document why the rest of them are still on the list.
2017-11-16 20:44:00 +01:00
# Have wide output in code blocks
docs: Reorganize developer docs to improve navigation. This commit helps reduce clutter on the navigation sidebar. Creates new directories and moves relevant files into them. Modifies index.rst, symlinks, and image paths accordingly. This commit also enables expandable/collapsible navigation items, renames files in docs/development and docs/production, modifies /tools/test-documentation so that it overrides a theme setting, Also updates links to other docs, file paths in the codebase that point to developer documents, and files that should be excluded from lint tests. Note that this commit does not update direct links to zulip.readthedocs.io in the codebase; those will be resolved in an upcoming follow-up commit (it'll be easier to verify all the links once this is merged and ReadTheDocs is updated). Fixes #5265.
2017-11-08 17:55:36 +01:00
"docs/subsystems/logging.md",
docs: Move migration-renumbering.md to subsystems/. It belongs in that part of the documentation. We also update the links in docs/subsystems/schema-migrations.md.
2017-11-18 12:37:29 +01:00
"docs/subsystems/migration-renumbering.md",
docs: Clean a few files of too-long lines. We also document why the rest of them are still on the list.
2017-11-16 20:44:00 +01:00
# Have curl commands with JSON that would be messy to wrap
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
"zerver/webhooks/helloworld/doc.md",
"zerver/webhooks/trello/doc.md",
docs: Clean a few files of too-long lines. We also document why the rest of them are still on the list.
2017-11-16 20:44:00 +01:00
# Has a very long configuration line
integrations/perforce: Migrate docs to Markdown.
2017-06-23 01:04:51 +02:00
"templates/zerver/integrations/perforce.md",
docs: Move integration docs to /api.
2017-11-16 17:36:52 +01:00
# Has some example code that could perhaps be wrapped
"templates/zerver/api/webhook-walkthrough.md",
markdown: Add indented versions of multi-line Markdown macros. This is part of our efforts to change our integrations/webhooks docs to follow the same sort of numbered-list format as our /help docs. In order to indicate that paragraphs separated by newlines are part of the same numbered-list point, every paragraph must be indented 4 spaces.
2017-11-25 00:02:20 +01:00
# This macro has a long indented URL
"templates/zerver/help/include/git-webhook-url-with-branches-indented.md",
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
}
for fn in by_lang['md']:
max_length = None
if fn not in markdown_docs_length_exclude:
max_length = 120
rules = markdown_rules
if fn.startswith("templates/zerver/help"):
rules = help_markdown_rules
lint: Fix colors being randomized on every file with custom checks. Now, each custom check language gets a consistent color.
2017-07-28 00:59:53 +02:00
if custom_check_file(fn, 'md', rules, color, max_length=max_length):
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
failed = True
lint: Fix colors being randomized on every file with custom checks. Now, each custom check language gets a consistent color.
2017-07-28 00:59:53 +02:00
color = next(colors)
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
for fn in by_lang['txt'] + by_lang['text']:
lint: Fix colors being randomized on every file with custom checks. Now, each custom check language gets a consistent color.
2017-07-28 00:59:53 +02:00
if custom_check_file(fn, 'txt', txt_rules, color):
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
failed = True
docs: Lint rst files for trailing whitespace.
2017-11-16 22:20:50 +01:00
color = next(colors)
for fn in by_lang['rst']:
if custom_check_file(fn, 'rst', txt_rules, color):
failed = True
lint: Fix colors being randomized on every file with custom checks. Now, each custom check language gets a consistent color.
2017-07-28 00:59:53 +02:00
color = next(colors)
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
for fn in by_lang['yaml']:
lint: Fix colors being randomized on every file with custom checks. Now, each custom check language gets a consistent color.
2017-07-28 00:59:53 +02:00
if custom_check_file(fn, 'yaml', txt_rules, color):
Extract tools/linter_lib/custom_check.py. As part of extracting this, we exempt the library from all custom checks on itself. This is expedient, since a lot of our custom checks are naive about whether things are in strings, and it is also a pain to configure individual rules.
2017-06-05 16:29:04 +02:00
failed = True
return failed
return (check_custom_checks_py, check_custom_checks_nonpy)