Commit Graph

356 Commits

Author SHA1 Message Date
neiljp (Neil Pilgrim) dac67ebe67 mypy: Remove un-necessary comment re mypy issue #3145.
RuleList is required at global scope now, for later functions.
2017-11-04 19:22:06 -07:00
neiljp (Neil Pilgrim) 05ef052ef5 mypy: Improve typing in custom_check_file() in custom_check.py. 2017-11-04 16:18:27 -07:00
Brock Whittaker e86e259cb1 /api/: Add Markdown-based /api-new documentation.
This commit allows for the /api-new/ page to rendered similarly to our
/help pages.  It's based on the old content for /api, but we're not
replacing the old content yet, to give a bit of time to restructure
things reasonably.

Tweaked by eeshangarg and tabbott.
2017-10-30 22:32:43 -07:00
Greg Price eb55a3a1ba template context: Give better names to the URLs for the API.
The "subdomain" label is redundant, to the extent it's even
accurate -- this is really just the URL we want to display,
which may or may not involve a subdomain.  Similarly "external".

The former `external_api_path_subdomain` was never a path -- it's a
host, followed by a path, which together form a scheme-relative URL.
I'm not quite convinced that value is actually the right thing in
2 of the 3 places we use it, but fixing that can start by giving an
accurate name to the thing we have.
2017-10-30 18:29:29 -07:00
Tim Abbott 8e2cdedf9a lint: Fix lines in Python codebase longer than 120 characters. 2017-10-26 17:47:30 -07:00
Tim Abbott b936e8c24b lint: Fix lines in Python codebase longer than 125 characters. 2017-10-26 17:36:54 -07:00
Tim Abbott be619fe881 lint: Wrap many very long lines in the Python codebase.
This decreases the maximum line length in our Python codebase to 130.
2017-10-26 17:31:58 -07:00
Greg Price 4a6e867046 lint: Disable a buggy `pep8` rule on line breaks and operators.
I'd much rather see something like

    if (thing_is_permissible(user, thing)
            or (user_possesses_hammer(user)
                and glass_break_requested(thing))):

than

    if (thing_is_permissible(user, thing) or
            (user_possesses_hammer(user) and
                glass_break_requested(thing))):

because the former makes the overall logic much easier to scan.
Similarly for a formula full of arithmetic rather than Boolean
operators.  And the actual PEP 8 agrees (though until 2016 it
unfortunately had the opposite advice.)

The upstream linter still applies the backward rule, so disable that.
2017-10-26 10:29:17 -07:00
Tim Abbott fe3a8084d8 mypy: Fix annotations for linter. 2017-10-18 10:50:25 -07:00
Tim Abbott 2bc14d256f lint: Ban two spaces after comma in JS code.
We exclude the frontend tests, mostly because the lint rule isn't that
precise, and the test code has some sample user input that's a bit
funny.
2017-10-18 10:22:18 -07:00
Tim Abbott 8afe9835f6 lint: Ban two spaces after comma in prose. 2017-10-18 10:12:41 -07:00
Tim Abbott b634d9f38b lint: Ban two spaces after comma in CSS. 2017-10-18 10:11:33 -07:00
derAnfaenger 194e102077 linter: Add rule against multiple whitespaces after comma. 2017-10-18 10:04:23 -07:00
rht a603a4f9f5 Remove `from __future__ import absolute_import`.
Except in:
- docs/writing-bots-guide.md, because bots are supposed to be Python 2
  compatible
- puppet/zulip_ops/files/zulip-ec2-configure-interfaces, because this
  script is still on python2.7
- tools/lint
- tools/linter_lib
- tools/lister.py

For the latter two, because they might be yanked away to a separate repo
for general use with other FLOSS projects.
2017-10-17 22:59:42 -07:00
derAnfaenger e1660462e1 linter: Add custom shebang rules section.
Tweaked by tabbott for clarity and to use print_err.
2017-10-12 11:21:38 -07:00
Tim Abbott baee129eda lint: Exclude style= rules for email log.
This is temporary, because this PR was written before these rules were
put in place.
2017-10-11 10:41:40 -07:00
Steve Howell 86d372f9ce lint: Prevent "userid" in code.
We should always use user_id for consistency sake.
2017-10-07 12:16:45 -07:00
Greg Price 9cf44a77da lint: Narrow the space-around-% patterns, remove exclusions.
This lint rule has bitten me a couple of times in working on logging.
These regex rules will inevitably be heuristic, but we can make it a bit more
specific so that the heuristic mainly means it could occasionally miss
something, rather than get in the way with an obviously wrong complaint.
2017-10-06 19:21:40 -07:00
Tim Abbott a9a22a3651 lint: Use new exclude_pattern system to clean up inline style lint. 2017-10-06 13:16:32 -07:00
derAnfaenger 7ff697d053 linter: Add rule against using `pk` instead of `id`.
There's one exception here, for model._meta.pk.  To support this
nicely, we added this exclude-pattern feature.
2017-10-06 12:56:26 -07:00
derAnfaenger 64b8930225 linter: Add rules against improper shebangs. 2017-10-06 12:46:19 -07:00
derAnfaenger c022b7a8fa linter: Add rule against using the `style` attribute.
This has a ton of exclude rules, for two reasons:

(1) We haven't been particularly systematic about avoiding unnecessary
inline style in the past, so there's a lot of code we need to fix.

(2) There are cases where one wants to dynamically compute style
rules. For the latter category, ideally we'd figure out a way to
exclude these automatically (e.g. checking for mustache tags in the
style tag).
2017-10-06 08:33:10 -07:00
derAnfaenger 2cdde8b168 linter: Add rule against verbose $(document).ready() calls. 2017-10-05 10:11:34 -07:00
derAnfaenger 6b99022a02 linter: Add rule against using inline event handlers. 2017-10-05 10:08:40 -07:00
derAnfaenger c01981a65f linter: Extend no-space-before-bracket linting rule. 2017-10-05 10:05:27 -07:00
derAnfaenger 6b879a2b04 docs: Split bots guide into running and writing guides. 2017-10-04 11:49:20 -07:00
derAnfaenger 6c8b405c7e linter: Add rule against using the deprecated Logging.warn.
Tweaked by tabbott to improve the test cases.
2017-10-02 09:31:11 -07:00
Greg Price 0d5f77cf86 logging: Abbreviate log-level names to 4 characters.
These are long enough to still be self-explanatory (the only one I'm
at all in doubt about there is DEBG; I avoided "DBUG" because it reads
"BUG" which suggests a high-priority message, and those are the
opposite of that), while saving a good bit of horizontal space
vs. padding everything to the 8 characters of "CRITICAL".

Also add a linter exception to allow easy-to-read alignment here,
similar to several existing exceptions for other alignment cases.
2017-09-28 18:26:39 -07:00
Greg Price 0163920577 logging: Make our own Formatter class.
This doesn't yet do much, but it gives us a suitable place to
add code to customize how log messages are displayed, beyond what
a format string passed to the default formatter can do.
2017-09-28 18:26:39 -07:00
Robert Hönig f6f601f22b linter: Add tests for first 10 python custom rules. 2017-09-27 18:27:43 -07:00
Robert Hönig fa8199819c linter: Remove unnecessary escape sequences in custom rules. 2017-09-27 09:02:44 -07:00
Robert Hönig 48696ac9a1 linter: Catch all malformatted string formatting clauses.
This also catches code like `"foo%s"%('bar')`, whereas before,
the first tuple entry could not be a string.
2017-09-27 09:02:44 -07:00
Robert Hönig 41e6704b99 custom linter rules test: Add pattern test.
The pattern test method `test_rule_patterns` tests each rule by
fetching two strings from it: `test_good` and `test_bad`. Each
string is then presented as an input file to `custom_check_file`,
which should return True or False.
All lines in a string need to end with `\n`. Since the linter
expects an additional newline at the end of a file, the test case
adds `\n` to each string on top of that.

Fixes #6320.
2017-09-25 10:53:06 +02:00
Juliana Bacelar 928dd06cc8 linter: Add lint rule banning 'import os.path' 2017-09-22 10:32:21 -07:00
julianasobreira df2d448f7b python: Add lint rule banning 'from os.path import'.
This enforces our use of a consistent style in how we access Python
modules; "from os.path import dirname" is a particularly popular
abbreviation inconsistent with our style, and so it deserves a lint
rule.

Commit message and error text tweaked by tabbott.

Fixes #6543.
2017-09-22 04:55:38 -07:00
Steve Howell 6ce68dfc3d minor: Remove obsolete lint check.
Some code that we had a lint exception for got cleaned up in a
recent commit.
2017-09-20 13:05:04 -07:00
Steve Howell f13cb94d84 Have exclude_topic_mutes() accept a stream id.
We want to convert stream names to stream ids as close
to the "edges" of our system as possible, so we let our
caller do the work of finding the stream id for a stream
narrow.
2017-09-20 10:31:33 -07:00
Supermanu 184bfe7cfd docs: Expand the french translation style guide.
As the french word organisation is used, the related line is excluded
from the linter spell check.
2017-09-14 08:35:51 -07:00
Umair Khan df70fa962e linter: Improve i18n related regexes.
There are four regexes which try to ensure that the i18n strings are
properly captured.

1) The one which disallows multiline strings.
```
i18n\.t\([^)]+[^,\{\)]$

// Disallows:
i18n.t('some '
       + 'text');
```

2) The one which disallows concatenation within argument to i18n.t():
```
i18n\.t\([\'\"].+?[\'\"]\s*\+

// Disallows:
i18n.t("some " + "text");
```

3) There are two which disallow concatenation with i18n.t():
```
i18n\.t\(.+\).*\+

// Disallows:
i18n.t('some text') +

\+.*i18n\.t\(.+\)

// Disallows:
+ i18n.t('some text')
```

The ideal case is that you try to bring the string argument to the
i18n.t() on one line. In case this is not possible, you can do the
following:

```
var1 = i18n.t("Some text to be translated");
var2 = i18n.t("Some more text to be translated");
complete = var1 + var2;
2017-09-13 12:58:27 -07:00
Robert Hönig b5cf31efdc linter: Move custom_check_file to global namespace.
This is needed in order to mock the method when testing
`custom_check.py`. The diff for this commit is a bit broken;
all it really does is moving the method out of `build_custom_checkers`.
2017-09-12 03:24:59 -07:00
Robert Hönig 7a2234dfa2 linter: Remove redundant custom rule file delimitors. 2017-09-12 03:24:59 -07:00
Robert Hönig 2248c7ac0d linter: Add explanation for custom rule file delimitation operators. 2017-09-12 03:24:59 -07:00
Steve Howell 4dfe6bb320 Add migration to fix unread messages. 2017-09-07 07:06:03 -07:00
Robert Hönig 34e3329919 linter: Add rule against misspelled "Terms of Service". 2017-09-07 06:29:33 -07:00
Umair Khan ee6235d716 lint: Add regex to disallow i18n string concatenation.
Fixes #6398
2017-09-06 07:01:43 -07:00
Tim Abbott 6de77db93f lint: Fix problematic merging of pyflakes stdout/stderr.
We were getting pyflakes lint error output without line numbers like
this:

pyflakes  |     if user_profile.is_realm_admin and
pyflakes  |                                       ^
pyflakes  |

Apparently the cause was that stdout and stderr was getting mixed
badly, creating "unused import"s lines that had the first of that
error (containing the line number) just above.

As a result, printing out the lines of output from pyflakes' merged
stdout/stderr feed looked like this:

b"zproject/settings.py:95: 'from .prod_settings import *' used; unable to detect undefined nameszerver/views/users.py:49:39: invalid syntax\n"

Note the lack of newline in between the end of the first error at
"names" and the start of the second at "zerver".

This appears to be a change in Pyflakes behavior when we switched to
Python 3; probably they're missing a flush() somewhere.
2017-08-29 10:44:50 -07:00
Tim Abbott 130b5847ad pyflakes: Remove now-unnecessary exclude lines. 2017-08-29 10:27:29 -07:00
Yago González 7d5d8ac61e lint: Add rule for spacing inside Handlebars translation tags.
This commit also includes the fixes to make all Handlebars templates
compliant with the new rule.
2017-08-23 13:08:57 -07:00
Yago González 3a4c6f83fc lint: Ban trailing spaces in Handlebars translation tags. 2017-08-23 13:08:57 -07:00
Yago González aa5f57fda9 lint: Ban leading spaces in Handlebars translation tags. 2017-08-23 13:08:57 -07:00
Rishi Gupta b32fe9f2a0 features: Update screenshot and add tool to generate.
This adds a new management command that should make generating or
updating screenshots a bit easier in the future.
2017-08-22 11:32:22 -07:00
Greg Price ceccd0c0f6 lint: Accept mentions of specific Python versions.
We're about to do this (a) in a number of places mentioning
system packages like `python3-dev` to install, and (b) in the
shebangs of every script in the tree.
2017-08-15 17:30:31 -07:00
Tim Abbott ffc1ceeaa4 lint: Ban use of get_stream in the rest of messages.py.
This will help avoid potential future security bugs.
2017-08-15 10:40:02 -07:00
Robert Hönig c5f0478374 linter: Add trailing whitespace check for json files. 2017-08-15 09:28:11 -07:00
Jack Zhang c546016964 portico: Add translations bundle to portico pages. 2017-08-10 10:21:53 -07:00
neiljp (Neil Pilgrim) b7103752dc mypy: Add assertions relating to Popen.stdout with subprocess.PIPE use. 2017-08-07 21:27:50 -07:00
neiljp (Neil Pilgrim) 6ac6d6ff5d mypy: Add note re mypy supporting only global type aliases.
https://github.com/python/mypy/issues/3145
2017-08-05 17:53:18 -07:00
neiljp (Neil Pilgrim) 353f6fdb88 mypy: Hoist RuleList type alias out of function. 2017-08-05 12:08:59 -07:00
Rishi Gupta 3e9a85ecb2 onboarding: Add turtle reaction to an initial message. 2017-08-01 22:51:05 -07:00
Greg Price 5784724d00 api: Update lint and analysis code to forget about now-gone files.
These files don't exist in this repository anymore, so there's
no need to mention them in exclude lists for lint and mypy.
2017-07-31 21:24:52 -07:00
Tim Abbott 53e4d8562b lint: Add a lint check for bare `type: ignore`s. 2017-07-27 16:31:55 -07:00
Tim Abbott d7766d7b6e lint: Fix colors being randomized on every file with custom checks.
Now, each custom check language gets a consistent color.
2017-07-27 16:03:12 -07:00
Jack Zhang 8e2ec1cddc portico: Update registration form placeholders. 2017-07-21 13:09:06 -07:00
rht af430fe24d tools/linter_lib: Clean unused imports told by F401 pyflakes.
The '  # NOQA' comment is added so that the unused imports check (F401)
can still be run while still having the Mypy type check.
2017-07-20 16:46:07 -07:00
Tim Abbott 5cdfd61f2b lint: Require translation tags on aria-label tags.
Since these are, by their nature, directly user-facing strings, they
should always be tagged for translation.
2017-07-14 14:48:17 -07:00
Aditya Bansal de1dc50204 linter/pep8: Remove code infra to ignore the E261 violations. 2017-07-11 15:17:32 -07:00
Aditya Bansal 84e53f960d pep8: Add compliance with rule E261 to tests/test_upload.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 1b5e3190e3 pep8: Add compliance with rule E261 to tests/test_subs.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 2643a40176 pep8: Add compliance with rule E261 to tests/test_signup.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 59b26e582a pep8: Add compliance with rule E261 to tests/test_realm.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 0970d29ec1 pep8: Add compliance with rule E261 to tests/test_narrow.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal ce70ded488 pep8: Add compliance with rule E261 to tests/test_messages.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 8c233198f8 pep8: Add compliance with rule E261 to tests/test_events.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal ee97ea1356 pep8: Add compliance with rule E261 to tests/test_bugdown.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 0c4165a5bd pep8: Add compliance with rule E261 to populate_db.py. 2017-07-11 11:56:50 -07:00
Aditya Bansal 856e005d64 linter_lib/pep8.py: Remove files which don't have E261 conflicts. 2017-07-11 11:55:30 -07:00
Aditya Bansal d46cf59b0d pep8: Add compliance with rule E261 to worker/queue_processors.py. 2017-07-11 11:55:02 -07:00
Aditya Bansal 1e3b57570e pep8: Add compliance with rule E261 to tornado/websocket_client.py. 2017-07-11 11:55:02 -07:00
Aditya Bansal 94b790c28d pep8: Add compliance with rule E261 to tornado/socket.py. 2017-07-11 11:55:02 -07:00
Aditya Bansal b9bffaa446 pep8: Add compliance with rule E261 to prod_settings_template.py. 2017-07-11 11:55:01 -07:00
Aditya Bansal 5f87a0f8f8 pep8: Add compliance with rule E261 zproject/settings.py. 2017-07-11 11:53:33 -07:00
Aditya Bansal 7c6ad4a770 pep8: Add compliance with rule E261 zproject/dev_settings.py. 2017-07-11 11:53:33 -07:00
Aditya Bansal 44edab1cff pep8: Add compliance with rule E261 tools/run-dev.py. 2017-07-11 11:53:33 -07:00
Aditya Bansal 73ad834129 pep8: Add compliance with rule E261 bugdown/__init__.py. 2017-07-11 11:53:33 -07:00
Aditya Bansal f5d5c48831 pep8: Add compliance with rule E261 models.py. 2017-07-11 11:53:33 -07:00
Tommy Ip 2dc35e6c11 linter: Update custom linters' visual output. 2017-07-05 23:31:09 -07:00
Tommy Ip d188c54ebc linter: Update pyflakes visual output. 2017-07-05 23:31:09 -07:00
Tommy Ip cff2b1ba9e linter: Update pep8 visual output. 2017-07-05 23:31:09 -07:00
Tommy Ip e0d4a007e8 linter: Add `identifier` argument to `custom_check_file` function.
In preparation for the upcoming linter output update, each custom
linter now has a identifier argument. This would be used for separating
each line of lints, eg:

    py        | Missing space around "%" at analytics/views.py line 485:
    py        |         output += '<hr>%s\n'%(string_id,)
    css       | 64c64,65
    css       | <     opacity: 1; }
    css       | ---
    css       | >     opacity: 1;
    css       | > }
    css       | static/styles/lightbox.css seems to be broken:
    swagger   | In static/swagger/zulip.yaml:
    swagger   | Duplicate operationId: registerQueue
2017-07-05 23:23:45 -07:00
Tommy Ip 3b8864bafa linter: Create error printing library.
For performance reasons, we spawn each linter in a separate OS thread.
The downside of this is that all lints would end up in stdout without
much visual separation, resulting in confusing error log. This commit
introduce the `print_err` function, which shows which linter each line
of lint is from.
2017-07-06 13:46:10 +08:00
Umair Khan f60d4849b7 Revert "lint: Don't allow handlerbars within translatable text."
This reverts commit 3c7b9cdc94.
2017-06-30 10:12:31 -04:00
Umair Khan 3c7b9cdc94 lint: Don't allow handlerbars within translatable text.
Fixes: #5544
2017-06-29 07:39:20 -04:00
Eeshan Garg 4ad951f256 integrations/perforce: Migrate docs to Markdown. 2017-06-26 12:30:04 -04:00
Robert Hönig d45d9c9ab5 bots: Add linter rule to prevent open() calls.
Addresses #5395.
2017-06-21 10:16:59 -04:00
Robert Hönig fd41a481e4 linter: Fix `include_only` rule path comparison.
In order for the `include_only` linter rule to not have
any side effects, we need to explicitly add a trailing '/'
after every directory we want to include.
2017-06-21 10:16:59 -04:00
Steve Howell e49afe3ebc lint: Add code to suggest removing exclusions.
If excluded lines no longer exist in a file, print a message
asking the user to remove the exclusion.
2017-06-21 08:12:58 -04:00
Steve Howell 572ea370ea Speed up custom checkers in `./tools/lint`.
The main thing here is to make looping over lines be the inner
loop, instead of looping over rules.  This keeps regexes in
cache, and it also avoids some O(N) checks.

This is a significant speedup for me, reducing time from 16s
to 11s.
2017-06-20 22:26:12 -04:00
vaibhav 33c0c00cd6 Outgoing webhook System: first Iteration of outgoing webhook UI. 2017-06-19 16:05:54 -04:00
Eeshan Garg bbe7ef6c5e linter_lib: Exclude webhook URLs from long line checks.
Now, URLs starting with `{{ external_api_uri_subdomain }}` are
exempted from the 120 char per line requirement.
2017-06-14 17:42:17 -07:00
Aditya Bansal ae7781dd76 Fix bug in pep8 linter.
Basically pep8 linter was not running on a selected ignore list of
files for which an extra pep8 rule e261 was to be ignored.
2017-06-07 16:00:01 -06:00
Aditya Bansal 24b1efc74e Fix pep8 linter bug.
In this commit we fix the bug where pep8 linter does not report
false and exit when we got linting issues.
2017-06-07 14:05:03 -07:00
Eeshan Garg fab20ec6fd templates: Add macro for appending branch names to Git webhook URLs. 2017-06-05 11:20:48 -07:00
Steve Howell a606628b16 Extract tools/linter_lib/pep8.py 2017-06-05 09:20:21 -07:00
Steve Howell 9fab41aebb Extract tools/linter_lib/pyflakes.py 2017-06-05 09:20:21 -07:00
Steve Howell 29affda956 Extract tools/linter_lib/exclude.py 2017-06-05 09:20:21 -07:00
Steve Howell 40a2ffad98 Extract tools/linter_lib/custom_check.py.
As part of extracting this, we exempt the library from all custom
checks on itself.  This is expedient, since a lot of our
custom checks are naive about whether things are in strings, and
it is also a pain to configure individual rules.
2017-06-05 09:20:21 -07:00