I have looked at maybe ~100 errors in the last week as part
of fixing the tooling, and it's quite common to want to just
see what the improved file would look like. Now I show the
desired output with line numbers.
I also try to encourage devs to scroll up, since newbies
often don't do that for some reason when confronted with
error output.
Finally, I add some color. I try to repeat myself without
color for certain things in case colors on certain
backgrounds are hard to read.
A fast way to test this is to just break up a long tag
into two lines.
Now we only tokenize the file once, and we pass
**validated** tokens to the pretty printer.
There are a few reasons for this:
* It obviously saves a lot of extra computation
just in terms of tokenization.
* It allows our validator to add fields
to the Token objects that help the pretty
printer.
I also removed/tweaked a lot of legacy tests for
pretty_print.py that were exercising bizarrely
formatted HTML that we now simply ban during the
validation phase.
We now create tokens for whitespace and text, such that you
could rebuild the template file with "".join(token.s for
token in tokens).
I also fixed a few bugs related to not parsing
whitespace-control tokens.
We no longer ignore template variables, although we could do
a lot better at validating them.
The most immediate use case for the more thorough parser is
to simplify the pretty printer, but it should also make it
less likely for us to skip over new template constructs
(i.e. the tool will fail hard rather than acting strange).
Note that this speeds up the tool by almost 3x, which may be
slightly surprising considering we are building more tokens.
The reason is that we are now munching efficiently through
big chunks of whitespace and text at a time, rather than
checking each individual character to see if it starts one
of the N other token types.
The changes to the pretty_print module here are a bit ugly,
but they should mostly be made irrelevant in subsequent
commits.
I rewrote most of tools/lib/pretty-printer.py, which
was fairly easy due to being able to crib some
important details from the previous implementation.
The main motivation for the rewrite was that we weren't
handling else/elif blocks correctly, and it was difficult
to modify the previous code. The else/elif shortcomings
were somewhat historical in nature--the original parser
didn't recognize them (since they weren't in any Zulip
templates at the time), and then the pretty printer was
mostly able to hack around that due to the "nudge"
strategy. Eventually the nudge strategy became too
brittle.
The "nudge" strategy was that we would mostly trust
the existing templates, and we would just nudge over
some lines in cases of obviously faulty indentation.
Now we are bit more opinionated and rigorous, and
we basically set the indentation explicitly for any
line that is not in a code/script block. This leads
to this diff touching several templates for mostly
minor fix-ups.
We aren't completely opinionated, as we respect the
author's line wrapping decisions in many cases, and
we also allow authors not to indent blocks within
the template language's block constructs.
Fixes#2665.
Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.
Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start. I expect this change will increase pressure for us to split
those files, which isn't a bad thing.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Automatically generated by the following script, based on the output
of lint with flake8-comma:
import re
import sys
last_filename = None
last_row = None
lines = []
for msg in sys.stdin:
m = re.match(
r"\x1b\[35mflake8 \|\x1b\[0m \x1b\[1;31m(.+):(\d+):(\d+): (\w+)", msg
)
if m:
filename, row_str, col_str, err = m.groups()
row, col = int(row_str), int(col_str)
if filename == last_filename:
assert last_row != row
else:
if last_filename is not None:
with open(last_filename, "w") as f:
f.writelines(lines)
with open(filename) as f:
lines = f.readlines()
last_filename = filename
last_row = row
line = lines[row - 1]
if err in ["C812", "C815"]:
lines[row - 1] = line[: col - 1] + "," + line[col - 1 :]
elif err in ["C819"]:
assert line[col - 2] == ","
lines[row - 1] = line[: col - 2] + line[col - 1 :].lstrip(" ")
if last_filename is not None:
with open(last_filename, "w") as f:
f.writelines(lines)
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Generated by `pyupgrade --py3-plus --keep-percent-format` on all our
Python code except `zthumbor` and `zulip-ec2-configure-interfaces`,
followed by manual indentation fixes.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Previous cleanups (mostly the removals of Python __future__ imports)
were done in a way that introduced leading newlines. Delete leading
newlines from all files, except static/assets/zulip-emoji/NOTICE,
which is a verbatim copy of the Apache 2.0 license.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
`valid_indent_html` allows for replacing the incorrectly
indented file with the correct pretty-printed version
if `--fix` is passed to `tools/lint`.
Fixes#12641.
We fix the issue of check-templates spitting out diff between
expected and found indentation of a file before mentioning the
error message and the file name. Basically stuff was being in the
wrong order despite the fact that in code stuff was happening in the
correct order ie, first print the error message along with the filename
and then the actual diff between expected and found file indentation.
Fixes: #9533.
We make some specific cases of tags use 2 space indents.
The case description:
* A tag with opening tag spread over multiple lines and closing tag
on the same line as of the closing angle bracket of the opening tag.
* A tag with opening tag spread over multiple lines and closing tag
not on the same line as of the closing angle bracket of the opening
tag.
Example:
Case 1:
Not linted:
<button type="button"
class="btn btn-primary btn-small">{{t "Yes" }}</button>
After linting:
<button type="button"
class="btn btn-primary btn-small">{{t "Yes" }}</button>
Case 2:
Before linting:
<div class = "foo"
id = "bar"
role = "whatever">
{{ bla }}
</div>
After linting:
<div class = "foo"
id = "bar"
role = "whatever">
{{ bla }}
</div>
Tweaked by tabbott to not remove it from lister.py, linter_lib, and
friends, since those are intended to support both Python 2 and 3
(we're planning to extract them from the repository).
In this commit we add the ability of recognizing comments in
handlebar and Jinja2 templates to the template parser. This
fixes issue of template parser picking up code tags which are
commented out in files.
Also we fix the problem of too much deep nesting in the Jinja2
logic statement. Now only nested Jinja2 logic statements will get
a deeper indentation level.With this another fix was introduced
relating with the tags before a nested handlebar or django tag getting
wrong indent.
We also fix the issue with wrong offsets with closing tags in
cases if they were not matching indent level of the starting
tag intially.
Also we also start to ignore any tags occuring in between 'pre'
tags for deeper indent levels. As well we start to filter out django
non block tags from getting deeper indent level.
Anders Kaseorg
Tim Abbott
Steve Howell
Anders Kaseorg
Wyatt Hoodes
Rafid Aslam
Anders Kaseorg
Aditya Bansal
rht
Jordan Gedney
Raghav Jajodia