Mirror
/
zulip
mirror of https://github.com/zulip/zulip.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
zulip/tools/tests/test_html_branches.py

161 lines
5.5 KiB
Python
Raw Normal View History

Lint for duplicate ids in templates. In this commit we enhance our current template linter to detect duplicate ids and report them during lint checks. html_branches.py was topped up with a new function build_id_dict for the purpose. Also the get_tag_info function in same file was updated to parse ids and classes more robustly in cases of template variables. split_for_id_and_class function was added to serve this purpose. Unit tests for both the functions were created under tests/test_html_branches. Also a directory under tests called test_template_data was created to hold templates for testing under newly created functionality. check_templates was modified to print to console any duplicates detected. showell reviewed my commit and helped me out. Fixes #2950.
2017-01-06 15:11:15 +01:00
import os
python: Sort imports with isort. Fixes #2665. Regenerated by tabbott with `lint --fix` after a rebase and change in parameters. Note from tabbott: In a few cases, this converts technical debt in the form of unsorted imports into different technical debt in the form of our largest files having very long, ugly import sequences at the start. I expect this change will increase pressure for us to split those files, which isn't a bad thing. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 00:54:34 +02:00
import unittest
Factor out HtmlTreeBranch and related code from template parser. This code is not directly related to the template parser, so it can safely live in its own file. The only significant change to the code is to the signature of `html_branches` so that it can be called without requiring a file. Since it's only used in html_grep, that has been updated to reflect this change. Fixes: #1774.
2016-09-11 20:23:29 +02:00
import tools.lib.template_parser
from tools.lib.html_branches import (
templates: Mark all void tags as self-closing. This reverses the policy that was set, but incompletely enforced, by commit 951514dd7dd2fafb67e9e9c6930d31b9295741c0. The self-closing tag syntax is clearer, more consistent, simpler to parse, compatible with XML, preferred by Prettier, and (most importantly now) required by FormatJS. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-21 00:46:14 +02:00
Node,
python: Sort imports with isort. Fixes #2665. Regenerated by tabbott with `lint --fix` after a rebase and change in parameters. Note from tabbott: In a few cases, this converts technical debt in the form of unsorted imports into different technical debt in the form of our largest files having very long, ugly import sequences at the start. I expect this change will increase pressure for us to split those files, which isn't a bad thing. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 00:54:34 +02:00
build_id_dict,
Factor out HtmlTreeBranch and related code from template parser. This code is not directly related to the template parser, so it can safely live in its own file. The only significant change to the code is to the signature of `html_branches` so that it can be called without requiring a file. Since it's only used in html_grep, that has been updated to reflect this change. Fixes: #1774.
2016-09-11 20:23:29 +02:00
get_tag_info,
html_branches,
html_tag_tree,
Lint for duplicate ids in templates. In this commit we enhance our current template linter to detect duplicate ids and report them during lint checks. html_branches.py was topped up with a new function build_id_dict for the purpose. Also the get_tag_info function in same file was updated to parse ids and classes more robustly in cases of template variables. split_for_id_and_class function was added to serve this purpose. Unit tests for both the functions were created under tests/test_html_branches. Also a directory under tests called test_template_data was created to hold templates for testing under newly created functionality. check_templates was modified to print to console any duplicates detected. showell reviewed my commit and helped me out. Fixes #2950.
2017-01-06 15:11:15 +01:00
split_for_id_and_class,
Factor out HtmlTreeBranch and related code from template parser. This code is not directly related to the template parser, so it can safely live in its own file. The only significant change to the code is to the signature of `html_branches` so that it can be called without requiring a file. Since it's only used in html_grep, that has been updated to reflect this change. Fixes: #1774.
2016-09-11 20:23:29 +02:00
)
tools: Fix regressions in tools/tests/test_html_branches.py
2017-01-18 22:04:32 +01:00
ZULIP_PATH = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
TEST_TEMPLATES_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "test_template_data")
Factor out HtmlTreeBranch and related code from template parser. This code is not directly related to the template parser, so it can safely live in its own file. The only significant change to the code is to the signature of `html_branches` so that it can be called without requiring a file. Since it's only used in html_grep, that has been updated to reflect this change. Fixes: #1774.
2016-09-11 20:23:29 +02:00
python: Reformat with Black, except quotes. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:19:30 +01:00
class TestHtmlBranches(unittest.TestCase):
tools/tests: Use Python 3 syntax for typing.
2017-12-13 06:45:46 +01:00
def test_get_tag_info(self) -> None:
check-templates: Make parser more thorough (and faster). We now create tokens for whitespace and text, such that you could rebuild the template file with "".join(token.s for token in tokens). I also fixed a few bugs related to not parsing whitespace-control tokens. We no longer ignore template variables, although we could do a lot better at validating them. The most immediate use case for the more thorough parser is to simplify the pretty printer, but it should also make it less likely for us to skip over new template constructs (i.e. the tool will fail hard rather than acting strange). Note that this speeds up the tool by almost 3x, which may be slightly surprising considering we are building more tokens. The reason is that we are now munching efficiently through big chunks of whitespace and text at a time, rather than checking each individual character to see if it starts one of the N other token types. The changes to the pretty_print module here are a bit ugly, but they should mostly be made irrelevant in subsequent commits.
2021-12-01 18:03:31 +01:00
html = """<p id="test" class="test1 test2">foo</p>"""
Factor out HtmlTreeBranch and related code from template parser. This code is not directly related to the template parser, so it can safely live in its own file. The only significant change to the code is to the signature of `html_branches` so that it can be called without requiring a file. Since it's only used in html_grep, that has been updated to reflect this change. Fixes: #1774.
2016-09-11 20:23:29 +02:00
check-templates: Make parser more thorough (and faster). We now create tokens for whitespace and text, such that you could rebuild the template file with "".join(token.s for token in tokens). I also fixed a few bugs related to not parsing whitespace-control tokens. We no longer ignore template variables, although we could do a lot better at validating them. The most immediate use case for the more thorough parser is to simplify the pretty printer, but it should also make it less likely for us to skip over new template constructs (i.e. the tool will fail hard rather than acting strange). Note that this speeds up the tool by almost 3x, which may be slightly surprising considering we are building more tokens. The reason is that we are now munching efficiently through big chunks of whitespace and text at a time, rather than checking each individual character to see if it starts one of the N other token types. The changes to the pretty_print module here are a bit ugly, but they should mostly be made irrelevant in subsequent commits.
2021-12-01 18:03:31 +01:00
start_tag, text, end_tag = tools.lib.template_parser.tokenize(html)
Factor out HtmlTreeBranch and related code from template parser. This code is not directly related to the template parser, so it can safely live in its own file. The only significant change to the code is to the signature of `html_branches` so that it can be called without requiring a file. Since it's only used in html_grep, that has been updated to reflect this change. Fixes: #1774.
2016-09-11 20:23:29 +02:00
start_tag_info = get_tag_info(start_tag)
end_tag_info = get_tag_info(end_tag)
python: Normalize quotes with Black. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:20:45 +01:00
self.assertEqual(start_tag_info.text(), "p.test1.test2#test")
self.assertEqual(end_tag_info.text(), "p")
check-templates: Make parser more thorough (and faster). We now create tokens for whitespace and text, such that you could rebuild the template file with "".join(token.s for token in tokens). I also fixed a few bugs related to not parsing whitespace-control tokens. We no longer ignore template variables, although we could do a lot better at validating them. The most immediate use case for the more thorough parser is to simplify the pretty printer, but it should also make it less likely for us to skip over new template constructs (i.e. the tool will fail hard rather than acting strange). Note that this speeds up the tool by almost 3x, which may be slightly surprising considering we are building more tokens. The reason is that we are now munching efficiently through big chunks of whitespace and text at a time, rather than checking each individual character to see if it starts one of the N other token types. The changes to the pretty_print module here are a bit ugly, but they should mostly be made irrelevant in subsequent commits.
2021-12-01 18:03:31 +01:00
self.assertEqual(text.s, "foo")
Factor out HtmlTreeBranch and related code from template parser. This code is not directly related to the template parser, so it can safely live in its own file. The only significant change to the code is to the signature of `html_branches` so that it can be called without requiring a file. Since it's only used in html_grep, that has been updated to reflect this change. Fixes: #1774.
2016-09-11 20:23:29 +02:00
tools/tests: Use Python 3 syntax for typing.
2017-12-13 06:45:46 +01:00
def test_html_tag_tree(self) -> None:
Factor out HtmlTreeBranch and related code from template parser. This code is not directly related to the template parser, so it can safely live in its own file. The only significant change to the code is to the signature of `html_branches` so that it can be called without requiring a file. Since it's only used in html_grep, that has been updated to reflect this change. Fixes: #1774.
2016-09-11 20:23:29 +02:00
html = """
<!-- test -->
<!DOCTYPE html>
<html>
<!-- test -->
<head>
<title>Test</title>
<meta charset="utf-8" />
<link rel="stylesheet" href="style.css" />
</head>
<body>
check-templates: Be stricter about singleton tags. We now forbid tags of the form `<foo ... />` in most places, and we also forbid it even for several void tags. We make exceptions for tags that are already formatted in two different ways in our codebase. This is mostly svg tags, plus these common cases: - br - hr - img - input It would be nice to lock down a convention for these, even though the HTML specification is unopinionated on these. We'll probably want to stay flexible for svg tags, since they are sometimes copy/pasted from other sources (although it's probably rare enough for them that we can tolerate just doing minor edits as needed).
2020-04-18 11:38:31 +02:00
<p>Hello<br>world!</p>
Factor out HtmlTreeBranch and related code from template parser. This code is not directly related to the template parser, so it can safely live in its own file. The only significant change to the code is to the signature of `html_branches` so that it can be called without requiring a file. Since it's only used in html_grep, that has been updated to reflect this change. Fixes: #1774.
2016-09-11 20:23:29 +02:00
<p>Goodbye<!-- test -->world!</p>
</body>
</html>
<!-- test -->
"""
tree = html_tag_tree(html)
templates: Mark all void tags as self-closing. This reverses the policy that was set, but incompletely enforced, by commit 951514dd7dd2fafb67e9e9c6930d31b9295741c0. The self-closing tag syntax is clearer, more consistent, simpler to parse, compatible with XML, preferred by Prettier, and (most importantly now) required by FormatJS. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-21 00:46:14 +02:00
def serialize(node: Node) -> object:
return (
node.token and (node.token.kind, node.token.tag),
[serialize(child) for child in node.children],
)
Factor out HtmlTreeBranch and related code from template parser. This code is not directly related to the template parser, so it can safely live in its own file. The only significant change to the code is to the signature of `html_branches` so that it can be called without requiring a file. Since it's only used in html_grep, that has been updated to reflect this change. Fixes: #1774.
2016-09-11 20:23:29 +02:00
templates: Mark all void tags as self-closing. This reverses the policy that was set, but incompletely enforced, by commit 951514dd7dd2fafb67e9e9c6930d31b9295741c0. The self-closing tag syntax is clearer, more consistent, simpler to parse, compatible with XML, preferred by Prettier, and (most importantly now) required by FormatJS. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-21 00:46:14 +02:00
expected = (
None,
[
(
("html_start", "html"),
[
(
("html_start", "head"),
[
(("html_start", "title"), []),
(("html_singleton", "meta"), []),
(("html_singleton", "link"), []),
],
),
(
("html_start", "body"),
[
(
("html_start", "p"),
[(("html_start", "br"), []), (("html_start", "p"), [])],
)
],
),
],
)
],
python: Reformat with Black, except quotes. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:19:30 +01:00
)
templates: Mark all void tags as self-closing. This reverses the policy that was set, but incompletely enforced, by commit 951514dd7dd2fafb67e9e9c6930d31b9295741c0. The self-closing tag syntax is clearer, more consistent, simpler to parse, compatible with XML, preferred by Prettier, and (most importantly now) required by FormatJS. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-21 00:46:14 +02:00
self.assertEqual(serialize(tree), expected)
Factor out HtmlTreeBranch and related code from template parser. This code is not directly related to the template parser, so it can safely live in its own file. The only significant change to the code is to the signature of `html_branches` so that it can be called without requiring a file. Since it's only used in html_grep, that has been updated to reflect this change. Fixes: #1774.
2016-09-11 20:23:29 +02:00
tools/tests: Use Python 3 syntax for typing.
2017-12-13 06:45:46 +01:00
def test_html_branches(self) -> None:
Factor out HtmlTreeBranch and related code from template parser. This code is not directly related to the template parser, so it can safely live in its own file. The only significant change to the code is to the signature of `html_branches` so that it can be called without requiring a file. Since it's only used in html_grep, that has been updated to reflect this change. Fixes: #1774.
2016-09-11 20:23:29 +02:00
html = """
<!-- test -->
<!DOCTYPE html>
<html>
<!-- test -->
<head>
<title>Test</title>
<meta charset="utf-8" />
<link rel="stylesheet" href="style.css" />
</head>
<body>
check-templates: Be stricter about singleton tags. We now forbid tags of the form `<foo ... />` in most places, and we also forbid it even for several void tags. We make exceptions for tags that are already formatted in two different ways in our codebase. This is mostly svg tags, plus these common cases: - br - hr - img - input It would be nice to lock down a convention for these, even though the HTML specification is unopinionated on these. We'll probably want to stay flexible for svg tags, since they are sometimes copy/pasted from other sources (although it's probably rare enough for them that we can tolerate just doing minor edits as needed).
2020-04-18 11:38:31 +02:00
<p>Hello<br>world!</p>
Factor out HtmlTreeBranch and related code from template parser. This code is not directly related to the template parser, so it can safely live in its own file. The only significant change to the code is to the signature of `html_branches` so that it can be called without requiring a file. Since it's only used in html_grep, that has been updated to reflect this change. Fixes: #1774.
2016-09-11 20:23:29 +02:00
<p>Goodbye<!-- test -->world!</p>
</body>
</html>
<!-- test -->
"""
branches = html_branches(html)
python: Reformat with Black, except quotes. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:19:30 +01:00
self.assertEqual(
templates: Mark all void tags as self-closing. This reverses the policy that was set, but incompletely enforced, by commit 951514dd7dd2fafb67e9e9c6930d31b9295741c0. The self-closing tag syntax is clearer, more consistent, simpler to parse, compatible with XML, preferred by Prettier, and (most importantly now) required by FormatJS. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-21 00:46:14 +02:00
[(branch.text(), branch.staircase_text()) for branch in branches],
[
("html head title", "\n html\n head\n title\n"),
("html head meta", "\n html\n head\n meta\n"),
("html head link", "\n html\n head\n link\n"),
("html body p br", "\n html\n body\n p\n br\n"),
("html body p p", "\n html\n body\n p\n p\n"),
],
python: Reformat with Black, except quotes. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:19:30 +01:00
)
Lint for duplicate ids in templates. In this commit we enhance our current template linter to detect duplicate ids and report them during lint checks. html_branches.py was topped up with a new function build_id_dict for the purpose. Also the get_tag_info function in same file was updated to parse ids and classes more robustly in cases of template variables. split_for_id_and_class function was added to serve this purpose. Unit tests for both the functions were created under tests/test_html_branches. Also a directory under tests called test_template_data was created to hold templates for testing under newly created functionality. check_templates was modified to print to console any duplicates detected. showell reviewed my commit and helped me out. Fixes #2950.
2017-01-06 15:11:15 +01:00
tools/tests: Use Python 3 syntax for typing.
2017-12-13 06:45:46 +01:00
def test_build_id_dict(self) -> None:
Lint for duplicate ids in templates. In this commit we enhance our current template linter to detect duplicate ids and report them during lint checks. html_branches.py was topped up with a new function build_id_dict for the purpose. Also the get_tag_info function in same file was updated to parse ids and classes more robustly in cases of template variables. split_for_id_and_class function was added to serve this purpose. Unit tests for both the functions were created under tests/test_html_branches. Also a directory under tests called test_template_data was created to hold templates for testing under newly created functionality. check_templates was modified to print to console any duplicates detected. showell reviewed my commit and helped me out. Fixes #2950.
2017-01-06 15:11:15 +01:00
templates = ["test_template1.html", "test_template2.html"]
templates = [os.path.join(TEST_TEMPLATES_DIR, fn) for fn in templates]
template_id_dict = build_id_dict(templates)
python: Reformat with Black, except quotes. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:19:30 +01:00
self.assertEqual(
python: Normalize quotes with Black. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:20:45 +01:00
set(template_id_dict.keys()), {"below_navbar", "hello_{{ message }}", "intro"}
python: Reformat with Black, except quotes. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:19:30 +01:00
)
self.assertEqual(
python: Normalize quotes with Black. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:20:45 +01:00
template_id_dict["hello_{{ message }}"],
python: Reformat with Black, except quotes. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:19:30 +01:00
[
python: Normalize quotes with Black. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:20:45 +01:00
f"Line 12:{ZULIP_PATH}/tools/tests/test_template_data/test_template1.html",
f"Line 12:{ZULIP_PATH}/tools/tests/test_template_data/test_template2.html",
python: Reformat with Black, except quotes. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:19:30 +01:00
],
)
self.assertEqual(
python: Normalize quotes with Black. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:20:45 +01:00
template_id_dict["intro"],
python: Reformat with Black, except quotes. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:19:30 +01:00
[
python: Normalize quotes with Black. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:20:45 +01:00
f"Line 10:{ZULIP_PATH}/tools/tests/test_template_data/test_template1.html",
f"Line 11:{ZULIP_PATH}/tools/tests/test_template_data/test_template1.html",
f"Line 11:{ZULIP_PATH}/tools/tests/test_template_data/test_template2.html",
python: Reformat with Black, except quotes. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:19:30 +01:00
],
)
self.assertEqual(
python: Normalize quotes with Black. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:20:45 +01:00
template_id_dict["below_navbar"],
[f"Line 10:{ZULIP_PATH}/tools/tests/test_template_data/test_template2.html"],
python: Reformat with Black, except quotes. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:19:30 +01:00
)
Lint for duplicate ids in templates. In this commit we enhance our current template linter to detect duplicate ids and report them during lint checks. html_branches.py was topped up with a new function build_id_dict for the purpose. Also the get_tag_info function in same file was updated to parse ids and classes more robustly in cases of template variables. split_for_id_and_class function was added to serve this purpose. Unit tests for both the functions were created under tests/test_html_branches. Also a directory under tests called test_template_data was created to hold templates for testing under newly created functionality. check_templates was modified to print to console any duplicates detected. showell reviewed my commit and helped me out. Fixes #2950.
2017-01-06 15:11:15 +01:00
tools/tests: Use Python 3 syntax for typing.
2017-12-13 06:45:46 +01:00
def test_split_for_id_and_class(self) -> None:
Lint for duplicate ids in templates. In this commit we enhance our current template linter to detect duplicate ids and report them during lint checks. html_branches.py was topped up with a new function build_id_dict for the purpose. Also the get_tag_info function in same file was updated to parse ids and classes more robustly in cases of template variables. split_for_id_and_class function was added to serve this purpose. Unit tests for both the functions were created under tests/test_html_branches. Also a directory under tests called test_template_data was created to hold templates for testing under newly created functionality. check_templates was modified to print to console any duplicates detected. showell reviewed my commit and helped me out. Fixes #2950.
2017-01-06 15:11:15 +01:00
id1 = "{{ red|blue }}"
id2 = "search_box_{{ page }}"
class1 = "chat_box message"
class2 = "stream_{{ topic }}"
class3 = "foo {{ a|b|c }} bar"
python: Normalize quotes with Black. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:20:45 +01:00
self.assertEqual(split_for_id_and_class(id1), ["{{ red|blue }}"])
self.assertEqual(split_for_id_and_class(id2), ["search_box_{{ page }}"])
Lint for duplicate ids in templates. In this commit we enhance our current template linter to detect duplicate ids and report them during lint checks. html_branches.py was topped up with a new function build_id_dict for the purpose. Also the get_tag_info function in same file was updated to parse ids and classes more robustly in cases of template variables. split_for_id_and_class function was added to serve this purpose. Unit tests for both the functions were created under tests/test_html_branches. Also a directory under tests called test_template_data was created to hold templates for testing under newly created functionality. check_templates was modified to print to console any duplicates detected. showell reviewed my commit and helped me out. Fixes #2950.
2017-01-06 15:11:15 +01:00
python: Normalize quotes with Black. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:20:45 +01:00
self.assertEqual(split_for_id_and_class(class1), ["chat_box", "message"])
self.assertEqual(split_for_id_and_class(class2), ["stream_{{ topic }}"])
self.assertEqual(split_for_id_and_class(class3), ["foo", "{{ a|b|c }}", "bar"])