User-supplied custom realm filter has had some sort of regex-based
validation of the format URL since their introduction in
d7e1e4a2c0 -- and this has always been
in addition to the URLValidator. The URLValidator is the one which
does the security-relevant work of validating that the schema is
reasonable, and that the overall shape of the URL is well-formed. The
regex has served primarily to arbitrary limit the characters that can
appear in the URL, in the mistaken name of safety.
Adjust the regex, such that its only purpose is to verify that the
usages of `%` characters in the URL are reasonable, and leave the URL
validation to the URLValidator, which can do a far better job. This
includes broadening the support to include `%%` as an escape
character; this is likely such a niche case as to be unnecessary, but
costs little.
Fixes#16013.
Zulip attempts to validate that the regular expressions that admins
enter for linkifiers are well-formatted, and only contain a specific
subset of regex grammar. The process of checking these
properties (via a regex!) can cause denial-of-service via
backtracking.
Furthermore, this validation itself does not prevent the creation of
linkifiers which themselves cause denial-of-service when they are
executed. As the validator accepts literally anything inside of a
`(?P<word>...)` block, any quadratic backtracking expression can be
hidden therein.
Switch user-provided linkifier patterns to be matched in the Markdown
processor by the `re2` library, which is guaranteed constant-time.
This somewhat limits the possible features of the regular
expression (notably, look-head and -behind, and back-references);
however, these features had never been advertised as working in the
context of linkifiers.
A migration removes any existing linkifiers which would not function
under re2, after printing them for posterity during the upgrade; they
are unlikely to be common, and are impossible to fix automatically.
The denial-of-service in the linkifier validator was discovered by
@erik-krogh and @yoff, as GHSL-2021-118.
This commit changes the edit-linkifier modal to use
dialog_widget instead of edit_fields_modal.
The edit_fields_modal module will be removed in
further commits to avoid code duplication.
This commit modifies the linkifier-edit modal to use newly added
edit_fields_modal framework.
One important change is that we remove the "edit-linkifier-status"
element as the corresponding "edit-fields-modal-status" element
is added in edit_fields_modal.hbs and we also modify the css
accordingly. This "edit-fields-modal-status" is used only for
this modal and remains empty for others, so this change does not
cause problems with other modals.
There is another element which uses "edit-linkifier-status" as a
class, but the css we defined was for "edit-linkifier-status" as
id, so the css change is also safe.
We now validate the linkifier urls and patterns together, and add
the following additional checks:
1. All groups in the pattern must be used in the URL format string.
2. All groups in the URL format string must be declared in the pattern.
Linkifier pattern is now validated inside the `clean` method.
`filter_pattern_validator` is moved from `clean_fields` to `clean`
method as a safe check. As a result of this, a Puppeteer test case
is updated.
NOTE: The changes here are IN ADDITION to the existing validations.
Fixes#16482.
Co-authored-by: akshatdalton <akshat.dak@students.iiit.ac.in>
I'm not sure that settings UI is particularly worth having puppeteer
tests for, so it's possible we shouldn't bring these back at all. But
in any case, it's worth disabling them as they've been failing for
some time.
This commit solves a rare flake, where the `realm
_linkifier.ts` test was failing because there was no
appropriate wait call for the table
(`#admin_linkifiers_table`) to get updated after editing
the pattern.
A modal is added to edit the realm linkifier which
supports ui_report error.
Puppeteer tests to verify linkifier update and an
invalid test to verify that linkifier error messages
are reported on the modal are added as well.
The only downside of this is that it makes it harder to control the
order of these tests; which isn't that important. And the structure
of naming each with its test order fundamentally requires renaming
files when adding/deleting tests, so if we want to control the default
test order, we'd be better off doing that by just hardcoding a list in
the test runner code.
Ganesh Pawar
Alex Vandiver
sahil839
Tim Abbott
Rohitt Vashishtha
Riken Shah
akshatdalton
Abhijeet Prasad Bodas