zulip/zerver
Alex Vandiver fe25517295 email_notifications: Handle empty rendered_messages.
The transforms called from `build_message_payload` use
`lxml.html.fromstring` to parse (and stringify, and re-parse) the HTML
generated by Markdown.  However, this function fails if it is passed
an empty document.  "empty" is broader than just the empty string; it
also includes any document made entirely out of control characters,
spaces, unpaired surrogates, U+FFFE, or U+FFFF, and so forth.  These
documents would fail to parse, and raise a ParserError.

Using `lxml.html.fragment_fromstring` handles these cases, but does by
wrapping the contents in a <div> every time it is called.  As such,
replacing each `fromstring` with `fragment_fromstring` would nest
another layer of `<div>`.

Instead of each of the helper functions re-parsing, modifying, and
stringifying the HTML, parse it once with `fragment_fromstring` and
pass around the parsed document to each helper, which modifies it
in-place.  This adds one outer `<div>`, requiring minor changes to
tests and the prepend-sender functions.

The modification to add the sender is left using BeautifulSoup, as
that sort of transform is much less readable, and more fiddly, in raw
lxml.

Partial fix for #19559.
2021-08-25 15:50:55 -07:00
..
data_import python: Remove locally dead code. 2021-08-19 01:51:37 -07:00
integration_fixtures/nagios tests: Add a test for fixtures for non-webhook integrations. 2021-04-28 08:16:51 -07:00
lib email_notifications: Handle empty rendered_messages. 2021-08-25 15:50:55 -07:00
management sync_ldap_user_data: Log all exceptions. 2021-08-24 07:51:48 -07:00
migrations realm: Add datetime field for scheduled deletion of demo orgs. 2021-08-19 03:09:36 -07:00
openapi api_docs: Mention special value of realm_message_retention_days feild. 2021-08-20 06:35:35 -07:00
tests email_notifications: Handle empty rendered_messages. 2021-08-25 15:50:55 -07:00
tornado typing: Improve typing with assertions. 2021-08-20 05:54:19 -07:00
views marketing_emails: Ask for user's consent at sign-up. 2021-08-20 06:43:26 -07:00
webhooks webhooks/clubhouse: Ignore label removals for story batch updates. 2021-08-20 23:27:14 -07:00
worker missedmessage_emails: Ensure forward progress. 2021-08-20 07:21:39 -07:00
__init__.py django: Upgrade Zulip to Django 3.2 LTS. 2021-05-03 08:36:22 -07:00
apps.py caching: Make sender type optional for flush_cache. 2021-07-26 14:48:07 -07:00
context_processors.py auth: show _OR_ during login only when other methods are available. 2021-07-20 14:29:48 -07:00
decorator.py python: Remove default "utf8" argument for encode(), decode(). 2021-08-02 15:53:52 -07:00
filters.py typing: Fix function signatures. 2021-08-20 05:54:19 -07:00
forms.py marketing_emails: Ask for user's consent at sign-up. 2021-08-20 06:43:26 -07:00
logging_handlers.py logging_handlers: Remove STAGING_ERROR_NOTIFICATIONS setting. 2021-07-07 09:26:02 -07:00
middleware.py typing: Use assertions for responses when appropriate. 2021-08-20 06:02:56 -07:00
models.py realm: Add datetime field for scheduled deletion of demo orgs. 2021-08-19 03:09:36 -07:00
signals.py python: Convert deprecated Django ugettext alias to gettext. 2021-04-15 18:01:34 -07:00