Mirror
/
zulip
mirror of https://github.com/zulip/zulip.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
zulip/zerver/lib/url_preview/oembed.py

52 lines
1.5 KiB
Python
Raw Normal View History

url_preview: Discard url in oembed if server returns invalid json. This fixes the scenario where we'd get errors in the FetchLinksEmbedData queue processor if oembed got invalid json from the URL.
2020-04-11 13:24:06 +02:00
import json
python: Sort imports with isort. Fixes #2665. Regenerated by tabbott with `lint --fix` after a rebase and change in parameters. Note from tabbott: In a few cases, this converts technical debt in the form of unsorted imports into different technical debt in the form of our largest files having very long, ugly import sequences at the start. I expect this change will increase pressure for us to split those files, which isn't a bad thing. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 00:54:34 +02:00
from typing import Any, Dict, Optional
from pyoembed import PyOembedException, oEmbed
Add oembed/Open Graph/Meta tags data retrieval from inline links. This change adds support for displaying inline open graph previews for links posted into Zulip. It is designed to interact correctly with message editing. This adds the new settings.INLINE_URL_EMBED_PREVIEW setting to control whether this feature is enabled. By default, this setting is currently disabled, so that we can burn it in for a bit before it impacts users more broadly. Eventually, we may want to make this manageable via a (set of?) per-realm settings. E.g. I can imagine a realm wanting to be able to enable/disable it for certain URLs.
2016-10-27 12:06:44 +02:00
python: Reformat with Black, except quotes. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:19:30 +01:00
def get_oembed_data(
url: str, maxwidth: int = 640, maxheight: int = 480
) -> Optional[Dict[str, Any]]:
Add oembed/Open Graph/Meta tags data retrieval from inline links. This change adds support for displaying inline open graph previews for links posted into Zulip. It is designed to interact correctly with message editing. This adds the new settings.INLINE_URL_EMBED_PREVIEW setting to control whether this feature is enabled. By default, this setting is currently disabled, so that we can burn it in for a bit before it impacts users more broadly. Eventually, we may want to make this manageable via a (set of?) per-realm settings. E.g. I can imagine a realm wanting to be able to enable/disable it for certain URLs.
2016-10-27 12:06:44 +02:00
try:
data = oEmbed(url, maxwidth=maxwidth, maxheight=maxheight)
url_preview: Discard url in oembed if server returns invalid json. This fixes the scenario where we'd get errors in the FetchLinksEmbedData queue processor if oembed got invalid json from the URL.
2020-04-11 13:24:06 +02:00
except (PyOembedException, json.decoder.JSONDecodeError):
Add oembed/Open Graph/Meta tags data retrieval from inline links. This change adds support for displaying inline open graph previews for links posted into Zulip. It is designed to interact correctly with message editing. This adds the new settings.INLINE_URL_EMBED_PREVIEW setting to control whether this feature is enabled. By default, this setting is currently disabled, so that we can burn it in for a bit before it impacts users more broadly. Eventually, we may want to make this manageable via a (set of?) per-realm settings. E.g. I can imagine a realm wanting to be able to enable/disable it for certain URLs.
2016-10-27 12:06:44 +02:00
return None
python: Normalize quotes with Black. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:20:45 +01:00
oembed_resource_type = data.get("type", "")
image = data.get("url", data.get("image"))
thumbnail = data.get("thumbnail_url")
html = data.pop("html", "")
if oembed_resource_type == "photo" and image:
url_preview: Fix parsing of open graph tags. Our open graph parser logic sloppily mixed data obtained by parsing open graph properties with trusted data set by our oembed parser. We fix this by consistenly using our explicit whitelist of generic properties (image, title, and description) in both places where we interact with open graph properties. The fixes are redundant with each other, but doing both helps in making the intent of the code clearer. This issue fixed here was originally reported as an XSS vulnerability in the upcoming Inline URL Previews feature found by Graham Bleaney and Ibrahim Mohamed using Pysa. The recent Oembed changes close that vulnerability, but this change is still worth doing to make the implementation do what it looks like it does.
2019-12-12 02:10:50 +01:00
return dict(
oembed=True,
image=image,
type=oembed_resource_type,
python: Normalize quotes with Black. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:20:45 +01:00
title=data.get("title"),
description=data.get("description"),
url_preview: Fix parsing of open graph tags. Our open graph parser logic sloppily mixed data obtained by parsing open graph properties with trusted data set by our oembed parser. We fix this by consistenly using our explicit whitelist of generic properties (image, title, and description) in both places where we interact with open graph properties. The fixes are redundant with each other, but doing both helps in making the intent of the code clearer. This issue fixed here was originally reported as an XSS vulnerability in the upcoming Inline URL Previews feature found by Graham Bleaney and Ibrahim Mohamed using Pysa. The recent Oembed changes close that vulnerability, but this change is still worth doing to make the implementation do what it looks like it does.
2019-12-12 02:10:50 +01:00
)
url preview: Use oEmbed html for videos. Ensure that the html is safe, before using it. The html is considered if it is in an iframe with a http/https src, based on the recommendations here: https://oembed.com/#section3 We directly embed the `iframe` html into the lightbox overlay.
2019-05-02 18:58:39 +02:00
python: Normalize quotes with Black. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:20:45 +01:00
if oembed_resource_type == "video" and html and thumbnail:
url_preview: Fix parsing of open graph tags. Our open graph parser logic sloppily mixed data obtained by parsing open graph properties with trusted data set by our oembed parser. We fix this by consistenly using our explicit whitelist of generic properties (image, title, and description) in both places where we interact with open graph properties. The fixes are redundant with each other, but doing both helps in making the intent of the code clearer. This issue fixed here was originally reported as an XSS vulnerability in the upcoming Inline URL Previews feature found by Graham Bleaney and Ibrahim Mohamed using Pysa. The recent Oembed changes close that vulnerability, but this change is still worth doing to make the implementation do what it looks like it does.
2019-12-12 02:10:50 +01:00
return dict(
oembed=True,
image=thumbnail,
type=oembed_resource_type,
html=strip_cdata(html),
python: Normalize quotes with Black. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:20:45 +01:00
title=data.get("title"),
description=data.get("description"),
url_preview: Fix parsing of open graph tags. Our open graph parser logic sloppily mixed data obtained by parsing open graph properties with trusted data set by our oembed parser. We fix this by consistenly using our explicit whitelist of generic properties (image, title, and description) in both places where we interact with open graph properties. The fixes are redundant with each other, but doing both helps in making the intent of the code clearer. This issue fixed here was originally reported as an XSS vulnerability in the upcoming Inline URL Previews feature found by Graham Bleaney and Ibrahim Mohamed using Pysa. The recent Oembed changes close that vulnerability, but this change is still worth doing to make the implementation do what it looks like it does.
2019-12-12 02:10:50 +01:00
)
url preview: Show inline images as previews for oEmbed photo pages.
2019-05-26 06:27:01 +02:00
url_preview: Fix parsing of open graph tags. Our open graph parser logic sloppily mixed data obtained by parsing open graph properties with trusted data set by our oembed parser. We fix this by consistenly using our explicit whitelist of generic properties (image, title, and description) in both places where we interact with open graph properties. The fixes are redundant with each other, but doing both helps in making the intent of the code clearer. This issue fixed here was originally reported as an XSS vulnerability in the upcoming Inline URL Previews feature found by Graham Bleaney and Ibrahim Mohamed using Pysa. The recent Oembed changes close that vulnerability, but this change is still worth doing to make the implementation do what it looks like it does.
2019-12-12 02:10:50 +01:00
# Otherwise, start with just the embed type.
return dict(
type=oembed_resource_type,
python: Normalize quotes with Black. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:20:45 +01:00
title=data.get("title"),
description=data.get("description"),
url_preview: Fix parsing of open graph tags. Our open graph parser logic sloppily mixed data obtained by parsing open graph properties with trusted data set by our oembed parser. We fix this by consistenly using our explicit whitelist of generic properties (image, title, and description) in both places where we interact with open graph properties. The fixes are redundant with each other, but doing both helps in making the intent of the code clearer. This issue fixed here was originally reported as an XSS vulnerability in the upcoming Inline URL Previews feature found by Graham Bleaney and Ibrahim Mohamed using Pysa. The recent Oembed changes close that vulnerability, but this change is still worth doing to make the implementation do what it looks like it does.
2019-12-12 02:10:50 +01:00
)
url preview: Use oEmbed html for videos. Ensure that the html is safe, before using it. The html is considered if it is in an iframe with a http/https src, based on the recommendations here: https://oembed.com/#section3 We directly embed the `iframe` html into the lightbox overlay.
2019-05-02 18:58:39 +02:00
python: Reformat with Black, except quotes. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:19:30 +01:00
oembed: Remove unsound HTML filtering. The frontend now takes care of confining the HTML. Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-12-12 09:39:41 +01:00
def strip_cdata(html: str) -> str:
# Work around a bug in SoundCloud's XML generation:
# <html>&lt;![CDATA[&lt;iframe ...&gt;&lt;/iframe&gt;]]&gt;</html>
python: Normalize quotes with Black. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:20:45 +01:00
if html.startswith("<![CDATA[") and html.endswith("]]>"):
url preview: Use oEmbed html for videos. Ensure that the html is safe, before using it. The html is considered if it is in an iframe with a http/https src, based on the recommendations here: https://oembed.com/#section3 We directly embed the `iframe` html into the lightbox overlay.
2019-05-02 18:58:39 +02:00
html = html[9:-3]
oembed: Remove unsound HTML filtering. The frontend now takes care of confining the HTML. Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-12-12 09:39:41 +01:00
return html