Mirror
/
zulip
mirror of https://github.com/zulip/zulip.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
zulip/zerver/lib/thumbnail.py

26 lines
765 B
Python
Raw Normal View History

docs: Add some basic subsystem documentation for thumbnailing.
2018-07-30 22:16:26 +02:00
# See https://zulip.readthedocs.io/en/latest/subsystems/thumbnailing.html
thumbor: Complete implementation of thumbnailing. Various pieces of our thumbor-based thumbnailing system were already merged; this adds the remaining pieces required for it to work: * a THUMBOR_URL Django setting that controls whether thumbor is enabled on the Zulip server (and if so, where thumbor is hosted). * Replaces the overly complicated prototype cryptography logic * Adds a /thumbnail endpoint (supported both on web and mobile) for accessing thumbnails in messages, designed to support hosting both external URLs as well as uploaded files (and applying Zulip's security model for access to thumbnails of uploaded files). * Modifies bugdown to, when THUMBOR_URL is set, render images with the `src` attribute pointing /thumbnail (to provide a small thumbnail for the image), along with adding a "data-original" attribute that can be used to access the "original/full" size version of the image. There are a few things that don't work quite yet: * The S3 backend support is incomplete and doesn't work yet. * The error pages for unauthorized access are ugly. * We might want to rename data-original and /thumbnail?size=original to use some other name, like "full", that better reflects the fact that we're potentially not serving the original image URL.
2018-03-08 09:37:09 +01:00
import os
import sys
thumbnail: Tighten fix for CVE-2019-19775 open redirect. Due to a known but unfixed bug in the Python standard library’s urllib.parse module (CVE-2015-2104), a crafted URL could bypass the validation in the previous patch and still achieve an open redirect. https://bugs.python.org/issue23505 Switch to using django.utils.http.is_safe_url, which already contains a workaround for this bug. Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-12-20 02:58:53 +01:00
from urllib.parse import urljoin
python: Sort imports with isort. Fixes #2665. Regenerated by tabbott with `lint --fix` after a rebase and change in parameters. Note from tabbott: In a few cases, this converts technical debt in the form of unsorted imports into different technical debt in the form of our largest files having very long, ugly import sequences at the start. I expect this change will increase pressure for us to split those files, which isn't a bad thing. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 00:54:34 +02:00
python: Convert deprecated Django is_safe_url. django.utils.http.is_safe_url is a deprecated alias of django.utils.http.url_has_allowed_host_and_scheme as of Django 3.0, and will be removed in Django 4.0. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-16 00:59:20 +02:00
from django.utils.http import url_has_allowed_host_and_scheme
thumbor: Complete implementation of thumbnailing. Various pieces of our thumbor-based thumbnailing system were already merged; this adds the remaining pieces required for it to work: * a THUMBOR_URL Django setting that controls whether thumbor is enabled on the Zulip server (and if so, where thumbor is hosted). * Replaces the overly complicated prototype cryptography logic * Adds a /thumbnail endpoint (supported both on web and mobile) for accessing thumbnails in messages, designed to support hosting both external URLs as well as uploaded files (and applying Zulip's security model for access to thumbnails of uploaded files). * Modifies bugdown to, when THUMBOR_URL is set, render images with the `src` attribute pointing /thumbnail (to provide a small thumbnail for the image), along with adding a "data-original" attribute that can be used to access the "original/full" size version of the image. There are a few things that don't work quite yet: * The S3 backend support is incomplete and doesn't work yet. * The error pages for unauthorized access are ugly. * We might want to rename data-original and /thumbnail?size=original to use some other name, like "full", that better reflects the fact that we're potentially not serving the original image URL.
2018-03-08 09:37:09 +01:00
thumbor: Fix __file__ typo. Replaced '__file__' typo with __file__ which used to add wrong path to sys.path.
2020-04-12 19:44:42 +02:00
ZULIP_PATH = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
thumbor: Complete implementation of thumbnailing. Various pieces of our thumbor-based thumbnailing system were already merged; this adds the remaining pieces required for it to work: * a THUMBOR_URL Django setting that controls whether thumbor is enabled on the Zulip server (and if so, where thumbor is hosted). * Replaces the overly complicated prototype cryptography logic * Adds a /thumbnail endpoint (supported both on web and mobile) for accessing thumbnails in messages, designed to support hosting both external URLs as well as uploaded files (and applying Zulip's security model for access to thumbnails of uploaded files). * Modifies bugdown to, when THUMBOR_URL is set, render images with the `src` attribute pointing /thumbnail (to provide a small thumbnail for the image), along with adding a "data-original" attribute that can be used to access the "original/full" size version of the image. There are a few things that don't work quite yet: * The S3 backend support is incomplete and doesn't work yet. * The error pages for unauthorized access are ugly. * We might want to rename data-original and /thumbnail?size=original to use some other name, like "full", that better reflects the fact that we're potentially not serving the original image URL.
2018-03-08 09:37:09 +01:00
sys.path.append(ZULIP_PATH)
from zerver.lib.camo import get_camo_url
python: Reformat with Black, except quotes. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:19:30 +01:00
thumbnails: Extract user_uploads_or_external as a function. This is a prepartory commit for the upcoming changes. It was meaningful to extract this one out because this function is essentially a condition check on whether a given url is one of the user_uploads or an external one. Based on its value we decide whether a url must be thumbnailed or not and thus this function will also be used in an upcoming commit patching lib/thumbnail.py to do the same check before thumbnail url generation.
2018-09-01 13:39:16 +02:00
def user_uploads_or_external(url: str) -> bool:
python: Convert deprecated Django is_safe_url. django.utils.http.is_safe_url is a deprecated alias of django.utils.http.url_has_allowed_host_and_scheme as of Django 3.0, and will be removed in Django 4.0. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-16 00:59:20 +02:00
return not url_has_allowed_host_and_scheme(url, allowed_hosts=None) or url.startswith(
"/user_uploads/"
)
thumbnails: Extract user_uploads_or_external as a function. This is a prepartory commit for the upcoming changes. It was meaningful to extract this one out because this function is essentially a condition check on whether a given url is one of the user_uploads or an external one. Based on its value we decide whether a url must be thumbnailed or not and thus this function will also be used in an upcoming commit patching lib/thumbnail.py to do the same check before thumbnail url generation.
2018-09-01 13:39:16 +02:00
python: Reformat with Black, except quotes. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:19:30 +01:00
python: Normalize quotes with Black. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:20:45 +01:00
def generate_thumbnail_url(path: str, size: str = "0x0", is_camo_url: bool = False) -> str:
CVE-2019-19775: Close open redirect in thumbnail view. This closes an open redirect vulnerability, one case of which was found by Graham Bleaney and Ibrahim Mohamed using Pysa. Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-12-12 01:28:29 +01:00
path = urljoin("/", path)
thumbor: Complete implementation of thumbnailing. Various pieces of our thumbor-based thumbnailing system were already merged; this adds the remaining pieces required for it to work: * a THUMBOR_URL Django setting that controls whether thumbor is enabled on the Zulip server (and if so, where thumbor is hosted). * Replaces the overly complicated prototype cryptography logic * Adds a /thumbnail endpoint (supported both on web and mobile) for accessing thumbnails in messages, designed to support hosting both external URLs as well as uploaded files (and applying Zulip's security model for access to thumbnails of uploaded files). * Modifies bugdown to, when THUMBOR_URL is set, render images with the `src` attribute pointing /thumbnail (to provide a small thumbnail for the image), along with adding a "data-original" attribute that can be used to access the "original/full" size version of the image. There are a few things that don't work quite yet: * The S3 backend support is incomplete and doesn't work yet. * The error pages for unauthorized access are ugly. * We might want to rename data-original and /thumbnail?size=original to use some other name, like "full", that better reflects the fact that we're potentially not serving the original image URL.
2018-03-08 09:37:09 +01:00
requirements: Remove Thumbor. Thumbor and tc-aws have been dragging their feet on Python 3 support for years, and even the alphas and unofficial forks we’ve been running don’t seem to be maintained anymore. Depending on these projects is no longer viable for us. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-07 00:38:24 +02:00
if url_has_allowed_host_and_scheme(path, allowed_hosts=None):
thumbnail: Tighten fix for CVE-2019-19775 open redirect. Due to a known but unfixed bug in the Python standard library’s urllib.parse module (CVE-2015-2104), a crafted URL could bypass the validation in the previous patch and still achieve an open redirect. https://bugs.python.org/issue23505 Switch to using django.utils.http.is_safe_url, which already contains a workaround for this bug. Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-12-20 02:58:53 +01:00
return path
requirements: Remove Thumbor. Thumbor and tc-aws have been dragging their feet on Python 3 support for years, and even the alphas and unofficial forks we’ve been running don’t seem to be maintained anymore. Depending on these projects is no longer viable for us. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-07 00:38:24 +02:00
return get_camo_url(path)