Commit Graph

199 Commits

Author SHA1 Message Date
Anders Kaseorg 365fe0b3d5 python: Sort imports with isort.
Fixes #2665.

Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.

Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start.  I expect this change will increase pressure for us to split
those files, which isn't a bad thing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:45:32 -07:00
Anders Kaseorg 69730a78cc python: Use trailing commas consistently.
Automatically generated by the following script, based on the output
of lint with flake8-comma:

import re
import sys

last_filename = None
last_row = None
lines = []

for msg in sys.stdin:
    m = re.match(
        r"\x1b\[35mflake8    \|\x1b\[0m \x1b\[1;31m(.+):(\d+):(\d+): (\w+)", msg
    )
    if m:
        filename, row_str, col_str, err = m.groups()
        row, col = int(row_str), int(col_str)

        if filename == last_filename:
            assert last_row != row
        else:
            if last_filename is not None:
                with open(last_filename, "w") as f:
                    f.writelines(lines)

            with open(filename) as f:
                lines = f.readlines()
            last_filename = filename
        last_row = row

        line = lines[row - 1]
        if err in ["C812", "C815"]:
            lines[row - 1] = line[: col - 1] + "," + line[col - 1 :]
        elif err in ["C819"]:
            assert line[col - 2] == ","
            lines[row - 1] = line[: col - 2] + line[col - 1 :].lstrip(" ")

if last_filename is not None:
    with open(last_filename, "w") as f:
        f.writelines(lines)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 16:04:12 -07:00
Graham Bleaney 461d5b1a3e pysa: Introduce sanitizers, models, and inline marking safe.
This commit adds three `.pysa` model files: `false_positives.pysa`
for ruling out false positive flows with `Sanitize` annotations,
`req_lib.pysa` for educating pysa about Zulip's `REQ()` pattern for
extracting user input, and `redirects.pysa` for capturing the risk
of open redirects within Zulip code. Additionally, this commit
introduces `mark_sanitized`, an identity function which can be used
to selectively clear taint in cases where `Sanitize` models will not
work. This commit also puts `mark_sanitized` to work removing known
false postive flows.
2020-06-11 12:57:49 -07:00
Anders Kaseorg 67e7a3631d python: Convert percent formatting to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 15:02:09 -07:00
Anders Kaseorg 444fbbf964 python: Whitespace fixes from autopep8.
Generated by autopep8.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 15:21:30 -07:00
whoodes cea7d713cd requirements: Upgrade boto to boto3.
Fixes: #3490

Contributors include:

Author:    whoodes <hoodesw@hawaii.edu>
Author:    zhoufeng1989 <zhoufengloop@gmail.com>
Author:    rht <rhtbot@protonmail.com>
2020-05-26 23:18:07 -07:00
Anders Kaseorg bdc365d0fe logging: Pass format arguments to logging.
https://docs.python.org/3/howto/logging.html#optimization

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-02 10:18:02 -07:00
Anders Kaseorg fead14951c python: Convert assignment type annotations to Python 3.6 style.
This commit was split by tabbott; this piece covers the vast majority
of files in Zulip, but excludes scripts/, tools/, and puppet/ to help
ensure we at least show the right error messages for Xenial systems.

We can likely further refine the remaining pieces with some testing.

Generated by com2ann, with whitespace fixes and various manual fixes
for runtime issues:

-    invoiced_through: Optional[LicenseLedger] = models.ForeignKey(
+    invoiced_through: Optional["LicenseLedger"] = models.ForeignKey(

-_apns_client: Optional[APNsClient] = None
+_apns_client: Optional["APNsClient"] = None

-    notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
-    signup_notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
+    notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
+    signup_notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)

-    author: Optional[UserProfile] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE)
+    author: Optional["UserProfile"] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE)

-    bot_owner: Optional[UserProfile] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL)
+    bot_owner: Optional["UserProfile"] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL)

-    default_sending_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
-    default_events_register_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
+    default_sending_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
+    default_events_register_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)

-descriptors_by_handler_id: Dict[int, ClientDescriptor] = {}
+descriptors_by_handler_id: Dict[int, "ClientDescriptor"] = {}

-worker_classes: Dict[str, Type[QueueProcessingWorker]] = {}
-queues: Dict[str, Dict[str, Type[QueueProcessingWorker]]] = {}
+worker_classes: Dict[str, Type["QueueProcessingWorker"]] = {}
+queues: Dict[str, Dict[str, Type["QueueProcessingWorker"]]] = {}

-AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional[LDAPSearch] = None
+AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional["LDAPSearch"] = None

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-22 11:02:32 -07:00
Mateusz Mandera 4018dcb8e7 upload: Include filename at the end of temporary access URLs. 2020-04-20 10:25:48 -07:00
Tim Abbott 0ccc0f02ce upload: Support requesting a temporary unauthenticated URL.
This is be useful for the mobile and desktop apps to hand an uploaded
file off to the system browser so that it can render PDFs (Etc.).

The S3 backend implementation is simple; for the local upload backend,
we use Django's signing feature to simulate the same sort of 60-second
lifetime token.

Co-Author-By: Mateusz Mandera <mateusz.mandera@protonmail.com>
2020-04-17 09:08:10 -07:00
Tim Abbott 7f582b3861 upload: Increase the lifetime of signed upload URLs.
For some mobile use cases, 15 seconds is potentially too short for a
busy+slow device to open a browser and fetch the URL.  60 seconds is
plenty, and doesn't carry a materially increased security risk.
2020-04-17 09:08:10 -07:00
Anders Kaseorg c734bbd95d python: Modernize legacy Python 2 syntax with pyupgrade.
Generated by `pyupgrade --py3-plus --keep-percent-format` on all our
Python code except `zthumbor` and `zulip-ec2-configure-interfaces`,
followed by manual indentation fixes.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-09 16:43:22 -07:00
Anders Kaseorg 7ff9b22500 docs: Convert many http URLs to https.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-03-26 21:35:32 -07:00
Vishnu Ks af3a37b58b upload: Refactor out realm_avatar_and_logo_path function. 2020-02-03 14:09:05 -08:00
Tim Abbott 7ccc8373e2 bugdown: Fix logic for extracting attachment path_id.
In 3892a8afd8, we restructured the
system for managing uploaded files to a much cleaner model where we
just do parsing inside bugdown.

That new model had potentially buggy handling of cases around both
relative URLs and URLS starting with `realm.host`.

We address this by further rewriting the handling of attachments to
avoid regular expressions entirely, instead relying on urllib for
parsing, and having bugdown output `path_id` values, so that there's
no need for any conversions between formats outside bugdowm.

The check_attachment_reference_change function for processing message
updates is significantly simplified in the process.

The new check on the hostname has the side effect of requiring us to
fix some previously weird/buggy test data.

Co-Author-By: Anders Kaseorg <anders@zulipchat.com>
Co-Author-By: Rohitt Vashishtha <aero31aero@gmail.com>
2019-12-12 20:30:26 -08:00
Rohitt Vashishtha 3fbb050216 messages: Remove dependence on regex for claiming attachments.
This commit wraps up the work to remove basic regex based parsing
of messages to handle attachment claiming/unclaiming. We now use
the more dependable Bugdown processor to find potential links and
only operate upon those links instead of parsing the full message
content again.
2019-12-11 11:03:49 -08:00
Tim Abbott 7e0ea61b00 upload: Support S3-compatible S3 hosting providers.
Previously, we were hardcoding the domain s3.amazonaws.com.  Given
that we already have an interface for configuring the host in
/etc/zulip/boto.cfg (which in turn, automatically configures boto), we
just need to actually use the value configured in boto for what S3
hostname to use.

We don't have tests for this new use case, in part because they're
likely annoying to write with `moto` and there hasn't been a huge
amount of demand for it.  Since this doesn't regress existing S3
backend support, it seems worth merging.
2019-09-24 17:17:21 -07:00
Tim Abbott b8b0ae362c uploads: Only initialize S3 connection once in __init__.
This should be a mild performance optimization for the S3
authentication backend, since we aren't initializing unnecessary
duplicate connections.
2019-09-24 17:15:44 -07:00
Tim Abbott 96726c00ce export: Fix broken URLs in UI with S3 backend.
Apparently, the Zulip notifications (and resulting emails) were
correct, but the download links inside the Zulip UI were incorrectly
not including S3 prefix on the URL, making them not work.

While we're at this, we rewrite the somewhat convoluted previous
system for formatting the data export output.
2019-09-24 13:56:49 -07:00
Anders Kaseorg 780ecb672b CVE-2019-16216: Fix MIME type validation.
* Whitelist a small number of image/ types to be served as
  non-attachments.
* Serve the file using the type that we validated rather than relying
  on an independent guess to match.

This issue can lead to a stored XSS security vulnerability for older
browsers that don't support Content-Security-Policy.

It primarily affects servers using Zulip's local file uploads backend
for servers running Ubuntu 16.04 Xenial or newer; the legacy local
file upload backend for (now EOL) Ubuntu 14.04 Trusty was not affected
and it has limited impact for the S3 upload backend (which uses an
unprivileged S3 bucket domain to serve files).

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-11 15:46:36 -07:00
neiljp (Neil Pilgrim) 5f673f5820 mypy: Remove type ignores after boto stub improvements. 2019-08-06 23:24:56 -07:00
Wyatt Hoodes 22481f63bf upload: Fix typing for key variable. 2019-07-29 15:23:10 -07:00
Wyatt Hoodes b1900c406a public_export: Add logic for deleting the export tarball.
The path to the uploaded tarball is reconstructed via the relative url and
removed with the canonical methods in `upload.py`.
2019-07-26 15:52:03 -07:00
Wyatt Hoodes e331a758c3 python: Migrate open statements to use with.
This is low priority, but it's nice to be consistently using the best
practice pattern.

Fixes: #12419.
2019-07-20 15:48:52 -07:00
Wyatt Hoodes af4eb8c0d5 export/upload: Refactor tarball upload logic to upload_backend.
The conditional block containing the tarball upload logic for both S3
and local uploads was deconstructed and moved to the more appropriate
location within `zerver/lib/upload.py`.
2019-07-03 15:40:35 -07:00
vinitS101 a6eda858d0 ldap: Fix avatar sync not working with the S3 backend.
This fixes an issue that caused LDAP synchronization to fail for
avatars.  The problem occurred due to the lack of a 'name' attribute
on the BytesIO object that we pass to the upload backend (which is
only used in the S3 backend for computing Content-Type).

Fixes #12411.
2019-06-13 15:12:13 -07:00
Anders Kaseorg 61982d9d47 uploads: Revert "Url encoded name of the file should be an ascii."
This reverts commit fd9dd51d16 (#1815).

The issue described does not exist in Python 3, where urllib.parse now
_only_ accepts (Unicode) str and does the right thing with it.  The
workaround was not being triggered and would have failed if it were.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-22 22:28:39 -07:00
Vishnu Ks e522308507 models: Move currently_used_upload_space function to Realm model. 2019-03-04 18:46:13 -08:00
sahil839 7157edf4af settings: Add support for uploading logo for night mode.
This adds a new field named realm_night_logo which is used for
displaying the organization logo when the user is in night mode.

Fixes #11176.
2019-02-18 15:15:57 -08:00
Anders Kaseorg f0ecb93515 zerver core: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:41:24 -08:00
Daniel Bradburn 1e65cdd072 emoji: Only resize custom emoji that need it.
This additional logic to prevent resizing is certain circumstances
(file size, dimensions) is necessary because the pillow gif handling
code seems to be rather flaky with regards to handling gif color
palletes, causing broken gifs after resizing.  The workaround is to
only resize when absolutely necessary (e.g. because the file is larger
than 128x128 or 128KB).

Fixes #10351.
2019-01-29 10:33:50 -08:00
Harshit Bansal 45d50715c5 uploads: Return a HTTP 400 error for a decompression bomb attack.
Fixes: #11215.
2019-01-13 08:17:24 +00:00
Harshit Bansal 22b67a5207 uploads: Tag error messages for translation. 2019-01-13 06:24:09 +00:00
Tim Abbott c03615b982 import: Fix uploading avatars with S3 upload backend.
This should hopefully be the last commit of this form; ultimately, my
hope is that we'll be able to refactor the semi-duplicated logic in
this file to avoid so much effort going into keeping this correct.
2019-01-02 15:23:09 -08:00
Tim Abbott c995e8e2ae import: Ensure presence of basic avatar images for HipChat.
Our HipChat conversion tool didn't properly handle basic avatar
images, resulting in only the medium-size avatar images being imported
properly.  This fixes that bug by asking the import tool to do the
thumbnailing for the basic avatar image (from the .original file) as
well as the medium avatar image.
2018-12-27 17:47:09 -08:00
Joshua Pan ad1df0ebeb settings: Add support for customizing the top-left logo.
This adds a new realm_logo field, which is a horizontal-format logo to
be displayed in the top-left corner of the webapp, and any other
places where we might want a wide-format branding of the organization.

Tweaked significantly by tabbott to rebase, fix styling, etc.

Fixing the styling of this feature's loading indicator caused me to
notice the loading indicator for the realm_icon feature was also ugly,
so I fixed that too.

Fixes #7995.
2018-12-18 12:44:52 -08:00
Tim Abbott 6a9e6503b7 python: Remove unnecessary NonBinaryStr TypeVar.
This was a potentially useful value (later superceded by use of Text)
when first introduced, but now that we're on Python 3, it should just
be str.
2018-11-27 11:44:09 -08:00
Tim Abbott 3bb0956046 upload: Fix ensure_medium_avatar_image for S3 backend.
Previously, it tried to interact with the wrong path for the original
image.
2018-11-26 20:20:27 -08:00
Tim Abbott 0a751567a3 upload: Fix missing mypy return type annotation. 2018-10-11 16:11:20 -07:00
Vishnu Ks 5bdadc8061 upload: Create function to delete avatar image. 2018-10-11 15:30:26 -07:00
Vishnu Ks 1d94fc7dbb upload: Extract function to delete file. 2018-10-11 15:30:26 -07:00
Harshit Bansal 25fa9a25ff emoji: Add support for animated GIF images.
This commit adds 'resize_gif()' function which extracts each frame,
resize it and coalesces them again to form the resized GIF while
preserving the duration of the GIF. I read some stackoverflow
answers all of which were referring to BiggleZX's script
(https://gist.github.com/BigglesZX/4016539) for working with animated
GIF. I modified the script to fit to our usecase and did some manual
testing but the function was failing for some specific GIFs and was not
preserving the duration of animation. So I went ahead and read about
GIF format itself as well as PIL's `GifImagePlugin` code and came up
with this simple function which gets the worked done in a much cleaner
way. I tested this function on a number of GIF images from giphy.com
and it resized all of them correctly.

Fixes: #9945.
2018-08-04 11:46:58 -07:00
Anders Kaseorg 037f696d26 Enable pycodestyle W605 (invalid escape sequence).
The only changes visible at the AST level, checked using
https://github.com/asottile/astpretty, are

zerver/lib/test_fixtures.py:
'\x1b\\[(1|0)m' ↦ '\\x1b\\[(1|0)m'
'\\[[X| ]\\] (\\d+_.+)\n' ↦ '\\[[X| ]\\] (\\d+_.+)\\n'

which is fine because re treats '\\x1b' and '\\n' the same way as
'\x1b' and '\n'.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-03 16:54:46 +02:00
Vishnu Ks 53237d39aa registration: Allow users to import profile picture. 2018-06-06 13:29:42 -07:00
Vishnu Ks ca87cf4c97 upload: Seperate writing avatars into s3 as function. 2018-06-06 13:29:42 -07:00
Vishnu Ks 7960785261 upload: Separate writing avatars as local files into function. 2018-06-06 13:29:42 -07:00
Shubham Padia cd1233d3f7 upload: Add test jpg image with no exif data. 2018-05-31 08:08:59 -07:00
Tim Abbott f8e8d1fb27 upload: Suppress coverage warnings for now. 2018-05-30 09:31:47 -07:00
Tim Abbott bbcdf8ca99 upload: Fix logic for orientation when uploading avatars.
The previous logic was broken for an avatar without exif data.
2018-05-30 09:22:36 -07:00
Shubham Padia 8b8a9be377 upload: Rotate image according to exif data when resizing avatar and emojis.
Fixes the avatar/emoji part of #8177.

Does not address the issue with uploaded images, since we don't do
anything with them.

Also adds 3 images with different orientation exif tags to
test-images.
2018-05-29 10:39:39 -07:00
RobbieClarken f81b936727 zerver/tests: Require 100% test coverage of zerver/lib/upload.py.
The last line here is impossible to test.

Fixes #4489.
2018-05-15 10:52:20 -07:00
Aditya Bansal a68376e2ba zerver/lib: Change use of typing.Text to str. 2018-05-12 15:22:39 -07:00
Tim Abbott 956bd74905 attachments: Send events for attachment updates.
We send add events on upload, update events when sending a message
referencing it, and delete updates on removal.

This should make it possible to do real-time sync for the attachments
UI.

Based in part on work by Aastha Gupta.
2018-05-04 16:22:27 -07:00
Puneeth Chaganti 4ce8f2aaa2 upload: Rename upload_message_image to upload_message_file.
Tweaked by tabbott to also fix a Slack import comment.
2018-03-30 13:38:31 -07:00
Shubham Padia 13664f1289 uploads: Convert CMYK to RGB when saving avatar/realm icon as png.
Fixes #8546.
PNG does not support CMYK mode. CMYK file is converted to RGB and
then saved as PNG.
2018-03-02 12:57:22 -08:00
neiljp (Neil Pilgrim) 4efb83ee29 mypy: Amend logic in check_upload_within_quota to avoid None. 2018-02-19 09:20:30 -08:00
rht 9161f8c39b zerver/lib: Remove u prefix from strings. 2018-02-05 12:12:58 -08:00
Vishnu Ks 43a6439b3b upload: Enforce per-realm quota. 2018-01-29 16:06:11 -08:00
Greg Price 55cf54c087 upload: Remove old per-user quota feature.
We'll replace this primarily with per-realm quotas (plus the simple
per-file limit of settings.MAX_FILE_UPLOAD_SIZE, 25 MiB by default).

We do want per-user quotas too, but they'll need some more management
apparatus around them so an admin has a practical way to set them
differently for different users.  And the error handling in this
existing code is rather confused.  Just clear this feature out
entirely for now; then we'll build the per-realm version more cleanly,
and then we can later add back per-realm quotas modelled after that.

The migration to actually remove the field is in a subsequent commit.

Based in part on work by Vishnu Ks (hackerkid).
2018-01-29 16:06:11 -08:00
Patrick Grave f2b322c32d mypy: Use Python 3 syntax for typing in upload.py 2018-01-23 08:49:42 -08:00
Vishnu Ks 7dbb4cdae2 uploads: Remove ExceededQuotaError class.
We are no longer using it.
2018-01-18 15:38:08 -08:00
rht 185fd99816 mypy: Use Python 3 type syntax in several files. 2017-12-30 07:34:51 -05:00
Rhea Parekh cc872ad976 zerver/lib/upload.py: Sweep force_bytes from ensure_medium_avatar_image. 2017-12-26 09:09:31 -05:00
Rhea Parekh b792a90842 Remove unused imports in the codebase. 2017-12-26 09:09:31 -05:00
Vaibhav Sagar 711f7bca3b upload: Fix typo in animated GIF error message.
s/must be have/must have the/.
2017-11-24 09:08:38 -08:00
rht 33b1a541d7 zerver/lib: Use python 3 syntax for typing.
With tweaks by tabbott to fix line spacing.
2017-11-18 16:09:04 -08:00
rht 09af29b051 zerver/lib: Text-wrap long lines exceeding 110. 2017-11-15 10:58:03 -08:00
rht cceba2023c Use makedirs directly instead of wrapping it in mkdirs. 2017-11-14 09:47:09 -08:00
rht 72ce1c6501 Use a more concise `mkdir -p` when the situation permits. 2017-11-14 09:47:09 -08:00
Steve Howell 3158ed01bf Remove force_str() in upload.py. 2017-11-09 10:43:20 -08:00
Steve Howell b0bb09cdb8 Remove force_text() in get_signed_upload_url(). 2017-11-09 10:43:20 -08:00
Steve Howell 7da6beac99 Remove force_text() in upload.get_file_info(). 2017-11-09 10:43:20 -08:00
Steve Howell a9f42e6266 Remove force_text() from sanitize_name(). 2017-11-09 10:43:20 -08:00
rht 5ee40bf718 Remove usage of six.moves.binary_type. 2017-11-09 10:00:00 -08:00
rht 19bd335cbb Change urllib import to be Python 3-specific. 2017-11-07 10:46:42 -08:00
rht e311842a1b zerver/lib: Remove inheritance from object. 2017-11-06 08:53:48 -08:00
rht fef7d6ba09 zerver/lib: Remove u prefix from strings.
License: Apache-2.0
Signed-off-by: rht <rhtbot@protonmail.com>
2017-11-03 15:34:37 -07:00
rht f43e54d352 zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
Tim Abbott 5184c64a92 upload: Fix uploading files with Python 3.
Apparently there's a bug in typeshed's stubs for uploading files,
which resulted in our S3 upload code being broken on Python 3.
2017-08-16 07:33:23 -07:00
Harshit Bansal 5a5072730c realm_emoji: Don't accept animated realm emoji with unequal dimensions.
Fixes: #3654.
2017-08-05 18:07:21 -07:00
Greg Price 37dbcefa58 upload: Use the new ErrorCode system in our custom error classes.
All JsonableError subclasses now have corresponding ErrorCode values
of their own, reducing the number of different patterns for using
the new JsonableError API.
2017-07-24 16:41:22 -07:00
Nathan Miller 2311e169ec mypy: Various strict-optional fixes in zerver. 2017-05-26 10:10:20 -07:00
Rick Chern 70d68f7e71 Refactoring: Replace get_user_profile_by_email() in lib/upload.py 2017-05-23 12:37:49 -07:00
Aditya Bansal 85aa07e2d4 pep8: Add compliance with rule E261 to zerver/lib/upload.py. 2017-05-18 03:00:32 +05:30
K.Kanakhin f13d6a18eb realm-emoji: Add realm emoji uploading instead url providing.
- Add file_name field to `RealmEmoji` model and migration.
- Add emoji upload supporting to Upload backends.
- Add uploaded file processing to emoji views.
- Use emoji source url as based for display url.
- Change emoji form for image uploading.
- Fix back-end tests.
- Fix front-end tests.
- Add tests for emoji uploading.

Fixes #1134
2017-05-01 14:50:20 -07:00
Tim Abbott e90748348b attachment: Remove unused claim_attachment return value. 2017-04-18 21:27:29 -07:00
Tim Abbott 568b59291b attachment: Improve rules for managing attachment ownership.
The previous logic was that anyone with a link to a file could send it
to other users, but only the owner could make a file realm-public.
This had some confusing corner cases.

The new logic is much simpler:
* Only the file's owner/uploader can include a file in a message for
  the first time.
* Anyone with access to read a file can share it with others by
  including it in messages they send.
* Once a file has been sent to a public stream, any user in the realm
  can access it.
2017-04-18 21:27:29 -07:00
Tim Abbott 1747427bf3 upload: Fix URL for realm icons with S3 backend. 2017-04-12 19:48:24 -07:00
Tim Abbott 8775a22663 avatar: Fix buggy avatar URL scheme with S3 backend.
Also adds tests for the avatar URL scheme.
2017-03-21 16:12:04 -07:00
adnrs96 3f58e20993 Refactor: Change user_avatar_hash with user_avatar_path at all calls.
In this commit we change user_avatar_hash with user_avatar_path which
now returns paths to avatars based on the email hash.

Tweaked by tabbott to avoid an import loop.
2017-03-04 18:39:49 -08:00
Philip Skomorokhov 866a7b06b2 upload: Limit total size of files uploaded by a user to 1GB.
Fixes #3884.
2017-03-04 18:08:30 -08:00
Raghav Jajodia a3a03bd6a5 mypy: Added Dict, List and Set imports.
Fixed mypy errors associated with the upgrade.
2017-03-04 14:33:44 -08:00
adnrs96 9eb47f108c Refactor: Change upload_avatar_image to accept two user profiles.
In this commit we just change the upload_avatar_image function to accept
two user_profiles acting_user_profile and target_user_profile. Basically
email param is dropped for a target_user_profile so that avatar's could
be moved lateron to user id based storage.
2017-03-03 18:15:15 -08:00
PhilSk 53f3d84af2 attachment: Add 'size' field tracking size of uploaded files.
This tracking will make it possible in the future to limit the total
size of uploads on a per-user or per-organization basis.

Fixes #3774.
2017-03-01 15:58:21 -08:00
K.Kanakhin 257bb40698 realm-icon: Add realm icon feature.
- Add realm icon fields to realm model.
- Add migration for new realm model's field.
- Add views for icon uploading and deleting.
- Add routes for realm icons views.
- Add JS widget for realm icon upload setting.
- Add realm icon upload to administration
  organization setting.
- Add tests for realm icons.

Fixes #3660.
2017-02-26 12:16:07 -08:00
Tim Abbott c5ed119f89 upload: Rename BadImageError exception to not mention avatars.
This is preparation for using that function to decode realm icons as
well.
2017-02-26 12:16:07 -08:00
Tim Abbott 9f0c1db430 mypy: clean up strict optional errors in upload.py. 2017-02-10 23:53:44 -08:00
Tim Abbott bb5d81281c mypy: Upgrade to new package name and version 0.571.
Fixes #3448.
2017-02-10 23:53:44 -08:00
Eklavya Sharma dd0e1f6a4c Use correct string type in boto function parameters.
boto's stubs have been updated in mypy 0.4.7, which has given us
more information about what type of strings are expected as
parameters in various functions.
2017-02-06 22:37:37 -08:00
Tim Abbott 4e171ce787 lint: Clean up E126 PEP-8 rule. 2017-01-23 22:06:13 -08:00