Commit Graph

8563 Commits

Author SHA1 Message Date
Alex Vandiver e4a8304f57 thumbnail: Store the post-orientation-transformation dimensions.
Modern browsers respect the EXIF orientation information of images,
applying rotation and/or mirroring as specified in those tags.  The
the `width="..."` and `height="..."` tags are to size the image
_after_ applying those orientation transformations.

The `.width` and `.height` properties of libvips' images are _before_
any transformations are applied.  Since we intend to use these to hint
to rendering clients the size that the image should be _rendered at_,
change to storing (and providing to clients) the dimensions of the
rendered image, not the stored bytes.
2024-07-24 09:56:42 -07:00
Mateusz Mandera 8f461b3c03 email_mirror: Change default topic name if subject ends up empty.
If the email subject is something like `Fwd:`, it gets stripped to an
empty string, activating the "(no topic)" override. This however leads
to failure if the organization enables the setting forcing every message
to have a topic. Such emails should still go through, so we should just
change the topic value used.
2024-07-22 22:31:45 -07:00
Alex Vandiver 2ea0cc0005 thumbnail: Add a data-original-dimensions attribute.
This allows clients to potentially lay out the thumbnails more
intelligently, or to provide a better "progressive-load" experience
when enlarging the thumbnail.
2024-07-22 22:41:10 -04:00
Alex Vandiver 65828b20e9 thumbnail: Factor out a dataclass for markdown image metadata. 2024-07-22 22:41:10 -04:00
Prakhar Pratyush f48e87cd3c fetch_initial_state: Avoid doing one db query per announcement stream.
In 'fetch_initial_state_data' we were doing one database query
per announcement stream.

This commit updates the logic to prefetch those streams using
select_related hence avoiding the extra db queries.

Fixes #28909.
2024-07-22 10:21:22 -07:00
Alex Vandiver 3ac14632d8 thumbnail: Disable libvips cache.
The libvips cache is 100MB, 100 operations, or 100 files, whichever is
less.  A single Django process or worker is extremely unlikely to ever
see the same image twice, much less within those timeframes.

Disable the cache, since it is mostly useless memory usage for our use
case.
2024-07-22 10:19:33 -07:00
Alex Vandiver b42863be4b markdown: Show thumbnails for uploaded images.
Fixes: #16210.
2024-07-21 18:41:59 -07:00
Alex Vandiver 71406ac767 thumbnail: Factor frames into account for IMAGE_BOMB_TOTAL_PIXELS. 2024-07-21 18:41:59 -07:00
Alex Vandiver aacf28f7e3 test_classes: Extract a thumbnailing output format helper. 2024-07-21 18:41:59 -07:00
Alex Vandiver 94ff443c00 test_classes: Merge verbose assertEquals into ZulipTestCase. 2024-07-21 18:41:59 -07:00
Mateusz Mandera f50104d475 import_realm: Update incorrect comment about emoji dir in exported data.
The emoji dir is present in the data from our export tool. This was
added in 468afe4840.

This comment hasn't been updated since
c4b886d8ae, so probably we just forgot to
refresh it when custom emoji export was added.
2024-07-21 13:16:49 -07:00
Kenneth Rodrigues 8b489f4b96 streams: Convert to typed_enpoint. 2024-07-21 12:57:50 -07:00
Kenneth Rodrigues 6815cded83 zerver: Migrate some files to typed_endpoint.
Migrates `invite.py`, `registration.py` and
`email_mirror.py` to use `typed_endpoint`.
2024-07-20 15:46:48 -07:00
Niloth P 16abd82fa5 integrations: Remove Solano integration. 2024-07-20 15:44:17 -07:00
Sahil Batra c6e9173297 exceptions: Add new error class to be used for invalid parameter values.
This would help us in avoiding adding translation everytime we use
this error for a new pair of parameters.
2024-07-19 09:24:54 -07:00
Anders Kaseorg d574200423 tests: Consume streaming responses.
Fixes warnings like “ResourceWarning: unclosed file <_io.FileIO
name='/srv/zulip/var/044e5d44-87aa-4c43-abbb-28a144fa6654/test-backend/run_1238680/worker_0/test_uploads/files/thumbnail/2/1e/jmUuDhQC8WlaSRCuc0zQyx7D/img.tif/100x75.webp'
mode='rb' closefd=True>” with warnings enabled.

deque(…, 0) is an efficient way to consume an iterator documented at
https://docs.python.org/3/library/itertools.html#itertools-recipes
under consume.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-19 09:20:56 -07:00
Tim Abbott de35f44624 zulip_updates: Mention default for send navigation.
Apparently I forgot to merge this before -beta1.
2024-07-18 14:06:04 -07:00
Alex Vandiver 4351cc5914 thumbnail: Move get_image_thumbnail_path and split_thumbnail_path. 2024-07-18 13:50:28 -07:00
Alex Vandiver ecdf5713c2 tests: Factor out ZulipVerboseEqualTest class, and use more. 2024-07-18 13:50:28 -07:00
Alex Vandiver 08191d3f69 tests: Default ENABLE_FILE_LINKS to False.
Test configuration should mirror default production as much as possible.
2024-07-18 13:50:28 -07:00
Alya Abbott d775b6ffff updates: Add update message (level=7). 2024-07-18 12:38:26 -07:00
Mateusz Mandera 4a93149435 settings: Rework how push notifications service is configured.
Instead of the PUSH_NOTIFICATIONS_BOUNCER_URL and
SUBMIT_USAGE_STATISTICS settings, we want servers to configure
individual ZULIP_SERVICE_* settings, while maintaining backward
compatibility with the old settings. Thus, if all the new
ZULIP_SERVICE_* are at their default False value, but the legacy
settings are activated, they need to be translated in computed_settings
to the modern way.
2024-07-17 17:14:06 -07:00
Anders Kaseorg dd0d482d76 narrow: Migrate legacy SQLAlchemy select syntax.
Fixes “sqlalchemy.exc.RemovedIn20Warning: Deprecated API features
detected! These feature(s) are not compatible with SQLAlchemy 2.0.”
with warnings enabled.

https://docs.sqlalchemy.org/en/14/changelog/migration_14.html#change-5284

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-16 14:50:30 -07:00
Prakhar Pratyush df7ed437c2 compose: Show banner to explain interleaved view messages fading.
In an interleaved view when composing a message we fade messages
which the user is not replying to, to reduce the chance they send
a message to a recipient they didn't intend to.
Also, it reduces the visual/cognitive processing required
to figure out where their message is going to go.

But, it's not necessarily clear to users that what the
fading means, so this commit adds a one-time compose banner
to explain what's going on the first time this comes up.

Fixes part of #29076.
2024-07-16 13:52:30 -07:00
Prakhar Pratyush 35380b095f compose: Show banner to explain non interleaved view messages fading.
In a non interleaved view when composing a message to another
conversation we fade messages which the user is not replying to,
to reduce the chance they send a message to a recipient they didn't
intend to. Also, it reduces the visual/cognitive processing required
to figure out where their message is going to go.

But, it's not necessarily clear to users that what the
fading means, so this commit adds a one-time compose banner
to explain what's going on the first time this comes up.

Fixes part of #29076.
2024-07-16 13:52:29 -07:00
Alex Vandiver 556b92810b thumbnail: Advertize the thumbnail formats at client registration. 2024-07-16 13:22:15 -07:00
Alex Vandiver 2e38f426f4 upload: Generate thumbnails when images are uploaded.
A new table is created to track which path_id attachments are images,
and for those their metadata, and which thumbnails have been created.
Using path_id as the effective primary key lets us ignore if the
attachment is archived or not, saving some foreign key messes.

A new worker is added to observe events when rows are added to this
table, and to generate and store thumbnails for those images in
differing sizes and formats.
2024-07-16 13:22:15 -07:00
Alex Vandiver 63dbf67120 tests: Ensure we start with a local storage backend. 2024-07-16 13:22:15 -07:00
Alex Vandiver 229dcd0218 upload: Clean up empty directories in local storage. 2024-07-16 13:22:15 -07:00
Prakhar Pratyush 0de9f7f5cf import_realm: Add support to import 'OnboardingUserMessage' table. 2024-07-16 09:36:02 -07:00
Prakhar Pratyush 762b1c0f5b import_realm: Improve assert statements verifying table name.
This commit improves the assert statements to verify
that the table name is not "usermessage' instead of
verifying that table name doesn't include a substring
"usermessage".

This prep commit will help to avoid assertion error when
importing "onboardingusermessage" table.
2024-07-16 09:36:02 -07:00
Prakhar Pratyush 7d379e00b0 export: Fix 'OnboardingUserMessage' table not being exported.
Earlier, the export tool was logging a warning:
"??? NO DATA EXPORTED FOR TABLE zerver_onboardingusermessage!!!"

This bug was due to not configuring a Config object for
'OnboardingUserMessage' in 'get_realm_config()'.

This commit fixes the bug to export the table properly.
2024-07-16 09:36:02 -07:00
Kenneth Rodrigues a7da24a36f validators: Use cleaner syntax for AfterValidator.
Created a function that returns an `AfterValidator` for `check_int_in`
and `check_string_in` instead of having to use a
`lambda` wraper everytime.
2024-07-15 16:49:32 -07:00
Vector73 a07ebba860 zerver: Replace uri with url in local variables and comments. 2024-07-14 22:30:28 -07:00
Anders Kaseorg 80d3e60df8 ruff: Fix FURB180 Use of `metaclass=abc.ABCMeta`.
This is a preview rule, not yet enabled by default.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-14 13:53:40 -07:00
Anders Kaseorg e3a191b99b ruff: Fix FURB154 Use of repeated consecutive `global`, `nonlocal`.
This is a preview rule, not yet enabled by default.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-14 13:53:18 -07:00
Anders Kaseorg 6412c2d630 ruff: Fix FURB142 Use of set.add() in a for loop.
This is a preview rule, not yet enabled by default.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-14 13:52:59 -07:00
Anders Kaseorg 1e9b6445a9 ruff: Fix PLR6104 Use `+=` to perform an augmented assignment directly.
This is a preview rule, not yet enabled by default.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-14 13:49:51 -07:00
Anders Kaseorg b96feb34f6 ruff: Fix SIM117 Use a single `with` statement with multiple contexts.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-14 13:48:32 -07:00
Tim Abbott b0f144327d narrow: Update new function for modern Python typing syntax.
I apparently needed to be more careful rebasing the typing syntax
modernization work.
2024-07-13 23:06:47 -07:00
Anders Kaseorg 48202389b8 ruff: Bump target-version from py38 to py310.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 3f29bc42b1 ruff: Fix B905 `zip()` without an explicit `strict=` parameter.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 1464009fae ruff: Fix UP038 Use `X | Y` in `isinstance` call instead of `(X, Y)`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 0fa5e7f629 ruff: Fix UP035 Import from `collections.abc`, `typing` instead.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 531b34cb4c ruff: Fix UP007 Use `X | Y` for type annotations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg e08a24e47f ruff: Fix UP006 Use `list` instead of `List` for type annotation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg a2fad74a28 typed_endpoint: Make is_optional understand T | None.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Aman Agrawal 70be9e8c51 narrow: Implement rendering of `with` narrow operators.
Adds server and web app support for processing the new `with`
search operator.

Fixes part of #21505.

Co-authored-by: roanster007 <rohan.gudimetla07@gmail.com>
Co-authored-by: Tim Abbott <tabbott@zulip.com>
2024-07-13 07:18:24 -07:00
Alya Abbott bd04a30bbc updates: Add update message (level=6). 2024-07-12 16:40:41 -07:00
Alex Vandiver 4df9b03451 migrations: Re-thumbnail and rename existing emoji. 2024-07-12 13:26:47 -07:00