Commit Graph

139 Commits

Author SHA1 Message Date
Alex Vandiver 79f858b4b8 upload: Pass bytes to create_attachment.
This will be used to analyze the bytes for image metadata.
2024-07-07 14:40:07 -07:00
Alex Vandiver f97a30f240 upload: Reorder arguments to parallel upload_message_attachment. 2024-07-07 14:40:07 -07:00
Alex Vandiver 0a296b2a6e upload: Start storing content-type for new uploads. 2024-07-07 14:40:07 -07:00
roanster007 52692a6448 refactor: Rename `huddle` to `direct_message_group` in non API.
This commit performs a sweep on the first batch of non API
files to rename "huddle" to "direct_message_group`.

It also renames variables and methods of type -
"huddle_message" to "group_direct_message".

This is a part of #28640
2024-07-04 07:56:31 -07:00
Alex Vandiver 17fb23746f upload: Move methods into zerver.lib.upload from .base. 2024-06-26 16:43:11 -07:00
Kenneth Rodrigues 15accfc983 narrow: Use NarrowParameter instead of dictionary.
Earlier we were using the type `OptionalNarrowListT` for all functions
that required "narrow" as a parameter.
This commit changes all the functions accepting a "narrow"
to use a list of the new `NarrowParameter`
instead of `OptionalNarrowListT` which is a list of dicts.
2024-06-20 12:22:37 -07:00
Kenneth Rodrigues 4e05381897 message_fetch: Migrate to typed_endpoint.
Removed the old `narrow_parameter` as we have shifted to
the new `NarrowParameter` Pydantic Object.
This new object provides better error messages for data validation,
hence changed the error messages in `test_message_fetch`.
2024-06-20 12:22:37 -07:00
Kenneth Rodrigues f791d1e1c0 test_message_fetch: Add support for list of error messages. 2024-06-20 12:22:37 -07:00
Kenneth Rodrigues a865977bd5 get_messages_backend: Pass required parameters.
Earlier `num_before` and `num_after` wasn't being passed while
testing.
Moving to typed_endpoint requires all the "required" parameters
to be passed to the function explicitly in tests.
2024-06-20 12:22:37 -07:00
nimish 81ea09be19 search: Add is:followed filter.
Create the is:followed search operator.
Fetch all messages that are from followed topics
using exists.
Update API documentation and changelog.

Co-authored-by: Kenneth Rodrigues <kenneth.nrk123@gmail.com>

Fixes #27309.
2024-06-17 18:22:50 -07:00
roanster007 384a43c79c narrow: Fix server error of operand of "id" operator at large values.
Previously, when the operand of id operator was more than
2147483647, it was raising server error. This is because the
maximum permissible PostgreSQL integers value is 2147483647.

This is fixed by raising a BadNarrowOperatorError in case the
id operand is larger than 2147483647.
2024-06-01 22:38:18 -07:00
Kenneth Rodrigues ba92cd8b23
search: Include history for is:resolved queries.
The previous logic assumed that all `is:` operators were those like `is:muted`
or `is:starred`, which depend on personal data.
2024-05-20 14:37:08 -07:00
Anders Kaseorg 38cae070ed typos: Fix typos caught by typos.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-05-20 13:55:00 -07:00
Prakhar Pratyush fd335c06eb test_message_fetch: Fix a flaky test due to random message populated.
The messages populated in the test database are random in nature,
so there is a possibility of more than one resolved topic
messages returned by 'self.get_and_check_messages()' query
in 'test_get_messages_for_resolved_topics'.

That would result in a flaky behavior. This commit fixes the
flaky behavior.
2024-05-09 14:05:35 -07:00
Alex Vandiver 16a08a8b6b narrow: Do not create huddles when fetching messages.
9a682fb20a started performing message fetching in a read-only
transaction.  However, our use of `get_or_create_huddle` can violate
the read-only promise, and result in a user-facing 500.

In the cases where we're attempting to narrow to a huddle that does
not exist, this is equivalent to a false condition; catch those,
without making the huddle row, and insert a false.
2024-04-16 09:57:55 -07:00
Alex Vandiver 823f732132 narrow: Move python-only implemenation into its own file.
Tornado uses this, and none of the heavy-weight SQLAlchemy code; so it
just adds to the startup time of the Tornado server.
2024-04-16 09:48:11 -07:00
Lauryn Menard 608b30577a narrow: Add backend support for "channels" operator.
Adds backend support for "channels" operator.

This will deprecate/replace the "streams" operator eventually, but
we will keep support of the operator for backwards compatibility
for a while.

Part of renaming stream to channel project.
2024-04-12 13:44:25 -07:00
Lauryn Menard 0e972e26da narrow: Add backend support for "channel" operator.
Adds backend support for "channel" operator.

This will deprecate/replace the "stream" operator eventually, but
we will keep support of the operator for backwards compatibility
for a while.

Part of renaming stream to channel project.
2024-04-12 13:44:25 -07:00
Lauryn Menard 692d5e9706 narrow: Update check to prevent conflicting dm and channel narrow.
Prep commit for deprecating the "stream" search operator and
replacing it with the "channel" operator, and for depreacting the
"streams" operator and replacing it with the "channels" operator.
2024-04-12 13:44:25 -07:00
Lauryn Menard 94ac3f2b58 test-message-fetch: Replace stream with channel for variables/names.
Prep commit for deprecating the "stream" search operator and
replacing it with the "channel" operator, and for depreacting the
"streams" operator and replacing it with the "channels" operator.

Replaces test names, variables and strings to use channel instead
of stream, so that subsequent commits that change the code
are clearer.
2024-04-12 13:44:25 -07:00
Kenneth Rodrigues c3408b56f0 search: Add server support for has:reaction search operator.
Web app support will be a follow-up commit.
2024-04-12 11:57:57 -07:00
Anders Kaseorg 869d9d9a79 ruff: Fix RUF025 Unnecessary dict comprehension for iterable.
This is a preview rule, not yet enabled by default.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-01 09:30:04 -08:00
Alex Vandiver 737a751f5c message: Split MessageDict and friends into its own file. 2024-02-14 12:27:03 -08:00
Alex Vandiver 0f9b7f112b message: Move render_markdown into zerver.lib.markdown. 2024-02-14 12:27:03 -08:00
Anders Kaseorg 4aa2d76bea models: Extract zerver.models.streams.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Anders Kaseorg 3c11fd9466 models: Move some functions to zerver.lib.display_recipient.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Anders Kaseorg cd96193768 models: Extract zerver.models.realms.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Sahil Batra 72aa4b256d message: Do not allow guest to mention inaccessible users. 2023-12-09 16:59:38 -08:00
Alex Vandiver 8d5573b395 narrow: Prevent contradicting DM and stream narrows.
These confused `ok_to_include_history` and caused exceptions looking
for the "flags" column.
2023-12-06 16:06:15 -08:00
Anders Kaseorg 8a7916f21a python: Consistently use from…import for datetime.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-05 12:01:18 -08:00
Prakhar Pratyush c597de6a1d topic_mentions: Rename wildcard_mentioned to stream_wildcard_mentioned.
Rename the existing 'wildcard_mentioned' flag to
'stream_wildcard_mentioned'.

The 'wildcard_mentioned' flag is deprecated and exists for
backwards compatibility.

We have two separate flags for stream and topic wildcard mentions,
i.e., 'stream_wildcard_mentioned' and 'topic_wildcard_mentioned',
respectively.

* stream wildcard mentions: `@all`, `@everyone`, and `@stream`
* topic wildcard mentions: `@topic`

The `wildcard_mentioned` flag is included in the events and
API response if either `stream_wildcard_mentioned` or
`topic_wildcard_mentioned` is set.
2023-11-10 11:06:26 -08:00
Prakhar Pratyush b0ef76bf27 topic_mentions: Set 'topic_wildcard_mentioned' flag on @topic mention.
Earlier, the 'wildcard_mentioned' flag was set for both the
stream and topic wildcard mentions.

Now, the 'topic_wildcard_mentioned' flag is set for topic
wildcard mentions, and the 'wildcard_mentioned' flag is set for
stream wildcard mentions.

We will rename the 'wildcard_mentioned' flag to
'stream_wildcard_mentioned' in a later commit.
2023-11-02 09:25:51 -07:00
Anders Kaseorg a50eb2e809 mypy: Enable new error explicit-override.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-12 12:28:41 -07:00
Alex Vandiver e1aca65d7d narrow: Add usermessage "private" flag limit to direct message queries.
This flag exists to accelerate searches -- use it.
2023-09-27 10:22:42 -07:00
Alex Vandiver 2405b23ad0 narrow: Add realm_id limits for direct message queries.
These queries benefit from the increased specificity of using the
realm / recipient / sender indexes.  The argument from 11a1cb9630
does not apply in these cases, since there are only 2 usermessage rows
for each matching message row for DMs, and few more than that for
huddles.
2023-09-27 10:22:42 -07:00
Alex Vandiver 11a1cb9630 narrow: Skip the realm_id limit on joins across to usermessage.
The unique index on `(user_id, message_id)` that is the
`zerver_usermessage` table is rather specific, and even the PostgreSQL
extended statistics are not enough for it to realize there is a
correlation between the `realm_id` in the message table and the
`user_id` in the usermessage table.  This means that adding the
`realm_id` limit when there is a join to `zerver_usermessage` flips
the query plan from a nested loop of unique usermessage index-only
scan, with an index scan of the messages pkey -- to a parallel hash
join of the messages limit with a index scan of just the user_id limit
on usermessages.  It thinks this is necessary because it thinks that
the `realm_id` limit may remove a large number of messages from the
usermessage set -- which is totally untrue.

Remove the `realm_id` limit if we have a usermessage join.
2023-09-21 13:58:34 -07:00
Anders Kaseorg 2665a3ce2b python: Elide unnecessary list wrappers.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-13 12:41:23 -07:00
Alex Vandiver b94402152d models: Always search Messages with a realm_id or id limit.
Unless there is a limit on `id`, always provide a `realm_id` limit as
well.  We also notate which index is expected to be used in each
query.
2023-09-11 15:00:37 -07:00
Alex Vandiver f9dd2549eb narrow: Set a realm_id limit on messages in user searches. 2023-09-11 15:00:37 -07:00
Anders Kaseorg 48a3588cdb docs: Fix typos caught by ‘typos’.
https://github.com/crate-ci/typos

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-06 18:59:05 -07:00
Tim Abbott ea83c911e9 Revert "narrow: Fix topic highlighting issue with apostrophes in search results."
This reverts commit baede93f69.

This failed tests after rebasing it on top of
5151dd7ff8.
2023-08-15 17:51:03 -07:00
Akshat baede93f69 narrow: Fix topic highlighting issue with apostrophes in search results.
This commit addresses the issue where the topic highlighting
in search results was offset by one character when an
apostrophe was present. The problem stemmed from the disparity
in HTML escaping generated by the function `func.escape_html` which
is used to obtain `topic_matches` differs from the escaping performed
by the function `django.utils.html.escape` for apostrophes (').

func.escape_html | django.utils.html.escape
-----------------+--------------------------
      &#39;      |           &#x27;

To fix this SQL query is changed to return the HTML-escaped
topic name generated by the function `func.escape_html`.

Fixes: #25633.
2023-08-15 17:29:20 -07:00
Akshat 5151dd7ff8 messages_in_narrow: Use `add_narrow_conditions`.
The `messages_in_narrow_backend` function was directly
calling `NarrowBuilder` instead of utilizing the
`add_narrow_conditions` method like `fetch_messages` does.

This behaviour was not combing any search operands together
as it happens inside the `add_narrow_conditions`.

Fixes: https://chat.zulip.org/#narrow/stream/3-backend/topic/messages_in_narrow_backend.20calling.20NarrowBuilder.20directly/near/1611193.

Test added by tabbott.

Signed-off-by: Akshat <akshat25iiit@gmail.com>
2023-08-15 13:18:10 -07:00
Steve Howell f8ec00b895 mypy: Improve type checks for user display recipients. 2023-08-10 18:13:43 -07:00
Steve Howell a54760da0e tests: Add assert_message_stream_name
The get_display_recipient helper is a clumsy way to get
stream names, and it's not even representative of how
most of our code retrieves stream names.

The new helper also double-checks that the Stream
object has the correct recipient id.
2023-08-10 18:13:43 -07:00
Steve Howell 538f498447 tests: Fix clumsy narrow test.
We now explicitly write messages to three different streams,
as well a DM, to make sure each narrow result filters out
all the noise.
2023-08-10 18:13:43 -07:00
Anders Kaseorg 562a79ab76 ruff: Fix PERF401 Use a list comprehension to create a transformed list.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-07 17:23:55 -07:00
Anders Kaseorg 55aa29bef4 ruff: Fix FLY002 Consider f"…" instead of string join.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-07 17:12:41 -07:00
Anders Kaseorg 3b09197fdf ruff: Fix RUF015 Prefer `next(...)` over single element slice.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-23 15:20:53 -07:00
Lauryn Menard 3255281a83 narrow: Support string and integer encoding of "id" operator.
Expands support for the message ID operand for id" operator to be either
a string or an integer. Previously, this operand was always validated as
a string.
2023-07-20 13:14:20 -07:00