Commit Graph

16860 Commits

Author SHA1 Message Date
Lauryn Menard d84fd73db4 markdown-processor: Update insertion_index check for multiple classes.
Updates find_proper_insertion_index to check for the inline image
classes as matching at least one of the classes in the element's
attrib["class"] so that cases where an inline preview image has
multiple classes, like YouTube video previews, will have the
correct insertion index.

Fixes #26186.
2023-07-07 11:07:45 -04:00
Alex Vandiver ff53ee8e28 markdown: Only attempt to adjust /wiki/File: paths on Wikipedia. 2023-07-06 17:50:25 -07:00
Lalit 46b582689a tests: Improve automated tests for submessages.
Added an additional test case to `test_submessages.py` for testing the
message object containing `submessages` meta data.

Previous to this commit we were never validating the `submessage` schema
in the `message` objects.

Fixes #25896.
2023-07-06 16:35:46 -07:00
Zixuan James Li eebe46ad1c test_classes: Do not necessary wrap test cases in a transaction.
By relocating helper methods into a mixin class, we can be more flexible
with managing transactions in test cases, without always forcing the
django.test.TestCase behavior of always putting the test case into an
atomic transaction.

We include a check for side effects in ZulipTransactionTestCase. It only
checks for the set of row ids in all tables before and after each test.
It is not a comprehensive check for side effects, but should be
sufficient for the basics without much performance overhead.
2023-07-06 11:44:50 -07:00
arghyadeep10 1808cdec90 uploads: Improve file not found message.
It replaces the "File not found." text with:
"This file does not exist or has been deleted."

At present when a file is deleted it results in a confusing
experience when looking at the "File not found." message.
In order to clarify the situation is not a bug, the message
has been replaced with a better alternative.

Fixes part of Issue #23739.
2023-07-06 09:32:41 -07:00
Prakhar Pratyush 179d5cb37d mention: Replace 'wildcards' with 'stream_wildcards'.
This prep commit replaces the 'wildcard' keyword in the codebase
with 'stream_wildcard' at some places for better readability, as
we plan to introduce 'topic_wildcards' as a part of the
'@topic mention' project.

Currently, 'wildcards = ["all", "everyone", "stream"]' which is an
alias to mention everyone in the stream, hence better renamed as
'stream_wildcards'.

Eventually, we will have:
'stream_wildcard' as an alias to mention everyone in the stream.
'topic_wildcard' as an alias to mention everyone in the topic.
'wildcard' refers to 'stream_wildcard' and 'topic_wildcard' as a whole.
2023-07-03 22:03:17 -07:00
Prakhar Pratyush d80779435a tests: Add the missing tests.
This commit adds the missing tests for
'followed_topic_wildcard_mention'.

These tests should have been included in
b052c8980e.
2023-07-03 22:03:17 -07:00
Prakhar Pratyush 0bf6eb6786 notifications: Fix 'get_gcm_alert' and 'get_apns_alert_subtitle'.
The 'get_gcm_alert' and 'get_apns_alert_subtitle' functions
don't include the case when the trigger is
'NotificationTriggers.FOLLOWED_TOPIC_WILDCARD_MENTION'.

This commit updates the functions to include
'NotificationTriggers.FOLLOWED_TOPIC_WILDCARD_MENTION'.
2023-07-03 22:03:17 -07:00
Prakhar Pratyush 5f6dd83696 notifications: Fix the if/elif order in the manage_preferences block.
The emails sent for missed messages have a text at the bottom
explaining the reason why the email was sent.

This commit reorders the conditional statements in the email
template to align with the trigger priority order defined
in the 'get_email_notification_trigger'.
2023-07-03 22:03:17 -07:00
Prakhar Pratyush f7e41499fe notifications: Fix the 'senders' list.
This commit fixes the incorrect calculation of the
'senders' list.

The effect of 'followed_topic_wildcard_mention'
wasn't considered earlier.

The bug was introduced in b052c8980e.
2023-07-03 22:03:17 -07:00
Prakhar Pratyush bf673546cb notifications: Eliminate the stale context variable 'mention_count'.
This commit removes the context variable 'mention_count',
which is not being used anywhere.
2023-07-03 22:03:17 -07:00
Prakhar Pratyush e12cd9862a notification_triggers: Reduce loose string usage.
This commit uses 'NotificationTriggers' class attributes
instead of directly using loose strings.

This should have been ideally included in the commit
c3319a5231.
2023-07-03 22:03:17 -07:00
Alex Vandiver e2847790b6 upload: Provide a default upload file name, rather than 500. 2023-07-03 21:51:58 -07:00
Alex Vandiver 8a77cca341 middleware: Detect reverse proxy misconfigurations.
Combine nginx and Django middlware to stop putting misleading warnings
about `CSRF_TRUSTED_ORIGINS` when the issue is untrusted proxies.
This attempts to, in the error logs, diagnose and suggest next steps
to fix common proxy misconfigurations.

See also #24599 and zulip/docker-zulip#403.
2023-07-02 16:20:21 -07:00
Alex Vandiver cf0b803d50 zproject: Prevent having exactly 17/18 middlewares, for Python 3.11 bug.
Having exactly 17 or 18 middlewares, on Python 3.11.0 and above,
causes python to segfault when running tests with coverage; see
https://github.com/python/cpython/issues/106092

Work around this by adding one or two no-op middlewares if we would
hit those unlucky numbers.  We only add them in testing, since
coverage is a requirement to trigger it, and there is no reason to
burden production with additional wrapping.
2023-07-02 16:20:21 -07:00
Steve Howell af11ddb3cf doc tests: Mock html_to_text.
This shaves a couple seconds off an expensive test.
2023-07-02 16:18:24 -07:00
Steve Howell f7614e7109 url tests: Mock out html_to_text.
This makes the test about 3s faster, and we actually do
more meaningful checking than before.
2023-07-02 16:18:24 -07:00
Steve Howell c94bbfbc7a url tests: Split out expensive test for help pages.
It takes about 31ms per page on my box, but 191
help pages adds up quickly. I am not sure how to
optimize this test, but it will be a good litmus
test for a future better markdown processor.
2023-07-02 16:18:24 -07:00
Steve Howell c5ea79b9a4 doc tests: Create dedicated zephyr test.
This did not speed up the tests as much as I expected,
but it certainly makes the code easier to read, and
Tim is pretty confident that the zephyr logic is
fairly stable, so it's sufficient to test it on a
subset of representative urls.
2023-07-02 16:18:24 -07:00
Steve Howell ae9303ab3f doc tests: Remove landing_missing_strings.
dbe930394f changed the
"missing string" from "Log in" to "xyz" for some
unknown reason.  The current code makes no sense.

Also, even the original test code here had the common
pitfall of only testing one side of the condition.
Presumably if you are testing that a certain string
is missing in a landing-page scenario, then you also
want to check that it **does** exist in other
scenarios.  Otherwise, the flag would have been
named something more generic. Of course, I am mostly
guessing due to lack of comments.

If there is some test logic here that we need to
resurrect, then we should just write a custom test
for the /hello page rather than crufting up
all our helpers.
2023-07-02 16:18:24 -07:00
Steve Howell 9890f63153 doc tests: Clean up code related to robots.
This removes some confusing default boolean flags, and
it checks both sides of the do-you-want-to-allow-robots
condition, so it's more thorough.

For the two strange exceptions to the normal policy,
I now handle them together in the helper function with
a comment.

I also disentangle the logic to look for og tags from
the robot logic, and this should also lead to more
thorough testing.
2023-07-02 16:18:24 -07:00
Steve Howell 63f561d448 doc tests: Rename test for integration 404s.
The prior name was just strange.  This test could really
use a better comment explaining its purpose.

Also, presumably these pages don't always get 404s, so
we should really have the test exercise both conditions.
2023-07-02 16:18:24 -07:00
Steve Howell 447d9ebd1d doc tests: Handle expected strings more nicely.
This shaves a few seconds off the running time,
since we now consolidate content checks in one
of our loops.
2023-07-02 16:18:24 -07:00
Steve Howell 018634bbfd doc tests: Introoduce _is_landing_page helper.
This makes us correctly run landing page logic where we
didn't before, and, more importantly, lets us skip landing
page logic where we had been erroneously running it.

This speeds up my runs from 35s to 25s.
2023-07-02 16:18:24 -07:00
Sahil Batra 95f8ab1626 templates: Improve email confirmation page.
This commit updates the text on email confirmation page to
make it more clear what's going on and why the user needs
to check their email.

Fixes #25900.
2023-07-02 16:14:41 -07:00
Sahil Batra 6b2ca03174 user_groups: Add support to update can_mention_group setting.
This commit adds API support to update can_mention_group setting
of a user group.

Fixes a part of #25927.
2023-06-30 17:28:33 -07:00
Sahil Batra 4bea6ffaa8 user_groups: Add support to set can_mention_group during creation.
This commit adds API support to set can_mention_group while
creating a user group.

Fixes a part of #25927.
2023-06-30 17:28:33 -07:00
Sahil Batra e6accb0ad9 user_groups: Add can_mention_group_id field to UserGroup objects.
This commit adds code to include can_mention_group_id field to
UserGroup objects passed with response of various endpoints
including "/register" endpoint and also in the group object
send with user group creation event.

Fixes a part of #25927.
2023-06-30 17:28:33 -07:00
Sahil Batra 7aaf34fd7e message_edit: Check group mention permission when editing message.
This commit adds backend code to check whether a user is allowed
to mention a user group while editing a message as per
can_mention_group setting of that group.

Fixes a part of #25927.
2023-06-30 17:28:33 -07:00
Sahil Batra 1fdffaec73 message_send: Check group mention permission when sending message.
This commit adds backend code to check whether user has permission
to mention a group while sending message as per the can_mention_group
setting of the group.

Fixes a part of #25927.
2023-06-30 17:28:33 -07:00
Sahil Batra 2763f9b575 user_groups: Add can_mention_group setting.
This commit adds a new can_mention_group setting which will be
used to determine who can mention a particular group.

Fixes a part of #25927.
2023-06-30 17:28:33 -07:00
Steve Howell c4d8f501d6 narrow: Split out narrow_helpers.
This will make more sense as I get deeper into modernizing
how we accept narrows from users via the API and represent
the narrows in event queues.
2023-06-30 11:26:23 -07:00
Steve Howell 6d57340d48 narrow: Pass modern narrow to do_events_register.
We now upstream the conversion of legacy tuples
into the callers of do_events_register. For the
codepath that builds the home view, this allows
for cleaner code in the caller.  For the /register
endpoint, we have to do the conversion, but that
isn't super ugly, as that's an appropriate place
to deal with legacy formats and clean them up.

We do have to have do_events_register downgrade
the format back to tuples to pass them into
request_event_queue, because I don't want to
change any serialization formats. The conversion
is quite simple, and it has test coverage.
2023-06-30 11:26:23 -07:00
Steve Howell c501621bb2 event tests: Make sure legacy narrow is passed along. 2023-06-30 11:26:23 -07:00
Steve Howell c38b72c014 narrow: Rename *narrow_filter to *narrow_predicate.
This is consistent with how we name similar functions on
the frontend, and "filter" is misleading when you are
not dealing with lists.
2023-06-30 11:26:23 -07:00
Steve Howell 7cb4c0bdfe narrow: Rename function to check_narrow_for_events.
`supported` is kind of implied, and `filter` is not
really precise.
2023-06-30 11:26:23 -07:00
Steve Howell 6a2b560b7f narrow (mypy): Add NarrowPredicate protocol.
(The next commit will fix some related naming things.)
2023-06-30 11:26:23 -07:00
Satyam Bansal 8e63c0e42c integrations: Remove label URL from GitHub issue labeled notifications.
The URL is not the correct one and redirects to an API page.
2023-06-29 13:17:30 -07:00
Tim Abbott b47ba02b02 narrow: Remove NamedArg usage.
mypy_extensions is not a production dependency, so this broke the
production build.
2023-06-29 13:16:01 -07:00
Steve Howell bee7166a86 doc tests: Improve check for number of /api endpoints.
I also add a better comment.
2023-06-29 12:40:23 -07:00
Steve Howell 0163da66eb doc tests: Eliminate redundant test.
We now check content for /api endpoints in the big test
that loops over every endpoint.
2023-06-29 12:40:23 -07:00
Steve Howell 6c23e4769d doc tests: Reduce work for test_api_doc_endpoints.
We eliminate 220 zephyr-related checks that are all fairly
expensive.

On my machine this test went from 46s to 23s.

Note that we still get coverage of the zephyr codepath
from other tests.
2023-06-29 12:40:23 -07:00
Steve Howell b196d5a2f0 doc tests: Test endpoints in sorted order.
I also distinguish between endpoints and urls a little
better.
2023-06-29 12:40:23 -07:00
Steve Howell 66a5c7cf4a doc tests: Split out zephyr helper.
(All the same code gets executed here, but in a slightly
different order.)

There is some code duplication between the two new
helper functions, but I didn't make the situation any
worse, and it's slightly non-trivial to consolidate
the logic. Hopefully the long term strategy is to remove
the zephyr checks or at least isolate a single test for
any specific zephyr quirks that we need to maintain.
2023-06-29 12:40:23 -07:00
Steve Howell 20583ec1d1 doc tests: Split out a couple tests.
There was no reason for this already extremely expensive
test to have two extra responsibilities.
2023-06-29 12:40:23 -07:00
Steve Howell cea5e67262 narrows: Use dataclasses in a couple internal functions.
This is a first step toward two goals:
    * support dictionary-like narrows when registering events
    * use readable dataclasses internally

This is gonna be a somewhat complicated exercise due to how
events get serialized, but fortunately this interim step
doesn't require any serious shims, so it improves the codebase
even if the long-term goals may take a while to get sorted
out.

The two places where we have to use a helper to convert narrows
from tuples to dataclasses will eventually rely on their callers
to do the conversion, but I don't want to re-work the entire
codepath yet.

Note that the new NarrowTerm dataclass makes it more explicit
that the internal functions currently either don't care about
negated flags or downright don't support them.  This way mypy
protects us from assuming that we can just add negated support
at the outer edges.

OTOH I do make a tiny effort here to slightly restructure
narrow_filter in a way that paves the way for negation support.

The bigger goal by far, though, is to at least support the
dictionary format.
2023-06-29 12:35:55 -07:00
Steve Howell d64d1c81a4 tests: Eliminate narrow.json fixture.
In 2484d870b4 I created tests
using a fixture called narrow.json.  I believe my intention
was to eventually use the fixture for similar tests on the
frontend, but that never happened.

Almost seven years later, I think it's time to just use
straightforward code in Python to test build_narrow_filter.
In particular, we want to move to dataclasses, so that would
create an addition nuisance for fixture-based tests.  The
fixture was already annoying in terms of being an extra moving
part, being hard to read, and not being type-safe.

In order to avoid typos, I mostly code-generated the new
Python code by instrumenting the old test:

                 narrow_filter = build_narrow_filter(narrow)
    +            print("###\n")
    +            print(f"narrow_filter = build_narrow_filter({narrow})\n")
                 for e in accept_events:
                     message = e["message"]
                     flags = e["flags"]
    @@ -610,6 +612,8 @@ class NarrowLibraryTest(ZulipTestCase):
                     if flags is None:
                         flags = []
                     self.assertTrue(narrow_filter(message=message, flags=flags))
    +                print(f"self.assertTrue(narrow_filter(message={message}, flags={flags},))")
    +            print()
                 for e in reject_events:
                     message = e["message"]
                     flags = e["flags"]
    @@ -618,6 +622,8 @@ class NarrowLibraryTest(ZulipTestCase):
                     if flags is None:
                         flags = []
                     self.assertFalse(narrow_filter(message=message, flags=flags))
    +                print(f"self.assertFalse(narrow_filter(message={message}, flags={flags},))")
    +            print()

I then basically pasted the output in and ran black to format it.
2023-06-29 12:35:55 -07:00
Steve Howell 8ea0c5bbad narrow_filter: Pass message/flags to narrow_filter.
We no longer pass in a big opaque event to narrow_filter
(which is inside build_narrow_filter). We instead explicitly
pass in message and flags. This leads to a bit more type
safety, and it's also more flexible. There's no reason to
build an entire event just to see if a message belongs to
a narrow.

The changes to the test work around the fact that the fixtures
are sloppy with types. I plan a subsequent commit to clean
up those tests significantly.
2023-06-29 12:35:55 -07:00
Ujjawal Modi a361c23aac alert_words: Refactor the code to flush alert_words cache.
Subsequent commits will add "on_delete=models.RESTRICT"
relationships, which will result in the AlertWord
objects being deleted after Realm has been deleted from
the database.

In order to handle this, we update realm_alert_words_cache_key,
realm_alert_words_automaton_cache_key, and flush_realm_alert_words
functions to accept realm_id as parameter instead of realm
object, so that  the code for flushing the cache works even
after the realm is deleted. This change is fine because
eventually only realm_id is used by these functions and there
is no need of the complete realm object.
2023-06-28 18:03:32 -07:00
Ujjawal Modi f7346f36fc attachments: Refactor code for flushing used_upload_space cache.
Subsequent commits will add "on_delete=models.RESTRICT"
relationships, which will result in the Attachment
objects being deleted after Realm has been deleted from
the database.

In order to handle this, we update
get_realm_used_upload_space_cache_key function to accept
realm_id as parameter instead of realm object, so that
the code for flushing the cache works even after the
realm is deleted. This change is fine because eventually
only realm_id is used by this function and there is no
need of the complete realm object.
2023-06-28 18:03:32 -07:00