Commit Graph

52445 Commits

Author SHA1 Message Date
Alex Vandiver 8a77cca341 middleware: Detect reverse proxy misconfigurations.
Combine nginx and Django middlware to stop putting misleading warnings
about `CSRF_TRUSTED_ORIGINS` when the issue is untrusted proxies.
This attempts to, in the error logs, diagnose and suggest next steps
to fix common proxy misconfigurations.

See also #24599 and zulip/docker-zulip#403.
2023-07-02 16:20:21 -07:00
Alex Vandiver cf0b803d50 zproject: Prevent having exactly 17/18 middlewares, for Python 3.11 bug.
Having exactly 17 or 18 middlewares, on Python 3.11.0 and above,
causes python to segfault when running tests with coverage; see
https://github.com/python/cpython/issues/106092

Work around this by adding one or two no-op middlewares if we would
hit those unlucky numbers.  We only add them in testing, since
coverage is a requirement to trigger it, and there is no reason to
burden production with additional wrapping.
2023-07-02 16:20:21 -07:00
Alex Vandiver 671b708c4b puppet: Remove loadbalancer configurations when they are unset. 2023-07-02 16:20:21 -07:00
Akshat e7f4700d76 test-backend: Enable single-threaded profiling when thread count is unspecified.
If the number of threads is not specified
while profiling then use a single thread. This is
because profiling across multiple threads (earlier
default behaviour) may obscure the accurate
measurement of which functions are the most costly
due to thread blocking.

Signed-off-by: Akshat <akshat25iiit@gmail.com>
2023-07-02 16:18:59 -07:00
Steve Howell af11ddb3cf doc tests: Mock html_to_text.
This shaves a couple seconds off an expensive test.
2023-07-02 16:18:24 -07:00
Steve Howell f7614e7109 url tests: Mock out html_to_text.
This makes the test about 3s faster, and we actually do
more meaningful checking than before.
2023-07-02 16:18:24 -07:00
Steve Howell c94bbfbc7a url tests: Split out expensive test for help pages.
It takes about 31ms per page on my box, but 191
help pages adds up quickly. I am not sure how to
optimize this test, but it will be a good litmus
test for a future better markdown processor.
2023-07-02 16:18:24 -07:00
Steve Howell c5ea79b9a4 doc tests: Create dedicated zephyr test.
This did not speed up the tests as much as I expected,
but it certainly makes the code easier to read, and
Tim is pretty confident that the zephyr logic is
fairly stable, so it's sufficient to test it on a
subset of representative urls.
2023-07-02 16:18:24 -07:00
Steve Howell ae9303ab3f doc tests: Remove landing_missing_strings.
dbe930394f changed the
"missing string" from "Log in" to "xyz" for some
unknown reason.  The current code makes no sense.

Also, even the original test code here had the common
pitfall of only testing one side of the condition.
Presumably if you are testing that a certain string
is missing in a landing-page scenario, then you also
want to check that it **does** exist in other
scenarios.  Otherwise, the flag would have been
named something more generic. Of course, I am mostly
guessing due to lack of comments.

If there is some test logic here that we need to
resurrect, then we should just write a custom test
for the /hello page rather than crufting up
all our helpers.
2023-07-02 16:18:24 -07:00
Steve Howell 9890f63153 doc tests: Clean up code related to robots.
This removes some confusing default boolean flags, and
it checks both sides of the do-you-want-to-allow-robots
condition, so it's more thorough.

For the two strange exceptions to the normal policy,
I now handle them together in the helper function with
a comment.

I also disentangle the logic to look for og tags from
the robot logic, and this should also lead to more
thorough testing.
2023-07-02 16:18:24 -07:00
Steve Howell 63f561d448 doc tests: Rename test for integration 404s.
The prior name was just strange.  This test could really
use a better comment explaining its purpose.

Also, presumably these pages don't always get 404s, so
we should really have the test exercise both conditions.
2023-07-02 16:18:24 -07:00
Steve Howell 447d9ebd1d doc tests: Handle expected strings more nicely.
This shaves a few seconds off the running time,
since we now consolidate content checks in one
of our loops.
2023-07-02 16:18:24 -07:00
Steve Howell 018634bbfd doc tests: Introoduce _is_landing_page helper.
This makes us correctly run landing page logic where we
didn't before, and, more importantly, lets us skip landing
page logic where we had been erroneously running it.

This speeds up my runs from 35s to 25s.
2023-07-02 16:18:24 -07:00
Anders Kaseorg 407c16fc77 eslint: Expand no-unused-vars check to all function parameters.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-02 16:16:38 -07:00
Sahil Batra 95f8ab1626 templates: Improve email confirmation page.
This commit updates the text on email confirmation page to
make it more clear what's going on and why the user needs
to check their email.

Fixes #25900.
2023-07-02 16:14:41 -07:00
Palash Baderia 987bbb641f message_feed: Fix edited notice to appear for disabled edit_history.
This commit fixes the bug where the edited notice is not displayed
for edited/moved messages when the edit_history is disabled for
the organization. However, this is incorrect because the edited
notice should still be shown for messages that are edited/moved,
even if the edit_history is disabled. This issue occurred because
the edit_history variable was not set in the message object when
the edit_history is disabled. Therefore, in this case, we check
for the availability of last_edit_timestr. If it is available, we
display the edited notice. Since we cannot determine if the message
was moved or edited, we show the edited notice for both cases.
2023-06-30 17:33:45 -07:00
Sahil Batra 632191b8c4 composebox_typeahead: Hide user groups from mention typeahead.
This commit adds code to hide the user groups which a user is
not allowed to mention from the mention typeahead.

Fixes a part of #25927.
2023-06-30 17:28:33 -07:00
Sahil Batra 767c2ebbe4 user_groups: Add can_mention_group_id field to user_group objects.
This commit adds can_mention_group_id field to user_group objects
in webapp.

Fixes a part of #25927.
2023-06-30 17:28:33 -07:00
Sahil Batra 6b2ca03174 user_groups: Add support to update can_mention_group setting.
This commit adds API support to update can_mention_group setting
of a user group.

Fixes a part of #25927.
2023-06-30 17:28:33 -07:00
Sahil Batra 4bea6ffaa8 user_groups: Add support to set can_mention_group during creation.
This commit adds API support to set can_mention_group while
creating a user group.

Fixes a part of #25927.
2023-06-30 17:28:33 -07:00
Sahil Batra e6accb0ad9 user_groups: Add can_mention_group_id field to UserGroup objects.
This commit adds code to include can_mention_group_id field to
UserGroup objects passed with response of various endpoints
including "/register" endpoint and also in the group object
send with user group creation event.

Fixes a part of #25927.
2023-06-30 17:28:33 -07:00
Sahil Batra 7aaf34fd7e message_edit: Check group mention permission when editing message.
This commit adds backend code to check whether a user is allowed
to mention a user group while editing a message as per
can_mention_group setting of that group.

Fixes a part of #25927.
2023-06-30 17:28:33 -07:00
Sahil Batra 1fdffaec73 message_send: Check group mention permission when sending message.
This commit adds backend code to check whether user has permission
to mention a group while sending message as per the can_mention_group
setting of the group.

Fixes a part of #25927.
2023-06-30 17:28:33 -07:00
Sahil Batra 2763f9b575 user_groups: Add can_mention_group setting.
This commit adds a new can_mention_group setting which will be
used to determine who can mention a particular group.

Fixes a part of #25927.
2023-06-30 17:28:33 -07:00
evykassirer 9265954fd2 search: Fix bug where textarea placeholder text didn't update for DMs.
When c0e9e463fb was merged, the
placeholder text started updating *before* we updated the recipient
information, so it wasn't updating with correct data. This commit
fixes that by moving the onPillCreate() call below the update.
2023-06-30 12:20:26 -07:00
evykassirer 367030ebfe search: Stop disabling and enabling the search close button.
This is logic from 10 years ago (dbc4798594)
that is no longer relevant. It seems like we used to show the
search bar open all the time and only showed the buttons when
there was focus in the search bar. Now we close the search bar
when it's not being used, and no longer need to update button
visibility or disable the search close button.
2023-06-30 12:05:30 -07:00
Steve Howell c4d8f501d6 narrow: Split out narrow_helpers.
This will make more sense as I get deeper into modernizing
how we accept narrows from users via the API and represent
the narrows in event queues.
2023-06-30 11:26:23 -07:00
Steve Howell 6d57340d48 narrow: Pass modern narrow to do_events_register.
We now upstream the conversion of legacy tuples
into the callers of do_events_register. For the
codepath that builds the home view, this allows
for cleaner code in the caller.  For the /register
endpoint, we have to do the conversion, but that
isn't super ugly, as that's an appropriate place
to deal with legacy formats and clean them up.

We do have to have do_events_register downgrade
the format back to tuples to pass them into
request_event_queue, because I don't want to
change any serialization formats. The conversion
is quite simple, and it has test coverage.
2023-06-30 11:26:23 -07:00
Steve Howell c501621bb2 event tests: Make sure legacy narrow is passed along. 2023-06-30 11:26:23 -07:00
Steve Howell c38b72c014 narrow: Rename *narrow_filter to *narrow_predicate.
This is consistent with how we name similar functions on
the frontend, and "filter" is misleading when you are
not dealing with lists.
2023-06-30 11:26:23 -07:00
Steve Howell 7cb4c0bdfe narrow: Rename function to check_narrow_for_events.
`supported` is kind of implied, and `filter` is not
really precise.
2023-06-30 11:26:23 -07:00
Steve Howell 6a2b560b7f narrow (mypy): Add NarrowPredicate protocol.
(The next commit will fix some related naming things.)
2023-06-30 11:26:23 -07:00
Anders Kaseorg a07bdb3fdb recent_topics_data: Simplify .entries() loop to .values().
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-06-29 15:54:40 -07:00
Satyam Bansal 8e63c0e42c integrations: Remove label URL from GitHub issue labeled notifications.
The URL is not the correct one and redirects to an API page.
2023-06-29 13:17:30 -07:00
Tim Abbott b47ba02b02 narrow: Remove NamedArg usage.
mypy_extensions is not a production dependency, so this broke the
production build.
2023-06-29 13:16:01 -07:00
Karl Stolley 7f4fe7282f search: Adjust text in has: message suggestions. 2023-06-29 13:02:06 -07:00
Steve Howell bee7166a86 doc tests: Improve check for number of /api endpoints.
I also add a better comment.
2023-06-29 12:40:23 -07:00
Steve Howell 0163da66eb doc tests: Eliminate redundant test.
We now check content for /api endpoints in the big test
that loops over every endpoint.
2023-06-29 12:40:23 -07:00
Steve Howell 6c23e4769d doc tests: Reduce work for test_api_doc_endpoints.
We eliminate 220 zephyr-related checks that are all fairly
expensive.

On my machine this test went from 46s to 23s.

Note that we still get coverage of the zephyr codepath
from other tests.
2023-06-29 12:40:23 -07:00
Steve Howell b196d5a2f0 doc tests: Test endpoints in sorted order.
I also distinguish between endpoints and urls a little
better.
2023-06-29 12:40:23 -07:00
Steve Howell 66a5c7cf4a doc tests: Split out zephyr helper.
(All the same code gets executed here, but in a slightly
different order.)

There is some code duplication between the two new
helper functions, but I didn't make the situation any
worse, and it's slightly non-trivial to consolidate
the logic. Hopefully the long term strategy is to remove
the zephyr checks or at least isolate a single test for
any specific zephyr quirks that we need to maintain.
2023-06-29 12:40:23 -07:00
Steve Howell 20583ec1d1 doc tests: Split out a couple tests.
There was no reason for this already extremely expensive
test to have two extra responsibilities.
2023-06-29 12:40:23 -07:00
Steve Howell cea5e67262 narrows: Use dataclasses in a couple internal functions.
This is a first step toward two goals:
    * support dictionary-like narrows when registering events
    * use readable dataclasses internally

This is gonna be a somewhat complicated exercise due to how
events get serialized, but fortunately this interim step
doesn't require any serious shims, so it improves the codebase
even if the long-term goals may take a while to get sorted
out.

The two places where we have to use a helper to convert narrows
from tuples to dataclasses will eventually rely on their callers
to do the conversion, but I don't want to re-work the entire
codepath yet.

Note that the new NarrowTerm dataclass makes it more explicit
that the internal functions currently either don't care about
negated flags or downright don't support them.  This way mypy
protects us from assuming that we can just add negated support
at the outer edges.

OTOH I do make a tiny effort here to slightly restructure
narrow_filter in a way that paves the way for negation support.

The bigger goal by far, though, is to at least support the
dictionary format.
2023-06-29 12:35:55 -07:00
Steve Howell d64d1c81a4 tests: Eliminate narrow.json fixture.
In 2484d870b4 I created tests
using a fixture called narrow.json.  I believe my intention
was to eventually use the fixture for similar tests on the
frontend, but that never happened.

Almost seven years later, I think it's time to just use
straightforward code in Python to test build_narrow_filter.
In particular, we want to move to dataclasses, so that would
create an addition nuisance for fixture-based tests.  The
fixture was already annoying in terms of being an extra moving
part, being hard to read, and not being type-safe.

In order to avoid typos, I mostly code-generated the new
Python code by instrumenting the old test:

                 narrow_filter = build_narrow_filter(narrow)
    +            print("###\n")
    +            print(f"narrow_filter = build_narrow_filter({narrow})\n")
                 for e in accept_events:
                     message = e["message"]
                     flags = e["flags"]
    @@ -610,6 +612,8 @@ class NarrowLibraryTest(ZulipTestCase):
                     if flags is None:
                         flags = []
                     self.assertTrue(narrow_filter(message=message, flags=flags))
    +                print(f"self.assertTrue(narrow_filter(message={message}, flags={flags},))")
    +            print()
                 for e in reject_events:
                     message = e["message"]
                     flags = e["flags"]
    @@ -618,6 +622,8 @@ class NarrowLibraryTest(ZulipTestCase):
                     if flags is None:
                         flags = []
                     self.assertFalse(narrow_filter(message=message, flags=flags))
    +                print(f"self.assertFalse(narrow_filter(message={message}, flags={flags},))")
    +            print()

I then basically pasted the output in and ran black to format it.
2023-06-29 12:35:55 -07:00
Steve Howell 8ea0c5bbad narrow_filter: Pass message/flags to narrow_filter.
We no longer pass in a big opaque event to narrow_filter
(which is inside build_narrow_filter). We instead explicitly
pass in message and flags. This leads to a bit more type
safety, and it's also more flexible. There's no reason to
build an entire event just to see if a message belongs to
a narrow.

The changes to the test work around the fact that the fixtures
are sloppy with types. I plan a subsequent commit to clean
up those tests significantly.
2023-06-29 12:35:55 -07:00
Karl Stolley 0c55fb7e89 stream_settings: Fix scroll at max-height.
Simplebar seems unaware of the `max-height: 1000px` on
`.subscriptions-container`, and therefore does not properly provide
a scrollbar when it's needed.

This commit adds a `max-height` to the stream Simplebar container,
ensuring that otherwise hidden content that Simplebar believes to
be visible can be scrolled to.

Finally, rather than rely on magic numbers or math done in comments,
this commit establishes CSS variables for all relevant modal-element
heights, doing the math inline using CSS calc().

Fixes #26107.
2023-06-29 11:19:01 -07:00
Lalit 1676d0b638 settings_emoji: Fix file upload bug in upload emoji modal.
Before this commit if a user pressed enter to submit the add emoji form
the uploaded emoji was getting cleared.

This was happening because when we pressed enter the browser tried to
submit the form, because we had two buttons inside that form one for
clearing uploaded emoji and one for uploading emoji they were being
treated as "submit" button and hence their callback ran which in result
was responsible for the bug.

Fixed this by explicitly setting the `type` attribute for those buttons
to `type=button` so that they will not be treated as "submit" button.
Also added the `form_id` option for `dialog_widget` which is needed if
we do want to correctly submit the form by pressing enter.

Fixes: #24972
2023-06-29 10:16:28 -07:00
Ujjawal Modi a361c23aac alert_words: Refactor the code to flush alert_words cache.
Subsequent commits will add "on_delete=models.RESTRICT"
relationships, which will result in the AlertWord
objects being deleted after Realm has been deleted from
the database.

In order to handle this, we update realm_alert_words_cache_key,
realm_alert_words_automaton_cache_key, and flush_realm_alert_words
functions to accept realm_id as parameter instead of realm
object, so that  the code for flushing the cache works even
after the realm is deleted. This change is fine because
eventually only realm_id is used by these functions and there
is no need of the complete realm object.
2023-06-28 18:03:32 -07:00
Ujjawal Modi f7346f36fc attachments: Refactor code for flushing used_upload_space cache.
Subsequent commits will add "on_delete=models.RESTRICT"
relationships, which will result in the Attachment
objects being deleted after Realm has been deleted from
the database.

In order to handle this, we update
get_realm_used_upload_space_cache_key function to accept
realm_id as parameter instead of realm object, so that
the code for flushing the cache works even after the
realm is deleted. This change is fine because eventually
only realm_id is used by this function and there is no
need of the complete realm object.
2023-06-28 18:03:32 -07:00
Ujjawal Modi 535a088d0b bots: Refactor code for flushing bots cache.
Subsequent commits will add "on_delete=models.RESTRICT"
relationships, which will result in the UserProfile
objects being deleted after Realm has been deleted from
the database.

In order to handle this, we update bot_dicts_in_realm_cache_key
function to accept realm_id as parameter instead of realm
object, so that  the code for flushing the cache works even
after the realm is deleted. This change is fine because
eventually only realm_id is used by this function and there is
no need of the complete realm object.
2023-06-28 18:03:32 -07:00