Commit Graph

47551 Commits

Author SHA1 Message Date
Alex Vandiver 4fd51cb5ad uwsgi: Increase request buffer size to 64k, from 8k default.
The default value in uwsgi is 4k; receiving more than this amount from
nginx leads to a 502 response (though, happily, the backend uwsgi does not
terminate).

ab18dbfde5 originally increased it from the unstated uwsgi default
of 4096, to 8192; b1da797955 made it configurable, in order to allow
requests from clients with many cookies, without causing 502's[1].

nginx defaults to a limitation of 1k, with 4 additional 8k header
lines allowed[2]; any request larger than that returns a response of
`400 Request Header Or Cookie Too Large`.  The largest header size
theoretically possible from nginx, by default, is thus 33k, though
that would require packing four separate headers to exactly 8k each.

Remove the gap between nginx's limit and uwsgi's, which could trigger
502s, by removing the uwsgi configurability, and setting a 64k size in
uwsgi (the max allowable), which is larger than nginx's default limit.

uWSGI's documentation of `buffer-size` ([3], [4]) also notes that "It
is a security measure too, so adapt to your app needs instead of
maxing it out."  Python has no security issues with buffers of 64k,
and there is no appreciable memory footprint difference to having a
larger buffer available in uwsgi.

[1]: https://chat.zulip.org/#narrow/stream/31-production-help/topic/works.20in.20Edge.20not.20Chrome/near/719523
[2]: https://nginx.org/en/docs/http/ngx_http_core_module.html#client_header_buffer_size
[3]: https://uwsgi-docs.readthedocs.io/en/latest/ThingsToKnow.html
[4]: https://uwsgi-docs.readthedocs.io/en/latest/Options.html#buffer-size
2022-06-28 16:14:24 -07:00
Zixuan James Li b65401ed47 tests: Tighten signature of the wrapped test client helpers.
We wrap methods of the django test client for the test suite, and
type keyword variadic arguments as `ClientArg` as it might called
with a mix of `bool` and `str`.

This is problematic when we call the original methods on the test
client as we attempt to unpack the dictionary of keyword arguments,
which has no type guarantee that certain keys that the test client
requires to be bool will certainly be bool.

For example, you can call
`self.client_post(url, info, follow="invalid")` without getting a
mypy error while the django test client requires `follow: bool`.

The unsafely typed keyword variadic arguments leads to error within
the body the wrapped test client functions as we call
`django_client.post` with `**kwargs` when django-stubs gets added,
making it necessary to refactor these wrappers for type safety.

The approach here minimizes the need to refactor callers, as we
keep `kwargs` being variadic while change its type from `ClientArg`
to `str` after defining all the possible `bool` arguments that might
previously appear in `kwargs`. We also copy the defaults from the
django test client as they are unlikely to change.

The tornado test cases are also refactored due to the change of
the signature of `set_http_headers` with the `skip_user_agent` being
added as a keyword argument. We want to unconditionally set this flag to
`True` because the `HTTP_USER_AGENT` is not supported. It also removes a
unnecessary duplication of an argument.

This is a part of the django-stubs refactorings.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:12:00 -07:00
Zixuan James Li 8a9f06d5bc test_stripe: Remove realm from upgrade.
This is an unused argument. We removed it so that we don't
need to create a `TypedDict` and unpack it when calling
the test client methods.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:12:00 -07:00
Zixuan James Li 4e518a3852 tests: Call client methods with explicit keyword arguments.
This is a prep commit for tightening the types for our wrapped test
client.

The callers of the test client methods are refactored to either call
them without unpacking at all or create a TypedDict for the keyword
arguments to be unpacked. This allows the type checker to know exactly what
keys are present and their corresponding type.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:12:00 -07:00
Zixuan James Li a3a0545aac typing: Populate POST and _files with `cast` and `setattr`.
`POST` is an immutable attribute and `_files` is an internal attribute
of `HttpRequest`. With type annotations provided by `django-stubs`, mypy
stops us from modifying these attributes. This uses `cast` and `setattr`
to avoid typing issues.

This is a part of django-stubs refactorings.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:07:18 -07:00
Zixuan James Li 27be27560b typing: Access LANGUAGE_CODE via `django.utils.translation.get_language`.
We no longer need to access the internal `LANGUAGE_CODE` attribute by
using `django.utils.translation.get_language`.

A test case overriding the translation is added to ensure the password
reset form sending to users requested from a wrong domain is properly
translated.

This is a part of django-stubs refactorings.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:07:18 -07:00
Zixuan James Li 417e1b5e81 logging_util: Save an assertion with __getitem__.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:05:24 -07:00
Zixuan James Li fc3de0ba41 typing: Tighten type annotation for serialized user groups.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:05:24 -07:00
Zixuan James Li 5285fbb4d0 typing: Fix type annotation for missing messages in soft reactivation.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:05:24 -07:00
Zixuan James Li 9bfeebf064 user_profile: Fallback to "" for timezone upon creation.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:05:24 -07:00
Zixuan James Li a9029c68ea tornado: Make _request an attribute on AsyncDjangoHandler.
For the same reason as `handler_id` has, we define `_request`
as an attribute. Note that the name `request` is already taken.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:03:09 -07:00
Zixuan James Li 56e1f3b725 tornado: Make handler_id an attribute on AsyncDjangoHandler.
This prevents us from relying on a side-effect of `allocate_handler_id`
that monkey-patches `handler_id` on the `AsyncDjangoHandler` object,
allowing mypy to acknowledge the existence of `handler_id` as an `int`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:03:09 -07:00
Zixuan James Li f0e505d557 retention: Guarantee type-safeness when calling move_rows.
This ensures that all the keyword arguments in `move_rows`
have the correct types. Note that `returning_id` is supposed to be a
flag instead of a `Composable` `Literal`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:01:35 -07:00
Alex Vandiver 2b02722d16 log-search: Add a filter to exclude all lines not explicitly wanted. 2022-06-28 15:59:31 -07:00
Alex Vandiver 180565d8d6 log-search: Fix copy/paste-o in filtering for presence. 2022-06-28 15:59:31 -07:00
Gaurav Pandey 3aa9e4c553 recent_topics: Link timestamp to latest message.
Link the timestamp in recent topics to the
"near" link of the latest message within that topic.
Fixes #21356.
2022-06-28 13:31:27 -07:00
aparna-bhatt 1298484ac8
compose: Add keyboard shortcut tooltip to Drafts link..
This is for consistency with how we indicate keyboard shortcuts in
other elements in the compose area.

Fixes: #22131.
2022-06-28 13:25:54 -07:00
madrix01 329f52ae82 compose: Make compose box tooltips consitent.
We make the tooltips of the button in the top row consistent.
This includes -
* Removing the bold formatting from the "Go to conversation" tooltip.
* Converting the "Expand compose" tooltip to Tippy.

Fixes #22132
2022-06-28 13:24:13 -07:00
Tim Abbott d6739078ad message_edit: Change initial default for old thread notifications.
After some discussion with the community, we've concluded that in most
cases, one probably wants to notify only the destination thread, not
the source location; so let's make that the default.

We preserve the model that we remember what you did during the last
move in a given browser session, which is very helpful if you're doing
an operation N times in a row.

Fixes #21838.
2022-06-28 13:23:34 -07:00
Raghav Luthra 890de402df stream_settings: Modify the message of the "Announce stream" hint tooltip.
Follow-up to commit bbda7a5bb0.

The "Announce stream" hint tooltip earlier read "Organization
administrators can change this in in the organization settings."
It wasn't obvious that "this" refers to the stream the notification
will go to, so the tootltip message has been modified to reflect this.
2022-06-28 13:19:54 -07:00
Advait J 6ceb7b5462 streams: Enable or disable subscribers tab based on access to subscribers.
This commit adds a function to disable the subscribers tab for private
streams if a user is not subscribed to the stream and is not an admin.
We also live update the state of subscribers tab on changing privacy
of stream.

Fixes #20916.

Co-authored-by: Sahil Batra <sahil@zulip.com>
2022-06-28 13:19:26 -07:00
Anders Kaseorg 251198044f docs: Update WIP PR convention to GitHub draft PRs.
Since GitHub supports draft PRs, our [WIP] convention is obsolete.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-28 13:16:38 -07:00
Anders Kaseorg bae4182e47 test_classes: Check responses in ZulipTestCase.register.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-28 13:15:21 -07:00
Harshit Agarwar 41e7d52c65
stream creation: Change the color of remove button in subscribers tab.
This button was red, which is appropriate when modifying an existing stream,
since that's a potentially disruptive action, but not appropriate in the context
of previewing subscribers for a new stream being created.

Fixes: #21863.
2022-06-28 13:11:35 -07:00
Alex Vandiver aabbb2ad10 bootstrap-awscli: Leave CWD unchanged. 2022-06-28 09:39:31 -04:00
Alex Vandiver 1b5e058d3b install-aws-server: Add a --debug-key to allow logins during bootstrap. 2022-06-28 09:39:31 -04:00
Alex Vandiver 1a1061e77e install-aws-server: Switch to getopt. 2022-06-28 09:39:31 -04:00
Alex Vandiver 1be9ab2690 install-aws-server: Assume zulip_ops::profile:: prefix on all roles.
This will require that any profile-specific sections of
`$HOME/.zulip-install-server.conf` be renamed to their short form.
2022-06-28 09:39:31 -04:00
Alex Vandiver 9a3be2af64 install-aws-server: Fix default_branch config name. 2022-06-28 09:39:31 -04:00
Alex Vandiver f5d4dea2f0 bootstrap-aws-installer: Leave `ubuntu` user deletion until the last.
This makes it easier to log in and figure out what went wrong if
something failed during the bootstrapping process.
2022-06-28 09:39:31 -04:00
Alex Vandiver f9b7b8e5d9 docs: Document that loadbalancer.ips can be CIDR ranges. 2022-06-27 17:41:38 -07:00
Anders Kaseorg ef3510fa6d nginx: Remove legacy X-XSS-Protection header.
Support for this header was removed in Chrome 78, Safari 15.4, and
Edge 17.  It was never supported in Firefox.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-27 17:38:18 -07:00
Anders Kaseorg 869fe60689 markdown: Parse included blocks in a new parser state.
This fixes inclusion of a multi-paragraph file into a list item.

Followup to commit dc33a0ae67 (#22315).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-27 17:37:00 -07:00
Anders Kaseorg a2e1d61172 integrations: Satisfy Python-Markdown’s archaic 4-space requirement.
Followup to commit dc33a0ae67 (#22315).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-27 17:36:17 -07:00
Lauryn Menard 4ce9799cc2 integrations-docs: Remove `zulip-config.md`.
The content in `zulip-config.md` was moved (in #22315) to
`change-zulip-config.file.md`. This removes the now out-of-date
reference to this macro in the documentation for writing
integrations and removes the file as it is no longer in use.
2022-06-27 17:23:26 -07:00
Lauryn Menard 7c4c01f2e6 help-docs: Fix keyboard shortcuts that use / reference Mac keys.
Fixes help center documentation that use or reference Mac keys
for keyboard shortcuts instead of non-Mac keys since these are
not updated for users with non-Mac keyboards, while the reverse
is true.
2022-06-27 14:57:10 -07:00
Lauryn Menard 57e3f7145f help-docs: Fix keyboard shortcuts with multiple keys.
Fixes existing errors in the help center documentation where a
keyboard shortcut that has multiple keys is all one inline code
tag (e.g. `Ctrl + [`) instead of being separated into individual
inline code tags for each key (e.g. `Ctrl` + `[`).
2022-06-27 14:57:10 -07:00
Anders Kaseorg 25f6b53e90 docs: Capitalize Handlebars consistently.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-27 13:35:10 -07:00
Anders Kaseorg 53231aa9d9 decorator: Type cache_info, cache_clear for ignore_unhashable_lru_cache.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-27 10:20:05 -07:00
Lauryn Menard 9db8a59a56 integrations-docs: Add line break in GitLab integration doc. 2022-06-27 10:14:51 -07:00
Anders Kaseorg a7e10ee47e tornado: Send request_started signal in Django thread.
Django’s ASGIHandler does this too and it seems like a good idea.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-27 10:14:02 -07:00
Anders Kaseorg d5c302feef run-dev: Remove unnecessary finish calls.
Tornado finishes the request automatically.  Avoids this error, hidden
until commit 81f7192ca3 (#22301), when
the browser tab is closed:

Traceback (most recent call last):
  File "/srv/zulip-py3-venv/lib/python3.8/site-packages/tornado/web.py", line 1683, in _execute
    result = await result
  File "tools/run-dev.py", line 280, in prepare
    await self.finish()
tornado.iostream.StreamClosedError: Stream is closed

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:38:43 -07:00
Anders Kaseorg 0dfde9db94 python: Simplify unnecessary getattr calls.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:38:22 -07:00
Anders Kaseorg 3bf8ee2156 python: Unquote some unnecessarily quoted type annotations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:37:41 -07:00
Anders Kaseorg 6fe5cb65a1 templates: Fix Python-Markdown extension type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:37:14 -07:00
Anders Kaseorg dc33a0ae67 markdown: Rewrite include plugin without markdown-include.
markdown-include is GPL licensed.

Also, rewrite it as a block processor, so that it works correctly
inside indented blocks.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:36:31 -07:00
Anders Kaseorg 7f0e11bd06 markdown: Rename preprocessor_priorities module to priorities.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:36:31 -07:00
Anders Kaseorg e069330084 check-thirdparty: License under GPLv2+.
Since python-debian is GPL licensed, our script that imports it should
arguably be GPL licensed as well.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:33:11 -07:00
Anders Kaseorg f3254bb558 mattermost: Run html2text as a subprocess.
html2text is GPL licensed.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:32:59 -07:00
Anders Kaseorg e10b7f2f3c email_notifications: Run html2text as a subprocess.
html2text is GPL licensed.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:32:59 -07:00