Commit Graph

28 Commits

Author SHA1 Message Date
Anders Kaseorg 7dcffca50e tornado: Construct Django BaseHandler once, not per-request.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-30 11:20:45 -07:00
Anders Kaseorg 6fd1a558b7 runtornado: Switch to asyncio event loop.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg c263bfdb41 queue: Use a thread-local Pika connection.
According to the documentation: “Pika does not have any notion of
threading in the code. If you want to use Pika with threading, make
sure you have a Pika connection per thread, created in that thread. It
is not safe to share one Pika connection across threads, with one
exception: you may call the connection method add_callback_threadsafe
from another thread to schedule a callback within an active pika
connection.”

https://pika.readthedocs.io/en/stable/faq.html

This also means that synchronous Django code running in Tornado will
use its own synchronous SimpleQueueClient rather than sharing the
asynchronous TornadoQueueClient, which is unfortunate but necessary as
they’re about to be on different threads.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg bded7180f7 tornado: Unfork tornado.autoreload.
We previously forked tornado.autoreload to work around a problem where
it would crash if you introduce a syntax error and not recover if you
fix it (https://github.com/tornadoweb/tornado/issues/2398).

A much more maintainable workaround for that issue, at least in
current Tornado, is to use tornado.autoreload as the main module.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg 11741543da python: Reformat with Black, except quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 72d6ff3c3b docs: Fix more capitalization issues.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-23 11:46:55 -07:00
Alex Vandiver 5c806fbd52 tornado: Remove a misleading comment and reformat.
tornado.web.Application does not share any inheritance with Django at
all; it has a similar router interface, but tornado.web.Application is
not an instance of Django anything.

Refold the long lines that follow it.
2020-09-18 15:13:40 -07:00
Alex Vandiver 4354386e69 tornado: Remove an unused port argument.
This was added in ec065e92ee for the WebSocket codepath, which was
subsequently removed in ea6934c26d.
2020-09-18 15:13:40 -07:00
Anders Kaseorg 365fe0b3d5 python: Sort imports with isort.
Fixes #2665.

Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.

Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start.  I expect this change will increase pressure for us to split
those files, which isn't a bad thing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:45:32 -07:00
Anders Kaseorg ea6934c26d dependencies: Remove WebSockets system for sending messages.
Zulip has had a small use of WebSockets (specifically, for the code
path of sending messages, via the webapp only) since ~2013.  We
originally added this use of WebSockets in the hope that the latency
benefits of doing so would allow us to avoid implementing a markdown
local echo; they were not.  Further, HTTP/2 may have eliminated the
latency difference we hoped to exploit by using WebSockets in any
case.

While we’d originally imagined using WebSockets for other endpoints,
there was never a good justification for moving more components to the
WebSockets system.

This WebSockets code path had a lot of downsides/complexity,
including:

* The messy hack involving constructing an emulated request object to
  hook into doing Django requests.
* The `message_senders` queue processor system, which increases RAM
  needs and must be provisioned independently from the rest of the
  server).
* A duplicate check_send_receive_time Nagios test specific to
  WebSockets.
* The requirement for users to have their firewalls/NATs allow
  WebSocket connections, and a setting to disable them for networks
  where WebSockets don’t work.
* Dependencies on the SockJS family of libraries, which has at times
  been poorly maintained, and periodically throws random JavaScript
  exceptions in our production environments without a deep enough
  traceback to effectively investigate.
* A total of about 1600 lines of our code related to the feature.
* Increased load on the Tornado system, especially around a Zulip
  server restart, and especially for large installations like
  zulipchat.com, resulting in extra delay before messages can be sent
  again.

As detailed in
https://github.com/zulip/zulip/pull/12862#issuecomment-536152397, it
appears that removing WebSockets moderately increases the time it
takes for the `send_message` API query to return from the server, but
does not significantly change the time between when a message is sent
and when it is received by clients.  We don’t understand the reason
for that change (suggesting the possibility of a measurement error),
and even if it is a real change, we consider that potential small
latency regression to be acceptable.

If we later want WebSockets, we’ll likely want to just use Django
Channels.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-01-14 22:34:00 -08:00
Anders Kaseorg becef760bf cleanup: Delete leading newlines.
Previous cleanups (mostly the removals of Python __future__ imports)
were done in a way that introduced leading newlines.  Delete leading
newlines from all files, except static/assets/zulip-emoji/NOTICE,
which is a verbatim copy of the Apache 2.0 license.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-06 23:29:11 -07:00
Anders Kaseorg 3968fe9fdf tornado: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:33:13 -08:00
Tim Abbott ec065e92ee tornado: Store port on SockJS connection object.
This will make it available for use inside our websockets code.
2018-11-02 16:55:33 -07:00
Tim Abbott 02ae71f27f api: Stop using API keys for Django->Tornado authentication.
As part of our effort to change the data model away from each user
having a single API key, we're eliminating the couple requests that
were made from Django to Tornado (as part of a /register or home
request) where we used the user's API key grabbed from the database
for authentication.

Instead, we use the (already existing) internal_notify_view
authentication mechanism, which uses the SHARED_SECRET setting for
security, for these requests, and just fetch the user object using
get_user_profile_by_id directly.

Tweaked by Yago to include the new /api/v1/events/internal endpoint in
the exempt_patterns list in test_helpers, since it's an endpoint we call
through Tornado. Also added a couple missing return type annotations.
2018-07-30 12:28:31 -07:00
Tim Abbott 4f4d56b021 tornado: Import autoreload module from upstream Tornado.
This allows to patch things directly.
2018-05-20 16:49:17 -07:00
Tim Abbott 3872e5521e tornado: Mark setup_tornado_rabbitmq for nocoverage.
We only use this in the direct management command, and it involves
some autoreload process setup stuff that we probably don't want to do
in our unit tests regardless.
2018-05-09 20:49:13 -07:00
Tim Abbott dc8dd2333c tornado: Sort imports in files with no merge conflicts. 2017-11-15 15:53:11 -08:00
rht 1047733486 zerver/tornado: Use python 3 syntax for typing. 2017-10-26 21:58:22 -07:00
rht b8aa92194c zerver/tornado: Remove print_function. 2017-09-27 18:05:45 -07:00
rht 74fd3d9f31 zerver/tornado: Remove absolute_import. 2017-09-27 10:00:39 -07:00
Tim Abbott b8e7369dee mypy: Remove type: ignores not needed in Python 3. 2017-08-25 11:04:20 -07:00
Umair Khan b4d49720b0 tornado: Close queue connection on reload.
Tornado reloads the app whenever there is a change in code. Due to this,
new connection is created to the client which also results in a new
channel. To avoid creating two channels for the queue in the RabbitMQ
broker we should close the old channel. Otherwise messages sent to the
queue will be distributed among these two channels in a round robin
scheme and we will end up losing one message since one of the channels
doesn't have an active consumer.

This commit closes the connection to the queue whenever Tornado reloads
the application using add_reload_hook().

Fixes #5824.
2017-07-21 11:19:26 -07:00
Umair Khan 758dbec9e2 tornado: Move setup_tornado_rabbitmq to application.py 2017-07-21 09:55:25 +05:00
Tim Abbott 25488b550f casper: Fix issues with server autoreloading on save.
This fixes an issue where if you saved a Python file (even just
changing whitespace) while casper tests were running, the Tornado
server being used would restart, triggering a confusing error like
this:

ReferenceError: Can't find variable: $
Traceback:
  undefined:2
  :4
Suite explicitly interrupted without any message given.
2017-03-17 20:45:07 -07:00
Tim Abbott 22d1aa396b lint: Clean up W503 PEP-8 warning. 2017-01-23 20:50:04 -08:00
AZtheAsian f7582f0050 pep8: Fix E124 violations 2016-12-01 14:18:11 -08:00
Tim Abbott 282f74609c tornado: Move socket code to zerver/tornado/. 2016-11-26 22:29:27 -08:00
Tim Abbott b83361bcfc tornado: Extract create_tornado_application helper. 2016-11-26 21:43:13 -08:00