Go to file
Tim Abbott 1ea2f188ce tornado: Rewrite Django integration to duplicate less code.
Since essentially the first use of Tornado in Zulip, we've been
maintaining our Tornado+Django system, AsyncDjangoHandler, with
several hundred lines of Django code copied into it.

The goal for that code was simple: We wanted a way to use our Django
middleware (for code sharing reasons) inside a Tornado process (since
we wanted to use Tornado for our async events system).

As part of the Django 2.2.x upgrade, I looked at upgrading this
implementation to be based off modern Django, and it's definitely
possible to do that:
* Continue forking load_middleware to save response middleware.
* Continue manually running the Django response middleware.
* Continue working out a hack involving copying all of _get_response
  to change a couple lines allowing us our Tornado code to not
  actually return the Django HttpResponse so we can long-poll.  The
  previous hack of returning None stopped being viable with the Django 2.2
  MiddlewareMixin.__call__ implementation.

But I decided to take this opportunity to look at trying to avoid
copying material Django code, and there is a way to do it:

* Replace RespondAsynchronously with a response.asynchronous attribute
  on the HttpResponse; this allows Django to run its normal plumbing
  happily in a way that should be stable over time, and then we
  proceed to discard the response inside the Tornado `get()` method to
  implement long-polling.  (Better yet might be raising an
  exception?).  This lets us eliminate maintaining a patched copy of
  _get_response.

* Removing the @asynchronous decorator, which didn't add anything now
  that we only have one API endpoint backend (with two frontend call
  points) that could call into this.  Combined with the last bullet,
  this lets us remove a significant hack from our
  never_cache_responses function.

* Calling the normal Django `get_response` method from zulip_finish
  after creating a duplicate request to process, rather than writing
  totally custom code to do that.  This lets us eliminate maintaining
  a patched copy of Django's load_middleware.

* Adding detailed comments explaining how this is supposed to work,
  what problems we encounter, and how we solve various problems, which
  is critical to being able to modify this code in the future.

A key advantage of these changes is that the exact same code should
work on Django 1.11, Django 2.2, and Django 3.x, because we're no
longer copying large blocks of core Django code and thus should be
much less vulnerable to refactors.

There may be a modest performance downside, in that we now run both
request and response middleware twice when longpolling (once for the
request we discard).  We may be able to avoid the expensive part of
it, Zulip's own request/response middleware, with a bit of additional
custom code to save work for requests where we're planning to discard
the response.  Profiling will be important to understanding what's
worth doing here.
2020-02-13 16:13:11 -08:00
.circleci circleci: Store XUnit test results. 2019-07-07 22:31:11 -07:00
.github github: Suggest GIFs too in PR template. 2018-02-16 09:59:22 -08:00
.tx cleanup: Delete trailing newlines. 2019-08-06 23:29:11 -07:00
analytics analytics: Added comments. 2020-01-28 14:57:32 -08:00
confirmation confirmation: Set confirmation object realm attribute in realm reactivation. 2019-10-21 16:52:46 -07:00
corporate python: Sort migrations/management command imports with isort. 2020-01-14 13:07:47 -08:00
docs pygments_data: Replace JS module with JSON module. 2020-02-12 10:09:12 -08:00
frontend_tests channel: Discard all HTTP responses while reloading. 2020-02-13 15:45:39 -08:00
locale i18n: Update translation data from transifex. 2019-12-12 20:34:46 -08:00
pgroonga python: Sort migrations/management command imports with isort. 2020-01-14 13:07:47 -08:00
puppet queue: Remove missedmessage_email_senders. 2020-01-31 12:13:51 -08:00
requirements requirements: Ask social-auth to pull in its own reqs for Azure, SAML. 2020-02-06 15:00:13 -08:00
scripts install: Don't create internal realm in the installation process. 2020-02-12 12:00:10 -08:00
static channel: Don't send outgoing HTTP requests during a reload. 2020-02-13 15:45:39 -08:00
stubs mypy: Remove daemon mode. 2019-08-25 15:04:12 -07:00
templates docs: Advertise support for GitLab authentication. 2020-02-11 14:13:39 -08:00
tools channel: Discard all HTTP responses while reloading. 2020-02-13 15:45:39 -08:00
zerver tornado: Rewrite Django integration to duplicate less code. 2020-02-13 16:13:11 -08:00
zilencer presence: Add realm_id to UserPresence. 2020-02-10 17:21:45 -08:00
zproject tornado: Rewrite Django integration to duplicate less code. 2020-02-13 16:13:11 -08:00
zthumbor zthumbor: Clean up type ignores. 2019-08-09 17:42:33 -07:00
.browserslistrc webpack: Transpile JS code with Babel. 2019-07-22 17:55:32 -07:00
.codecov.yml codecov: Change threshold to use percentage syntax. 2019-07-20 14:37:04 -07:00
.editorconfig editorconfig: Set JS max_line_length = 100, to match eslintrc. 2019-10-14 17:32:38 -07:00
.eslintignore blueslip: Apply ESLint. 2019-11-01 12:13:59 -07:00
.eslintrc.json pygments_data: Replace JS module with JSON module. 2020-02-12 10:09:12 -08:00
.gitattributes Revert "gitattributes: Mark yarn.lock as "binary", i.e. suppress diffs." 2019-05-20 19:31:14 -07:00
.gitignore i18n: Move static/locale back to locale. 2019-07-02 14:57:55 -07:00
.gitlint lint: Allow revert commit messages in gitlint. 2018-02-13 09:21:01 -08:00
.isort.cfg tornado: Fix logging of tornado activity level. 2018-04-17 15:59:01 -07:00
.npmignore Add proxy notes to new README.dev.md troubleshooting section. 2016-03-29 21:54:05 -07:00
.stylelintrc lint: Ban color names in CSS. 2019-01-22 15:33:18 -08:00
.travis.yml ci: Move backend and production tests to Ubuntu 16.04 (xenial). 2019-05-24 17:07:15 -07:00
.yarnrc .yarnrc: Set ignore-scripts true. 2019-08-28 16:15:54 -07:00
CODE_OF_CONDUCT.md docs: Add clarifying comma in CODE_OF_CONDUCT.md. 2019-04-05 18:01:37 -07:00
CONTRIBUTING.md docs: Remove mention of Dropbox CLA. 2019-12-14 22:07:38 -08:00
Dockerfile-postgresql base Zulip PostgreSQL Docker container on PGroonga official one 2019-12-30 10:20:25 -08:00
LICENSE license: Move license application notice from LICENSE to NOTICE. 2018-10-02 12:04:44 -07:00
NOTICE license: Move license application notice from LICENSE to NOTICE. 2018-10-02 12:04:44 -07:00
README.md README: Improve links to coverage/CI to point to master. 2019-10-01 15:31:55 -07:00
Vagrantfile Revert "vagrant: Add NFS backend for file synchronization for OSX." 2019-08-12 16:04:00 -07:00
babel.config.js babel: Enable loose mode. 2020-02-05 11:52:52 -08:00
manage.py manage.py: Revert sabotaging pika.adapters.twisted_connection import. 2019-01-31 10:04:28 -08:00
mypy.ini tornado: Rewrite Django integration to duplicate less code. 2020-02-13 16:13:11 -08:00
package.json package.json: Set "private": true to prevent NPM publication. 2020-02-09 22:02:49 -08:00
postcss.config.js webpack: Move CSS minification to optimization stage. 2019-09-02 21:58:13 -07:00
tsconfig.json tsconfig: Fix typescript-eslint memory usage disaster. 2019-11-22 11:38:25 -08:00
version.py pygments_data: Replace JS module with JSON module. 2020-02-12 10:09:12 -08:00
yarn.lock dependencies: Upgrade all JavaScript dependencies. 2020-02-04 22:13:47 -08:00

README.md

Zulip overview

Zulip is a powerful, open source group chat application that combines the immediacy of real-time chat with the productivity benefits of threaded conversations. Zulip is used by open source projects, Fortune 500 companies, large standards bodies, and others who need a real-time chat system that allows users to easily process hundreds or thousands of messages a day. With over 500 contributors merging over 500 commits a month, Zulip is also the largest and fastest growing open source group chat project.

CircleCI branch Coverage Status Mypy coverage GitHub release docs Zulip chat Twitter

Getting started

Click on the appropriate link below. If nothing seems to apply, join us on the Zulip community server and tell us what's up!

You might be interested in:

You may also be interested in reading our blog or following us on twitter. Zulip is distributed under the Apache 2.0 license.