Zulip has had a small use of WebSockets (specifically, for the code
path of sending messages, via the webapp only) since ~2013. We
originally added this use of WebSockets in the hope that the latency
benefits of doing so would allow us to avoid implementing a markdown
local echo; they were not. Further, HTTP/2 may have eliminated the
latency difference we hoped to exploit by using WebSockets in any
case.
While we’d originally imagined using WebSockets for other endpoints,
there was never a good justification for moving more components to the
WebSockets system.
This WebSockets code path had a lot of downsides/complexity,
including:
* The messy hack involving constructing an emulated request object to
hook into doing Django requests.
* The `message_senders` queue processor system, which increases RAM
needs and must be provisioned independently from the rest of the
server).
* A duplicate check_send_receive_time Nagios test specific to
WebSockets.
* The requirement for users to have their firewalls/NATs allow
WebSocket connections, and a setting to disable them for networks
where WebSockets don’t work.
* Dependencies on the SockJS family of libraries, which has at times
been poorly maintained, and periodically throws random JavaScript
exceptions in our production environments without a deep enough
traceback to effectively investigate.
* A total of about 1600 lines of our code related to the feature.
* Increased load on the Tornado system, especially around a Zulip
server restart, and especially for large installations like
zulipchat.com, resulting in extra delay before messages can be sent
again.
As detailed in
https://github.com/zulip/zulip/pull/12862#issuecomment-536152397, it
appears that removing WebSockets moderately increases the time it
takes for the `send_message` API query to return from the server, but
does not significantly change the time between when a message is sent
and when it is received by clients. We don’t understand the reason
for that change (suggesting the possibility of a measurement error),
and even if it is a real change, we consider that potential small
latency regression to be acceptable.
If we later want WebSockets, we’ll likely want to just use Django
Channels.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Resolves these warnings from puppet-lint.
puppet-lint| puppet/zulip/manifests/app_frontend_base.pp - WARNING: double quoted string containing no variables on line 14
puppet-lint| puppet/zulip/manifests/app_frontend_base.pp - WARNING: double quoted string containing no variables on line 19
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
This has been a spurious alert for a long time.
It's unclear that this check is useful at all, but if it spikes
dramatically above what's normal, there's perhaps still utility in
being alerted.
Since LoopQueueProcessingWorker jobs cannot be monitored by checking
for connected consumers (since they poll, rather than consuming as
events arrive), they can't be monitored with check_consumers. It's
OK, because that monitoring was redundant with monitoring for
potential growth in their queue that we have as well.
Also clean up the block comments for the two other similar queue
procesors.
Now that we're implemented tsearch_extras in pure postgres, we no
longer need a custom extension. This should help us considerably, as
it means we no longer need to ship custom apt packages at all.
Fixes#467.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
This commit finishes adding end-to-end support for the install script
on Debian Buster (making it production ready). Some support for this
was already added in prior commits such as
99414e2d96.
We plan to revert the postgres hunks of this once we've built
tsearch_extras for our packagecloud archive.
Fixes#9828.
The issue here was that the '.' character was unescaped and the
regex was not anchored with a terminal '$'. This was detected by
Anders Kaseorg.
Co-authored-by: Anders Kaseorg <anders@zulipchat.com>
`/etc/postfix/virtual` is of `regexp:` type, not `hash:` type, so
running `postmap` on it has no effect; we need to reload Postfix when
it changes.
http://www.postfix.org/DATABASE_README.html#detect
In the interest of forcing a reload now, optimize the regexes by
eliding the unanchored `.*`s at the beginnings and ends.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Introduced by #12966.
puppet/zulip/manifests/base.pp - WARNING: double quoted string containing no variables on line 93
puppet/zulip/manifests/base.pp - WARNING: string containing only a variable on line 93
scanf doesn’t accept a number as input, so uh, add a dummy space
character.
What. You can’t give me a bad language and then complain when I write
bad programs in it.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Fixes this warning:
Warning: The string '8167976' was automatically coerced to the numerical value 8167976 (file: /root/zulip/puppet/zulip/manifests/base.pp, line: 93, column: 19)
Fixes#9682.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Delete trailing newlines from all files, except
tools/ci/success-http-headers.txt and tools/setup/dev-motd, where they
are significant, and static/third, where we want to stay close to
upstream.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Previous cleanups (mostly the removals of Python __future__ imports)
were done in a way that introduced leading newlines. Delete leading
newlines from all files, except static/assets/zulip-emoji/NOTICE,
which is a verbatim copy of the Apache 2.0 license.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
This gives us access to typing_extensions.Deque, which was not added
to typing until 3.5.4.
(PROVISION_VERSION is not bumped because the transitive dependency set
in dev.txt hasn’t changed.)
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
As a result of dropping support for trusty, we can remove our old
pattern of putting `if False` before importing the typing module,
which was essential for Python 3.4 support, but not required and maybe
harmful on newer versions.
cron_file_helper
check_rabbitmq_consumers
hash_reqs
check_zephyr_mirror
check_personal_zephyr_mirrors
check_cron_file
zulip_tools
check_postgres_replication_lag
api_test_helpers
purge-old-deployments
setup_venv
node_cache
clean_venv_cache
clean_node_cache
clean_emoji_cache
pg_backup_and_purge
restore-backup
generate_secrets
zulip-ec2-configure-interfaces
diagnose
check_user_zephyr_mirror_liveness
Since 204 responses don’t contain a payload body, Content-Type is
neither required nor encouraged (RFC 7231 §3.1.1.5), and ours was
missing a semicolon to boot; Content-Length is expressly
forbidden (RFC 7230 §3.3.2).
Furthermore, these add_header directives were silencing the CORS
headers set in api_headers, because add_header inheritance doesn’t
work the way you think it does, as was known before commit
5614d51afc.
Fixes: #12902.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Disable TLS 1.0 and TLS 1.1. (We no longer need to support IE8 on
Windows XP.)
Prefer client-selected cipher order. (Now that all enabled ciphers
provide good security, this allows mobile clients lacking AES hardware
acceleration to pick ChaCha20 for better performance.)
Disable session tickets. (Mozilla discourages them based on
https://www.imperialviolet.org/2013/06/27/botchingpfs.html.)
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Since we no longer support Ubuntu Trusty, we no longer need this
backwards-compatibility cruft (which we only kept around to avoid
randomizing configuration for existing systems).
This was only used in Ubuntu 14.04 Trusty.
Removing this also finally lets us simplify our security model
discussion of uploaded files.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Anders Kaseorg
rht
Vishnu KS
Tim Abbott
Hemanth V. Alluri
Wyatt Hoodes