virtualenv on Ubuntu 16.04, when creating a new environment, downloads
the current version of setuptools, then replaces its pkg_resources
with an old copy from
/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl.
This causes problems, a simple example of which is reproducible from
the ubuntu:16.04 Docker base image as follows:
apt-get update
apt-get -y install python3-virtualenv
python3 -m virtualenv -p python3 /ve
/ve/bin/pip install sockjs-tornado
/ve/bin/pip download sockjs-tornado
→ `AttributeError: '_NamespacePath' object has no attribute 'sort'`
More relevantly, it breaks pip-compile in the same way. To fix this,
we need to force setuptools to be reinstalled, even if we’re asking
for the same version.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
This makes no changes to the locked versions in *.txt, but it reduces
duplicate information and gives us sane workflows for
* upgrading packages: remove some or all lines from *.txt and re-run
`update-locked-requirements`;
* marking packages as intentionally held back: add a version bound
to *.in with an explanatory comment.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
The reason that `pip-tools` running on Python 3 didn’t detect the
right requirements for `thumbor` on Python 2 is simply that some of
them are conditional on the Python version.
As for the requirements that had been manually added as a workaround:
`backports-abc` and `singledispatch` are now correctly detected, while
`backports.ssl-match-hostname` was vendored into `urllib3` some time
ago and `certifi` is no longer necessary.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Since LoopQueueProcessingWorker jobs cannot be monitored by checking
for connected consumers (since they poll, rather than consuming as
events arrive), they can't be monitored with check_consumers. It's
OK, because that monitoring was redundant with monitoring for
potential growth in their queue that we have as well.
Also clean up the block comments for the two other similar queue
procesors.
Add a specific command to restart Vagrant to adopt the new
configuration.
(When naïvely using only `vagrant halt` + `vagrant up --provision`,
external devices remained unable to connect; per `netstat -nltp`, the
host IP of forwarded ports remained `127.0.0.1`.)
This should dramatically improve the queue processor's performance in
cases where there's a very high volume of requests on a given endpoint
by a given user, as described in the new docstring.
Until we test this more broadly in production, we won't know if this
is a full solution to the problem, but I think it's likely. We've
never seen the UserActivityInterval worker end up backlogged without a
total queue processor outage, and it should have a similar workload.
Fixes#13180.
We don't actually need to go to the memcached (falling back to the
database) to fetch either user or client objects on every event. For
user objects, we actually can just pass through the user ID
transparently; for client objects, we can use an in-process cache,
since the mapping of string to ID never changes.
With the way these tests are, it's unnecessary to have 3 separate
classes, and it makes it confusing to decide where to add a potential
additional mm email test.
Bootstrap v2.2.0^2~40^2~6 changes this default to false, so this is a
prerequisite to upgrading Bootstrap, and it’s also safer.
This closes an HTML injection path via user full names in the emoji
reaction tooltip. It doesn’t appear to be exploitable for cross-site
scripting because we disallow `>` in full names, and the code happens
to be written such that the next `>` is in a different parser
invocation.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
This simple backwards-compatible change saves approximately 12% in the
compressed size of the chat.zulip.org page_params. We can do much,
much better by changing the format, but this seems like a good
intermediate step.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
In a gigantic realm where we send several MB of `page_params`, it’s
slightly better to have the rest of the `<body>` available to the
browser earlier, so it can show the “Loading…” spinner and start
fetching subresources.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
any_oauth_backend_enabled is all about whether we will have extra
buttons on the login/register pages for logging in with some non-native
backends (like Github, Google etc.). And this isn't about specifically
oauth backends, but generally "social" backends - that may not rely
specifically rely on Oauth. This will have more concrete relevance when
SAML authentication is added - which will be a "social" backend,
requiring an additional button, but not Oauth-based.
SOCIAL_AUTH_BACKEND / OAUTH_BACKEND_NAMES are currently the same
backends. All Oauth backends are social, and all social are oauth.
So we get rid of OAUTH_BACKEND_NAMES and use only SOCIAL_AUTH_BACKENDS.
Precompiling regexes gives a performance increase of around 10-15%
based on tests. See https://jsperf.com/typeahead-regex. This stacks
up when we have a lot of users in an organisation.
This changes the availability icon for bot users to user_circle_green;
previously it was accidentally defaulting to user_circle_empty, making
it appear that bots were never available.
Fixes#13149.
As it turns out, our rerender_the_whole_thing function (used whenever
we were adding messages and discovered that the resulting message list
would be out-of-order) was just broken and scrolled the browser to a
random location.
This caused two user-facing bugs:
* On very fast networks, if two users sent messages at very close to
the same time, we could end up with out-of-order message deliveries,
triggering this code path, which was intended to silently correct
the situation, but failed.
* In some narrows to streams with muted topics in the history but some
recent traffic, the user's browser-cached history might have some
gaps that mean the server fetch we do after narrowing discovers the
history is out-of-order, again triggering the
rerender_the_whole_thing code path.
The fix is to just remove that function, adding a new option to the
well-tested rerender_preserving_scrolltop (which has explicit logic to
preserve the scroll position) instead.
Fixes#12067. Likely also fixes#12498.
This sidesteps tricky escaping issues, and will make it easier to
build a strict Content-Security-Policy.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
This sidesteps tricky escaping issues, and will make it easier to
build a strict Content-Security-Policy.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Anders Kaseorg
Tim Abbott
Rishi Gupta
Ray Kraesig
Mateusz Mandera
Yago González
Rohitt Vashishtha
clarammdantas
Vishnu Ks