This adds a new system for copying packages from old virtualenvs that
are sufficiently similar to the new virtualenv required.
In practice, this results in a huge performance improvement for
re-provisioning Zulip development environments when the requirements
files have changed (which is the dominant performance problem with
provision today).
Fixes: #1507.
In HTML, the line break immediately following a start tag is ignored
(see: https://www.w3.org/TR/html4/appendix/notes.html#h-B.3.1). An
extra span tag has been introduced in the upstream Pygments
HtmlFormatter in order to preserve the first new line. The Bugdown
Tests as well as our fenced_code.js frontend markdown processor have
been updated to reflect this new behavior.
This hasn't been used since before Zulip was open source, and isn't
super reusable, so we can remove it. It'll always be there in the
history if someone ends up wanting it.
While we're at it, we remove the GitPython dependency (only used for
this tool) and the example MSMTP config for the review tool.
We install service-identity to prevent twisted from giving spammy
warnings. However, we don't need the security features provided
by service-identity. So we can remove most of its cryptographic
dependencies.
Since twisted is only used for running tools/run-dev.py, we only
need it in development. Also twisted is not python 3 compatible.
So it should be shifter to requirements/py2_dev.txt.
Upgrade alabaster and sphinx to latest version.
commonmark can't be updated because recommonmark uses an old
version of commonmark which is incompatible with latest version.
Twisted is not python 3 compatible. So for now create a python2
venv and install twisted in it when running provision.py in python3
mode and use twisted from the python2 venv.
This dependency is only needed if USING_STATSD is set, and so had
apparently been missed from our dependency list previously (since few
sites use that setting).
This reverts commit f1f48f305e.
The use of sklearn unfortunately caused a substantial slowdown to the
Zulip provisioning process, which didn't seem worth it for a
relatively minor feature.