Commit Graph

596 Commits

Author SHA1 Message Date
Tim Abbott 17723b1617 version: Move minimum desktop version configuration to version.py.
This makes it relatively easy for a system administrator to
temporarily override these values after a desktop app security
release that they want to ensure all of their users take.

We're not putting this in settings, since we don't want to encourage
accidental long-term overrides of these important-to-security values.
2020-04-01 13:23:08 -07:00
Tim Abbott 737a36a9f8 portico: Use a backend variable to determine desktop version.
This makes it possible to change this parameter without rebuilding all
the server's static assets.
2020-03-27 01:37:56 -07:00
Anders Kaseorg 586f78cb32 install-node: Upgrade to Node.js 12.16.1, Yarn 1.22.4, nvm 0.35.3.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-03-26 21:34:47 -07:00
Eeshan Garg 1d5d0e649b pypi: Upgrade Zulip's PyPI packages to version 0.6.4. 2020-03-26 17:17:33 -07:00
arpit551 351015128c requirements: Upgrade python-api-bindings.
python 3.8 support for python-api-bindings was fixed in commit
63bc9b8a4f
so upgraded python-api-bindings to tag 0.6.3 which included this fix.
Bumped PROVISION_VERSION.
2020-03-25 16:09:59 -07:00
arpit551 dc6b09dc04 requirements: Add importlib-metadata and importlib-resources in dev.in.
importlib-metadata and importlib-resources are dependent packages for jsonschema
and cfn-lint respectively. They are built-in modules in later versions
of python (3.8, 3.7). When update-locked-requirements is run within python3.7 or
3.8 they will generate difference in locked files so we build these modules separately
to avoid such conflicts.
2020-03-23 10:28:21 -07:00
Dinesh 9e65f3f354 provision: Add provision support for puppeteer. 2020-03-22 11:42:14 -07:00
Steve Howell eb9a252ec9 populate_db, tests: Restrict emails in zulip realm.
We now restrict emails on the zulip realm, and now
`email` and `delivery_email` will be different for
users.

This change should make it more likely to catch
errors where we leak delivery emails or use the
wrong field for lookups.
2020-03-19 16:21:31 -07:00
Mateusz Mandera f5e95c4fc1 requirements: Bump python-social-auth version.
We had a bunch of ugly hacks to monkey patch things due to upstream
being temporarily unmaintained and not merging PRs. Now the project is
active again and the fixes have been merged and included in the latest
version - so we clean up all that code.
2020-03-18 12:14:31 -07:00
Anders Kaseorg 1cdab5ae61 emoji: Resolve emoji sprite sheets and stylesheets through Webpack.
This gives them cache-compatible URLs, and also avoids some extra
copies of the sprite sheet images.

Comments on the Octopus emoji added by tabbott.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-25 14:43:46 -08:00
Steve Howell da1ce9a577 emoji sprites: Avoid hard coding CSS percentages.
This commit changes the calculation of the
background-size parameter that we use to
render emojis from sprite sheets.

In particular, it now makes the parameter
match the sizes of our latest sprite
sheets from Twitter/Google.

This should fix the geometry aspect of #13959,
but we also need to fix some issues with the
cache being sticky.

There is also some minor cleanup:

    - Remove obsolete -moz/-webkit CSS.
    - Remove needless precision in percentages.
    - Fix the transposed nrows/ncols names.
    - Add extensive commenting.

Finally, we add a minor bump to the provision
number.  This commit should be merged in the
same series as the other fix for this issue,
which will probably have a major bump, and we'll
need to rebase this appropriately.
2020-02-21 10:21:12 -08:00
Vishnu KS 5bab2a3762 upload: Replace jQuery filedrop with Uppy. 2020-02-13 16:43:19 -08:00
rht 41e3db81be dependencies: Upgrade to Django 2.2.10.
Django 2.2.x is the next LTS release after Django 1.11.x; I expect
we'll be on it for a while, as Django 3.x won't have an LTS release
series out for a while.

Because of upstream API changes in Django, this commit includes
several changes beyond requirements and:

* urls: django.urls.resolvers.RegexURLPattern has been replaced by
  django.urls.resolvers.URLPattern; affects OpenAPI code and related
  features which re-parse Django's internals.
  https://code.djangoproject.com/ticket/28593
* test_runner: Change number to suffix. Django changed the name in this
  ticket: https://code.djangoproject.com/ticket/28578
* Delete now-unnecessary SameSite cookie code (it's now the default).
* forms: urlsafe_base64_encode returns string in Django 2.2.
  https://docs.djangoproject.com/en/2.2/ref/utils/#django.utils.http.urlsafe_base64_encode
* upload: Django's File.size property replaces _get_size().
  https://docs.djangoproject.com/en/2.2/_modules/django/core/files/base/
* process_queue: Migrate to new autoreload API.
* test_messages: Add an extra query caused by .refresh_from_db() losing
  the .select_related() on the Realm object.
* session: Sync SessionHostDomainMiddleware with Django 2.2.

There's a lot more we can do to take advantage of the new release;
this is tracked in #11341.

Many changes by Tim Abbott, Umair Waheed, and Mateusz Mandera squashed
are squashed into this commit.

Fixes #10835.
2020-02-13 16:27:26 -08:00
Anders Kaseorg d7d8632525 pygments_data: Replace JS module with JSON module.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-12 10:09:12 -08:00
Anders Kaseorg e257253e64 emoji_codes: Replace JS module with JSON module.
webpack optimizes JSON modules using JSON.parse("{…}"), which is
faster than the normal JavaScript parser.

Update the backend to use emoji_codes.json too instead of the three
separate JSON files.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-12 10:09:12 -08:00
Anders Kaseorg 78ac9138aa requirements: Upgrade all Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-05 12:38:10 -08:00
Anders Kaseorg 4d49a20430 requirements: Upgrade django-sendfile2 from 0.4.3 to 0.5.1.
The module was renamed from sendfile to django_sendfile.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-05 12:38:10 -08:00
Anders Kaseorg 62497b394a dependencies: Upgrade all JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-04 22:13:47 -08:00
Anders Kaseorg 91b57be02e dependencies: Upgrade stacktrace-gps from 3.0.3 to 3.0.4.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-04 22:13:33 -08:00
Anders Kaseorg e88fac499f dependencies: Upgrade emoji-datasource from 4.0.4 to 5.0.1.
The “Smileys & People” category has been split into “Smilys & Emotion”
and “People & Body”.

Also, fix generate_sha1sum_emoji to read the emoji-datasource-google
version from yarn.lock, since package.json only gives a version range.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-04 21:30:51 -08:00
Vishnu KS db97e0e329 emails: Use the word email instead of message in do not reply sentence.
Fixes #13693
2020-01-31 12:29:58 -08:00
Vishnu KS a4dcdd9c7b emails: Set alt attribute to empty for leading images.
The alt text of the leading images were displayed as preview
content in inbox by email clients like gmail. Since the leading
images were used mostly for decoration this made the preview
content gibberish. It's fine to set the alt attributes to empty
from accessibility point of view since the old alt attributes
did't added any meaningful information.
2020-01-24 13:32:54 -08:00
Mateusz Mandera 8dd95bd057 tests: Replace httpretty with responses.
responses is an module analogous to httpretty for mocking external
URLs, with a very similar interface (potentially cleaner in that it
makes use of context managers).

The most important (in the moment) problem with httpretty is that it
breaks the ability to use redis in parts of code where httpretty is
enabled.  From more research, the module in general has tendency to
have various troublesome bugs with breaking URLs that it shouldn't be
affecting, caused by it working at the socket interface layer.  While
those issues could be fixed, responses seems to be less buggy (based
on both third-party reports like ckan/ckan#4755 and our own experience
in removing workarounds for bugs in httpretty) and is more actively
maintained.
2020-01-22 11:56:15 -08:00
Tim Abbott c4ba44824d Update changelog for Zulip Server 2.1.2 release. 2020-01-16 12:39:00 -08:00
Anders Kaseorg ea6934c26d dependencies: Remove WebSockets system for sending messages.
Zulip has had a small use of WebSockets (specifically, for the code
path of sending messages, via the webapp only) since ~2013.  We
originally added this use of WebSockets in the hope that the latency
benefits of doing so would allow us to avoid implementing a markdown
local echo; they were not.  Further, HTTP/2 may have eliminated the
latency difference we hoped to exploit by using WebSockets in any
case.

While we’d originally imagined using WebSockets for other endpoints,
there was never a good justification for moving more components to the
WebSockets system.

This WebSockets code path had a lot of downsides/complexity,
including:

* The messy hack involving constructing an emulated request object to
  hook into doing Django requests.
* The `message_senders` queue processor system, which increases RAM
  needs and must be provisioned independently from the rest of the
  server).
* A duplicate check_send_receive_time Nagios test specific to
  WebSockets.
* The requirement for users to have their firewalls/NATs allow
  WebSocket connections, and a setting to disable them for networks
  where WebSockets don’t work.
* Dependencies on the SockJS family of libraries, which has at times
  been poorly maintained, and periodically throws random JavaScript
  exceptions in our production environments without a deep enough
  traceback to effectively investigate.
* A total of about 1600 lines of our code related to the feature.
* Increased load on the Tornado system, especially around a Zulip
  server restart, and especially for large installations like
  zulipchat.com, resulting in extra delay before messages can be sent
  again.

As detailed in
https://github.com/zulip/zulip/pull/12862#issuecomment-536152397, it
appears that removing WebSockets moderately increases the time it
takes for the `send_message` API query to return from the server, but
does not significantly change the time between when a message is sent
and when it is received by clients.  We don’t understand the reason
for that change (suggesting the possibility of a measurement error),
and even if it is a real change, we consider that potential small
latency regression to be acceptable.

If we later want WebSockets, we’ll likely want to just use Django
Channels.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-01-14 22:34:00 -08:00
Anders Kaseorg 1f31d6d32c dependencies: Upgrade vnu-jar.
This version includes my fix for the ‘Attribute “placeholder”’ test
flake (https://github.com/validator/validator/pull/884).

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-01-05 22:44:41 -08:00
Tim Abbott b68ff6446c version: Update version and changelog for Zulip 2.1.1 release. 2019-12-13 17:19:45 -08:00
Tim Abbott 9812c6d445 version: Update version strings following 2.1 release. 2019-12-12 22:53:52 -08:00
Tim Abbott 03a3ae8b61 Release Zulip Server 2.1.0. 2019-12-12 22:23:22 -08:00
Tim Abbott 4adcd35698 version: Update version and changelog for Zulip 2.0.8 release. 2019-12-12 17:32:27 -08:00
Tim Abbott 17bde5944d requirements: Upgrade versions of indirect dependencies. 2019-12-11 15:59:30 -08:00
Anders Kaseorg b932525368 people: Use Unicode normalization for diacritic removal.
Fixes #13481.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-12-09 13:02:54 -08:00
Tim Abbott 38d2d23ad3 Release Zulip server 2.1.0-rc1. 2019-11-22 15:33:36 -08:00
Tim Abbott 1dfbeaf05b version: Update LATEST_RELEASE_VERSION for 2.0.7. 2019-11-21 12:52:00 -08:00
Anders Kaseorg bae5e40f25 version: Set ZULIP_VERSION = "2.1.dev+git".
Since we don’t support downgrading from master to any 2.0.x release,
we shouldn’t set a ZULIP_VERSION that might lead someone to mistake
any such downgrade for an upgrade.  ZULIP_VERSION should always be at
least a minor version ahead of LATEST_RELEASE_VERSION, except on the
release branch.

`.dev` is a decreasing suffix that sorts before `alpha`, `beta`, `rc`
according to PEP 440/`packaging.version.Version`.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-21 10:33:55 -08:00
Mateusz Mandera 06c2161f7e auth: Use zxcvbn to ensure password strength on server side.
For a long time, we've been only doing the zxcvbn password strength
checks on the browser, which is helpful, but means users could through
hackery (or a bug in the frontend validation code) manage to set a
too-weak password.  We fix this by running our password strength
validation on the backend as well, using python-zxcvbn.

In theory, a bug in python-zxcvbn could result in it producing a
different opinion than the frontend version; if so, it'd be a pretty
bad bug in the library, and hopefully we'd hear about it from users,
report upstream, and get it fixed that way. Alternatively, we can
switch to shelling out to node like we do for KaTeX.

Fixes #6880.
2019-11-21 10:23:37 -08:00
Anders Kaseorg 40f4ead738 mypy: Upgrade from 0.720 to 0.730.
Fixes #13269.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 12:38:45 -08:00
Anders Kaseorg 5179e65493 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-11 17:19:46 -08:00
Anders Kaseorg 5e89a7d646 install-shellcheck: Upgrade ShellCheck from 0.6.0 to 0.7.0.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-11 16:26:31 -08:00
Anders Kaseorg ac49736311 install-node: Upgrade Node 12.11.1 to 12.13.0, Yarn 1.19.0 to 1.19.1.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-11 16:26:31 -08:00
Anders Kaseorg 60aebc4b55 dependencies: Upgrade nearly all JavaScript dependencies.
Keep handlebars at 4.2.x because
https://github.com/wycats/handlebars.js/issues/1598 breaks our test
suite, and simplebar at 4.2.x because of
https://github.com/Grsmto/simplebar/issues/406.  Upgrade everything
else.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-11 16:26:31 -08:00
Anders Kaseorg fffef412bc dependencies: Upgrade to-markdown 3.1.1 to turndown 5.0.3.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-11 16:26:31 -08:00
Anders Kaseorg cce85f6ec7 dependencies: Upgrade katex from 0.10.2 to 0.11.1.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-11 16:26:31 -08:00
Anders Kaseorg ffe8ec3450 dependencies: Upgrade eslint from 6.0.1 to 6.6.0.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-11 16:26:31 -08:00
Anders Kaseorg de4685441c typescript: Type webpack.config.ts correctly.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-04 18:12:11 -08:00
Thomas Ip c93522d847 blueslip: Make stack trace more readable.
The stack trace popup is now sourcemapped and each stackframe have a
expandable code context window.

[anders@zulipchat.com: Rebased and simplified.]
2019-10-31 13:47:54 -07:00
Anders Kaseorg c31615a760 package.json: Remove @types/source-map.
`source-map` provides its own types, so with TypeScript configured
with `--moduleResolution node`, we don’t need the obsolete
`@types/source-map` package.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-30 13:13:21 -07:00
Anders Kaseorg 70f72a3ae8 security: Send SameSite=Lax cookies.
Send the `csrftoken` and `sessionid` cookies with `SameSite=Lax`.
This adds a layer of defense against CSRF attacks and matches the new
default in Django 2.1:

https://docs.djangoproject.com/en/2.1/releases/2.1/#samesite-cookies

This can be reverted when we upgrade to Django ≥ 2.1.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-30 13:12:11 -07:00
Rafid Aslam 447f74ae63 Upgrade pika to 1.1.*.
Upgrade pika to 1.1.* and make some changes accordingly
to comply with the new version.

Fixes #12899.
2019-10-29 17:01:12 -07:00
Anders Kaseorg d4443bc4f8 requirements: Upgrade Python requirements.
Addresses a potential Pillow DoS vulnerability, among other things.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 16:42:20 -07:00
Anders Kaseorg d1a3bf424a requirements: Use webpack4 fork of django-webpack-loader.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:53:15 -07:00
Anders Kaseorg ee9a6071fd 5xx.html: Build with webpack.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:53:15 -07:00
Mateusz Mandera 52324bd005 requirements: Upgrade fakeldap to master.
For simple directory search support.
2019-10-17 16:49:53 -07:00
Anders Kaseorg 173c9cee42 frontend_tests: Switch from ts-node to Babel; add rewire-ts plugin.
This will let tests rewrite TypeScript/ES6 module bindings that would
otherwise be read-only.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-17 16:48:23 -07:00
Anders Kaseorg dfd9ace7fa requirements: Add back future.
It happens that commonmark, python-jose, and python-twitter don’t
actually use future on Python 3, and moto uses aws-xray-sdk in such a
way that it doesn’t use future, but this was a weird game to be
playing just to remove one dependency, and it caused CI failures after
new releases of future, so let’s just include it.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-11 14:04:35 -07:00
Anders Kaseorg 13296d282d requirements: Upgrade apns2.
My PR https://github.com/Pr0Ger/PyAPNs2/pull/90 fixing Python 3.5.2
support was merged.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-08 17:15:39 -07:00
Anders Kaseorg cc791afc54 requirements: Upgrade django-sendfile2.
django-sendfile2 now always sends a Content-Disposition header even if
it’s inline.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-08 17:15:00 -07:00
Mateusz Mandera 46d3dc243b requirements: Use our fork of django-auth-ldap. 2019-10-05 17:28:59 -07:00
David Rosa 9545569dca docs: Upgrade recommonmark to 0.6.0, fixing issues.
- recommonmark: 0.5.0 -> 0.6.0
- Fixed build TypeError: sequence item 1: expected str instance, NoneType found
  caused by recommonmark/parser.py erroring on a newline character.
- Removed deprecated code in conf.py that was causing warning messages
- Updated conf.py according to instructions for Sphinx-1.4 or newer
https://github.com/readthedocs/recommonmark/blob/master/README.md#getting-started
https://www.sphinx-doc.org/en/master/usage/markdown.html

This commit is also relevant to PR #13232.
2019-10-02 12:29:24 -07:00
Mateusz Mandera c42077c12f dependencies: Add dependencies needed for SAML. 2019-09-28 12:15:13 -07:00
Tim Abbott 4223254548 version: Update version and changelog for Zulip 2.0.6 release. 2019-09-23 16:22:41 -07:00
Anders Kaseorg ba8a172897 requirements: Downgrade transifex-client so we can upgrade six, urllib3.
transifex-client 0.13.5 added overly strict version bounds on six and
urllib3.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 13:47:55 -07:00
Anders Kaseorg f671ca3780 requirements: Upgrade Python requirements.
This commit was generated by deleting these lock files and rerunning
update-locked-requirements.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 13:23:58 -07:00
Anders Kaseorg 2573ecb7d1 requirements: Bump thumbor Django to match main Django.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 11:56:36 -07:00
Anders Kaseorg 92b42573fb requirements: Compile thumbor requirements with pip-tools on Python 2.
The reason that `pip-tools` running on Python 3 didn’t detect the
right requirements for `thumbor` on Python 2 is simply that some of
them are conditional on the Python version.

As for the requirements that had been manually added as a workaround:
`backports-abc` and `singledispatch` are now correctly detected, while
`backports.ssl-match-hostname` was vendored into `urllib3` some time
ago and `certifi` is no longer necessary.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 11:56:36 -07:00
Anders Kaseorg 990f08da90 dependencies: Upgrade nearly all JavaScript dependencies to latest.
Keep katex at 0.10.x, mini-css-extract-plugin at 0.6.x
(https://github.com/webpack-contrib/mini-css-extract-plugin/issues/417),
and typescript-eslint at 1.x for now.  Upgrade everything else.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-19 12:08:52 -07:00
Tim Abbott fcf5936341 version: Update version and changelog for Zulip 2.0.5 release. 2019-09-11 16:36:19 -07:00
Anders Kaseorg 7af04690b9 requirements: Use PyPI fork of line_profiler supporting Python 3.7.
Also move it to dev.in.

Other notes for posterity: this should have been installed with a
pinned commit hash, and could have been installed directly from the
upstream Git repository, even on Python 3.7, as long as Cython was
installed as well.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-08 09:34:55 -07:00
Anders Kaseorg 177eb2e02e webpack: Move CSS minification to optimization stage.
Otherwise Bootstrap doesn’t get minified, and also the minification
state is incorrectly reflected in the webpack cache.

The Terser plugin is used by default; we need to include it explicitly
to avoid removing it.

Switch from cssnano to clean-css because it’s noticeably faster.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-02 21:58:13 -07:00
Anders Kaseorg 9b0401b76d package.json: Use caret ranges for more deduplication opportunities.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-02 19:30:09 -07:00
Anders Kaseorg e57ede9421 yarn.lock: Share duplicate packages with yarn-deduplicate.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-02 19:30:09 -07:00
Anders Kaseorg d312d04510 styles: Replace Sass with PostCSS.
It’s about as fast as node-sass (faster, according to their
benchmarks) and more flexible.  Autoprefixer is neat: we can now go
delete all our -moz-, -webkit-, etc. lines and have them autogenerated
as necessary based on .browserslistrc.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-29 16:35:51 -07:00
Anders Kaseorg db44d61aab setup-apt-repo: Remove PPA and packagecloud repository.
We no longer use tsearch_extras, and the camo patch is irrelevant on
systemd systems (Xenial and newer).  So we no longer need to
provide/install a PPA at all.

Closes #13027.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-29 12:53:04 -07:00
Anders Kaseorg 6701c4463c search: Remove now unnecessary tsearch_extra dependency.
Now that we're implemented tsearch_extras in pure postgres, we no
longer need a custom extension.  This should help us considerably, as
it means we no longer need to ship custom apt packages at all.

Fixes #467.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-29 12:49:26 -07:00
Anders Kaseorg 43c8c720ef node_cache: Run yarn with --ignore-scripts.
As predicted in https://www.kb.cert.org/vuls/id/319816/, a malicious
worm is beginning to spread across the npm ecosystem through package
postinstall scripts.  Only instead of direct self-replicating code,
the replication vector is the temptation to monetize postinstall
scripts by polluting the console logs with paid advertisements.  The
effect will be the same unless we all put a stop to this while we
still can.

Apply the recommended VU#319816 workaround, which is to disable
lifecycle scripts when installing npm packages.  The only fallout is:

* node-sass can’t run because it uses compiled native code; we replace
  it with Dart Sass.
* phantomjs-prebuilt doesn’t download the binary at install time; we
  tell it to download it in run-casper.
* ttf2woff2 transparently falls back from native code to an Emscripten
  build.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-26 14:36:03 -07:00
Anders Kaseorg 37ece9b8e2 webpack: Replace node-sass with Dart Sass.
It doesn't require scripts to install, allowing us to migrate yarn to
the more secure --ignore-scripts option.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-26 14:35:26 -07:00
Anders Kaseorg 29abbdaa07 requirements: Upgrade pip-tools to 4.1.0 and pip to 19.2.3.
Closes #13068.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-26 14:33:18 -07:00
Anders Kaseorg a053a925b3 requirements: Upgrade pip-tools to 3.8.0; downgrade pip to 19.1.1.
These are not the latest versions, but pip-tools 3.9.0 or 4.0.0 fails
to resolve dependencies from Git URLs:

pip._internal.exceptions.DistributionNotFound: No matching distribution found for zulip==0.6.1_git (from -r requirements/common.in (line 135))

while pip 19.2 breaks pip-tools 3.8.0:

TypeError: __init__() got an unexpected keyword argument 'find_links'

Fixes #10802.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-25 15:03:20 -07:00
Vishnu KS f32382f7b7 emails: Show preheader block only if preheader is present. 2019-08-17 11:32:28 -07:00
Anders Kaseorg a5596011a0 queue_processors, python_examples: Fix mypy errors.
zerver/openapi/python_examples.py:105: error: Argument 1 to "get_user_presence" of "Client" has incompatible type "str"; expected "Dict[str, Any]"
    zerver/openapi/python_examples.py:563: error: Argument 1 to "add_reaction" of "Client" has incompatible type "Dict[str, object]"; expected "Dict[str, str]"
    zerver/openapi/python_examples.py:576: error: Argument 1 to "remove_reaction" of "Client" has incompatible type "Dict[str, object]"; expected "Dict[str, str]"
    zerver/worker/queue_processors.py:587: error: Argument "client" to "extract_query_without_mention" has incompatible type "EmbeddedBotHandler"; expected "ExternalBotHandler"

These were only missed because mypy daemon mode requires us to set
`follow_imports = skip` for the `zulip` package.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-16 14:13:40 -07:00
Vishnu KS 7a433f279a requirements: Upgrade Django from 1.11.22 to 1.11.23. 2019-08-15 16:56:31 -07:00
Anders Kaseorg 72655611ce requirements: Use maintained fork django-sendfile2 of django-sendfile
The original seems to be unmaintained
(johnsensible/django-sendfile#65).  Notably, this fixes a bug in the
filename parameter, which perviously showed the Python 3 repr of a
byte string (johnsensible/django-sendfile#49).

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-12 15:40:08 -07:00
Anders Kaseorg b0859f4b1e linter_lib: Fix mypy errors.
tools/linter_lib/pyflakes.py:35: error: Argument 3 to "run_pyflakes" has incompatible type "List[Tuple[bytes, bytes]]"; expected "List[Tuple[str, str]]"
    tools/linter_lib/custom_check.py:110: error: Argument "rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"
    tools/linter_lib/custom_check.py:214: error: Argument "rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"
    tools/linter_lib/custom_check.py:214: error: Argument "shebang_rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"
    tools/linter_lib/custom_check.py:502: error: Argument "rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"
    tools/linter_lib/custom_check.py:502: error: Argument "shebang_rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"
    tools/linter_lib/custom_check.py:519: error: Argument "rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"
    tools/linter_lib/custom_check.py:706: error: Argument "rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"
    tools/linter_lib/custom_check.py:728: error: Argument "rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"
    tools/linter_lib/custom_check.py:738: error: Argument "rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"
    tools/linter_lib/custom_check.py:779: error: Argument "rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"
    tools/linter_lib/custom_check.py:779: error: Argument "length_exclude" to "RuleList" has incompatible type "Set[str]"; expected "List[str]"
    tools/linter_lib/custom_check.py:803: error: Argument "length_exclude" to "RuleList" has incompatible type "Set[str]"; expected "List[str]"
    tools/linter_lib/custom_check.py:805: error: Unsupported operand types for + ("List[Rule]" and "List[Dict[str, Any]]")
    tools/linter_lib/custom_check.py:819: error: Argument "rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"

These were missed the `zulint` package was missing PEP 561 type
annotation markers, and if it’d had them, mypy daemon mode would’ve
required us to set `follow_imports = skip` for it.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-09 17:22:45 -07:00
Anders Kaseorg 671844d667 provision: Add default-jre-headless for vnu-jar.
This was previously pulled in by yui-compressor.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-08 17:28:20 -07:00
Anders Kaseorg 7625f0ecae requirements: Use a pinned zulint commit, not master.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-06 23:05:43 -07:00
Vishnu KS 216403768d emails: Remove body tag from email_base_message.
Presence of body tag in missed_message results in
gmail not hiding footer in mobile app.
2019-08-02 13:54:32 +05:30
Mateusz Mandera 8f6f78b912 requirements: Upgrade redis from 2.10.6 to 3.2.1.
Fixes #11209.

This requires changing how zadd is used in rate_limiter.py:
In redis-py >= 3.0 the pairs to ZADD need to be passed as a dictionary,
not as *args or **kwargs, as described at
https://pypi.org/project/redis/3.2.1/ in the section
"Upgrading from redis-py 2.X to 3.0".

The rate_limiter change has to be in one commit with the redis upgrade,
because the dict format is not supported before redis-py 3.0.
2019-07-29 15:34:05 -07:00
Aman 2183a74040 zulint: Use zulint from the extracted repository.
zulint will be added as a "third-party" dependency in zulip from now
on.  See the new project at https://github.com/zulip/zulint for more
details.
2019-07-26 11:35:43 -07:00
Vishnu Ks 618d026941 version: Bump PROVISION_VERSION for py packages upgrade. 2019-07-26 10:57:31 -07:00
Anders Kaseorg 6d5a20ac62 requirements: Remove django-pipeline.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-24 17:40:31 -07:00
Anders Kaseorg f54a63e2f9 webpack: Transpile JS code with Babel.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-22 17:55:32 -07:00
Anders Kaseorg ecfb7c6a7f lint: Add TypeScript compiler as a linter.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-22 17:55:18 -07:00
neiljp (Neil Pilgrim) ba7a0934e3 requirements: Upgrade mypy to 0.711.
This comes with it a big performance improvement; mypy is now only
barely our slowest linter even if it wasn't previously running.

Fixes: #12058
2019-07-22 17:12:50 -07:00
Harshit Bansal bf14a0af4d auth: Migrate google auth to python-social-auth.
This replaces the two custom Google authentication backends originally
written in 2012 with using the shared python-social-auth codebase that
we already use for the GitHub authentication backend.  These are:

* GoogleMobileOauth2Backend, the ancient code path for mobile
  authentication last used by the EOL original Zulip Android app.

* The `finish_google_oauth2` code path in zerver/views/auth.py, which
  was the webapp (and modern mobile app) Google authentication code
  path.

This change doesn't fix any known bugs; its main benefit is that we
get to remove hundreds of lines of security-sensitive semi-duplicated
code, replacing it with a widely trusted, high quality third-party
library.
2019-07-21 20:51:34 -07:00
Anders Kaseorg 31bd3d617d version: Document when a dependency upgrade requires a major bump.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-21 20:13:52 -07:00
Anders Kaseorg 06aa87296c dependencies: Upgrade eslint to 6.0.1.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-20 16:12:51 -07:00
Vinit Singh 86073588be dependencies: Upgrade jquery-autosize 1.17.7 to autosize 4.0.2.
The API for the autosize library changed upstream, so several changes
had to be made to relevant js files for a successful upgrade.

Resolves #12695.
2019-07-18 14:33:16 -07:00
Anders Kaseorg 953b69e5ea dependencies: Upgrade simplebar to 4.1.0.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-18 12:19:55 -07:00
Anders Kaseorg ab89f40a66 generate-custom-icon-webfont: Replace with webpack webfonts-loader.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-18 12:00:00 -07:00
Vishnu Ks e32ee53fa3 emails: Improve wording of why you are getting this email.
Also remove <br> tag from missed_message text message.
2019-07-17 17:50:18 -07:00
Eeshan Garg 8dfc9d8bca pypi: Upgrade to release 0.6.1. 2019-07-15 12:28:16 -07:00
Tim Abbott 06d3194ff6 dependencies: Use the django-bitfield 1.9.5 release.
The release contains all the fixes from our fork.
2019-07-12 16:25:40 -07:00
Anders Kaseorg 218c60ae86 third: Upgrade spectrum-colorpicker to 1.8.0 from NPM.
We had several patches to spectrum, but the only essential one 
(0ea770fc18) had already been fixed upstream,
and another was just handling jQuery deprecation warnings for not yet removed features.

See #12749 for details.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-10 10:07:34 -07:00
Vishnu Ks 3c464d9d63 emails: Show realm_str inside bracket in subject. 2019-07-09 11:59:39 -07:00
Vishnu Ks 0b35bb96f0 requirements: Upgrade pillow from 5.4.1 to 6.1.0.
Fixes #12193
2019-07-07 22:28:54 -07:00
Rohitt Vashishtha d6e8bd6515 dependencies: Upgrade markdown from 3.0.1 -> 3.1.1.
Fixes #12192.
2019-07-03 15:06:19 -07:00
Anders Kaseorg 079ddae4c8 minify-js: Remove; everything has been migrated to Webpack.
min/sockjs-0.3.4.min.js is not used.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-03 13:58:21 -07:00
Anders Kaseorg 5f3d36ae87 tex: Bundle KaTeX CLI using Webpack.
We don’t need a hacked copy anymore.  We run the installed version out
of node_modules in development, and a Webpack-bundled version of that
in production.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-02 16:50:03 -07:00
Anders Kaseorg f346d0e511 dependencies: Use core-js for String.prototype polyfills.
It seems like the de facto standard ES polyfill library these days,
and we already depend on it through simplebar.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-02 16:50:03 -07:00
Anders Kaseorg 438c7c46ed third: Get jquery-caret-plugin from NPM.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-02 15:04:58 -07:00
Anders Kaseorg 1647582acf third: Get jquery-autosize from NPM.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-02 15:04:58 -07:00
Vishnu Ks a5c69fb707 emails: Simplify content styling of missed_message emails.
This is a dramatic redesign of the look and feel of our missed-message
emails, designed to decrease the feeling of clutter and just provide
the content users care about in a clear, visible fashion.
2019-06-28 11:58:52 -07:00
Anders Kaseorg f407a12ba2 test-documentation: Validate HTML with vnu.jar.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-06-27 14:53:21 -07:00
Thomas Ip 637b2b784a dependencies: Upgrade mini-css-extract-plugin to 0.6.0 and use HMR.
Given mini-css-extract-plugin can now do hot module replacement,
this commit also removed css-hot-loader. Not upgrading to 0.7.0
as that cause webpack to crash.
2019-06-24 17:58:51 -07:00
Wyatt Hoodes 6670b15234 provision: Clean up `var/` for the move to `var/<uuid>/test-backend`. 2019-06-24 12:42:00 -07:00
Wyatt Hoodes 6b033c7909 test-backend: Add steps to deal with potential database leaks.
A function was written in `test_fixtures.py` to drop a test database
template if the corresponding database id doesn't belong to a file.
Alongside this fact, every file that is written is removed after 60
minutes.  Meaning any potential database template can never exist
longer than one hour.

This follow-up work was added to deal with the potential race
conditions when running `test-backend`.  Ensuring that all templates
are properly dealt with.

Essentially rewritten by tabbott for cleanliness.

Fixes the remainder of #12426.
2019-06-14 15:23:25 -07:00
ppreethi 86840adda5 version: Show number of commits and commit sha in ZULIP_VERSION.
We use `git describe --tags` to get information about the number of commit since
the last major version, and the sha of the current HEAD. This is added to the
ZULIP_VERSION when a deploy is done from `git`.

Modified heavily by punchagan to:
* to use git describe instead of `git log` and `wc`
* use a separate script to run the git describe command
* write the file with version info to var/ and remove it from the repo

Fixes #4685.
2019-06-07 13:39:12 -07:00
Tim Abbott 9fbc6ab022 version: Update version and changelog for Zulip 2.0.4 release. 2019-06-06 22:04:41 -07:00
Thomas Ip d1d06af2c6 dependencies: Upgrade jQuery to 3.4.1.
API changes:
* Positional selectors (eg :first, :eq) are deprecated in 3.4.0 and
  will be removed in 4.0. Use positional methods instead.
2019-06-06 15:21:26 -07:00
Priyank Patel e590ce95ea dependencies: Add cache-loader for webpack build caching. 2019-06-03 20:15:51 -07:00
Tim Abbott cd1356f024 version: Bump major PROVISION_VERSION.
This should ensure that folks need to re-provision downgrade if they
switch branches to before this big frontend package upgrade.
2019-06-03 12:32:49 -07:00
Thomas Ip 3b105921bb dependencies: Upgrade winchan to 0.2.1. 2019-06-03 12:24:00 -07:00
Tim Abbott 43e634a9b8 Revert "dependencies: Upgrade webpack-dev-server to v3.4.1."
This seems to have broken the development server.

This reverts commit daf8dd1960.
2019-05-31 17:59:13 -07:00
Priyank Patel daf8dd1960 dependencies: Upgrade webpack-dev-server to v3.4.1. 2019-05-31 15:07:17 -07:00
Priyank Patel 352b864124 requirements: Add pyinotify dependency. 2019-05-29 16:49:33 -07:00
Anders Kaseorg 3b7f3f1887 dependencies: Upgrade simplebar to 4.0.0.
Fixes #12347.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-23 17:31:07 -07:00
Anders Kaseorg 141088586b Completely replace perfect-scrollbar with SimpleBar.
perfect-scrollbar replaces both the appearance and the behavior of the
scrollbar, and its emulated behavior will never feel native on most
platforms.  SimpleBar customizes the appearance while preserving the
native behavior.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-05-17 12:06:51 -07:00
Anders Kaseorg 491589579a dependencies: Upgrade simplebar to 4.0.0-alpha.9.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-17 12:06:51 -07:00
Puneeth Chaganti be762f9485 digest: Strip down the digest email removing a lot of fluff. 2019-05-06 17:43:52 -07:00
Puneeth Chaganti 0884764421 url preview: Add python-magic dependency for content-type sniffing. 2019-05-06 12:37:32 -07:00
Anders Kaseorg d4b060a520 dependencies: Upgrade simplebar to 4.0.0-alpha.6.
Fixes #12038.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-28 21:32:55 -07:00
Vishnu Ks 07291227d5 version: Bump PROVISION_VERSION for py packages upgrade. 2019-04-26 16:22:00 -07:00
Tim Abbott bf9f08a3e6 version: Update version and changelog for Zulip 2.0.3 release. 2019-04-23 16:05:12 -07:00
neiljp (Neil Pilgrim) 1348c8fd05 requirements: Upgrade mypy to 0.670.
Requires minor adjustment for accented 'i' in variable name.
2019-04-16 17:50:12 -07:00
neiljp (Neil Pilgrim) 2160ece8e2 requirements: Upgrade mypy to 0.660. 2019-04-16 17:50:12 -07:00
Thomas Ip 02cb85a8a9 typescript: Use ESLint instead of TSLint. 2019-04-13 11:42:47 -07:00
Priyank Patel 9c4ef94726 dependencies: Add lazysizes npm package. 2019-04-05 15:51:02 -07:00
Thomas Ip 42a666c22b dependencies: Downgrade ts-node to v7.0.1.
The current v8.0.0^ has a slow startup time due to some module
resolution issue.
2019-03-21 10:44:34 -07:00
Eeshan Garg a6abafa54b python-zulip-api: Upgrade to release 0.6.0. 2019-03-19 20:07:53 -02:30
Anders Kaseorg 8b76a55b16 dependencies: Upgrade simplebar to 3.1.3.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-03-18 22:24:29 -07:00
Rhythm Sharma e1212c04d1 emoji: Add a few useful aliases. 2019-03-17 13:38:31 -07:00
Tim Abbott cfbe316215 version: Update version and changelog for Zulip 2.0.2 release. 2019-03-15 12:25:50 -07:00
Puneeth Chaganti 8ca4ca1400 digest: Don't update context with data already in common context. 2019-03-07 15:29:51 -08:00
Vishnu Ks 3ac2be6c5f version: Bump PROVISION_VERSION to facilitate dependency upgrade. 2019-03-06 09:31:24 -08:00
Tim Abbott ddb965110f CSS: Remove manual antialiasing configuration.
The antialiasing decisions we made for the webapp should be constant
over the entire page, not limited to particular subsections or themes.

If we wanted antialiasing, we should do it on the entire page, not
individual random widgets.  But it's not clear we actually want to do
it on the entire page.  The `-moz-osx-font-smoothing: grayscale`
setting now happens by default in OSX Mojave (40% world market share
right now and growing), so there's no reason to override it.  And
without retina displays, generally, subpixel rendering provides better
results than antialiasing (which overrides subpixel rendering).

Thanks to Anders Kaseorg for advice on this issue.
2019-03-04 15:12:48 -08:00
Bennet Sunder 7c5f316cb8 alert_words: Performance improvements in looking for alert_words.
This commit leverages the ahocorasick algorithm to build a set of user_ids
that have their alert_words present in the message. It runs in linear time
of the order of length of the input message as opposed to number of
alert_words. This is after building a ahocorasick Automaton which runs
in O(number of alert_words in entire realm) which is usually cached.
2019-03-01 15:36:39 -08:00
Tim Abbott 34460173f4 version: Update version strings following 2.0 release. 2019-03-01 10:38:40 -08:00
Tim Abbott ac24fdd4eb Release Zulip Server 2.0.0. 2019-03-01 10:32:10 -08:00
Puneeth Chaganti b8e25677ef digest: Remove unnecessary scroll bar for private messages. 2019-02-22 13:09:26 -08:00
Eeshan Garg 8de84eea4c pypi_packages: Upgrade to release 0.5.9. 2019-02-17 02:57:03 -03:30
Puneeth Chaganti 43caf28e36 digest: Use the same font for the salutation as other paragraphs.
The fact that these were different was just a subtle visual bug.
2019-02-13 10:43:26 -08:00
Thomas Ip 60c9f22129 typescript: Update TS related dependencies. 2019-02-12 15:59:51 -08:00
Tim Abbott 23f950c60c version: Bump ZULIP_VERSION after 2.0.0-rc1 release. 2019-02-11 20:25:00 -08:00
Tim Abbott edb50cbf2c Release Zulip Server 2.0.0-rc1. 2019-02-11 20:07:38 -08:00
Anders Kaseorg 8fdb62a6bc tools: Create DEPLOY_ROOT/zulip-py3-venv symlink in development too.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-07 17:12:33 -08:00
Shoumorup e8ba08367b requirements: Upgrade pyflakes.
Pyflakes has been upgraded from 2.0.0 to 2.1.0 and
a few new linter errors have been fixed.

Fixes #11397.
2019-02-01 07:26:13 -08:00
Pragati Agrawal e1772b3b8f tools: Upgrade Pycodestyle and fix new linter errors.
Here, we are upgrading pycodestyle version from 2.4.0 to 2.5.0.

Fixes: #11396.
2019-01-31 12:21:41 -08:00
Vishnu Ks cf5b21d422 version: Bump provision version for mass dependency upgrade. 2019-01-30 09:50:43 -08:00
Tim Abbott dc663057ce docs: Add changelog from 1.9.2 release. 2019-01-29 16:54:45 -08:00
Eeshan Garg 9cdcfad920 pypi: Upgrade to release 0.5.8. 2019-01-22 22:49:51 -03:30
Vishnu Ks 8f3a0927c3 lint: Ban color names in CSS.
We already lint using HSL rather than RGB consistently.
2019-01-22 15:33:18 -08:00
Eeshan Garg 3c4e3ddcdb pypi: Upgrade to release 0.5.7. 2019-01-16 21:27:50 -03:30
Rohitt Vashishtha b7c5ae7bca dependencies: Upgrade markdown from 2.6.11 -> 3.0.1.
This is a major upgrade, and requires some significant compatibility
work:
* Migrating the pattern-removal logic to use the Registry feature.
* Handling the removal of positional arguments in markdown extensions.
* Handling the removal of safe mode.
2019-01-11 11:40:18 -08:00
Vishnu Ks c6352c4d9d version: Bump provision version to facilitate dependency upgrade. 2019-01-07 10:30:49 -08:00
Vishnu Ks 689d8221e5 emails: Fix the if condition logic in missed_message.html.
Refactoring in 4e1c058 was not correct since recipient_block
and message_content checked for if not condition while
recipient_header checked for if.

The naming of classes in 6077a33 was also not correct
semantically.
2019-01-05 10:22:39 -08:00
Vishnu Ks 7e8825bc6f emails: Change HSL colors to HEX for missed_message styles. 2019-01-03 17:55:29 -08:00
Eeshan Garg 88cf6777d0 python-zulip-api: Upgrade to latest PyPI release (0.5.6). 2018-12-17 18:37:20 -03:30
Anders Kaseorg 149132348f lint: Update ShellCheck to 0.6.0.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-12-12 08:24:01 -08:00
neiljp (Neil Pilgrim) 63768858ff dependencies: Upgrade mypy to version 0.650.
Broaden the type of the AbstractEnum __reduce_ex__ parameter to object; this
matches the parameter type specified in the latest enum.pyi file in typeshed.

Fixes #10996.
2018-12-09 22:20:12 -08:00
Vishnu Ks 6de6b0ed31 emails: Extract CSS to email.css in digest.html. 2018-12-03 17:43:53 -08:00
Tim Abbott 34bf5412b1 Add changelog and version updates for Zulip Server 1.9.1.
The actual changes in 1.9.1 are already present in master.
2018-11-30 15:51:02 -08:00
Raymond Akornor d00b889402 auth: Add an organization reactivation flow with admin confirmation.
This adds a web flow and management command for reactivating a Zulip
organization, with confirmation from one of the organization
administrators.

Further work is needed to make the emails nicer (ideally, we'd send
one email with all the admins on the `To` line, but the `send_email`
library doesn't support that).

Fixes #10783.

With significant tweaks to the email text by tabbott.
2018-11-27 10:58:34 -08:00
Cynthia Lin d36dc5e2d1 hello: Add link and label for latest Zulip release.
Tweaked by tabbott to add the plumbing for the 3 variables we need and
add the links, and move to a plain-text styling.
2018-11-19 12:00:40 -08:00
Vishnu Ks 3258ce1fa4 emails: Cleanup followup day1 email. 2018-11-19 11:16:36 -08:00
Vishnu Ks d3138d486e requirements: Upgrade setuptools to 40.6.1. 2018-11-13 12:15:07 -08:00
Vishnu Ks fba5b329ab version: Bump PROVISION_VERSION for py packages upgrade. 2018-11-12 15:27:26 -08:00
Tim Abbott db51d40f4e release: Update version strings following 1.9.0 release. 2018-11-07 07:53:12 -08:00
Tim Abbott 06d39f87b7 Release Zulip Server 1.9.0. 2018-11-07 07:24:30 -08:00
Tim Abbott e55a3b6cd1 version: Bump after 1.9.0-rc3 release. 2018-11-02 17:30:13 -07:00
Tim Abbott 27f6e9359d Release Zulip Server 1.9.0-rc3. 2018-11-02 17:19:38 -07:00
Aaditya Arora f4d4c55d92 dependencies: Upgrade mypy to version 0.641.
This requires upgrading a few other packages that mypy depends on.
2018-10-24 10:48:05 -07:00
Anders Kaseorg c3bd293eaf Add shellcheck as a linter.
Fixes #10162.

Tweaked by tabbott to cover documentation as well.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-18 15:32:26 -07:00
Harshit Bansal fc7e654d7b emoji: Remove single-digit names for number emojis.
Names like `:1:`, `:2:` etc don't seem to be of much benefit rather
they seem to get rendered at places not intended.
2018-10-02 11:30:11 -07:00
Eeshan Garg c9765da47d pypi: Upgrade to release 0.5.5. 2018-09-25 18:24:36 -07:00
Tim Abbott 9b0c702035 version: Bump after 1.9.0-rc2 release. 2018-08-31 16:13:15 -07:00
Tim Abbott 00cb9f8fe2 Release Zulip Server 1.9.0-rc2. 2018-08-31 16:10:47 -07:00
Eeshan Garg e1c4df0acb pypi: Upgrade to release 0.5.4. 2018-08-31 13:44:59 -07:00
Harshit Bansal e7311cdf5d emoji: Bring back the feature of changing emojisets.
This is largely inspired by requests from people not liking the
Google's new emojiset. A lot of people were requesting to revert
back to old blobs emojiset so we are re-enabling this feature
after making relevant infrastructure changes for supporting google's
old blob emojiset and re-adding support for twitter emojiset.

Fixes: #10158.
2018-08-31 11:52:24 -07:00
Shubham Dhama e70cf3bd67 emails/password_reset: Change text for listing other active accounts.
The main benefit of this change is that it reduces the amount of total
string that we might need to tag for translation.

Fixes: #10323.
2018-08-22 17:49:40 -07:00
Eeshan Garg 080b1eccea pypi: Upgrade to release 0.5.3. 2018-08-18 08:00:56 -07:00
Yago González 2f7e4484c8 requirements: Upgrade python-zulip-api/zulip to 7dfecf557. 2018-08-16 14:12:44 -07:00
Yago González 9acb98fa53 version: Bump PROVISION_VERSION for Python package updates. 2018-08-14 17:53:34 -07:00
Aditya Bansal 56d49980de puppet-lint: Add as a dependancy to install during provision process. 2018-08-13 12:30:57 -07:00
Harshit Bansal 29daf76420 emoji: Don't generate emoji farms corresponding to now banned emojisets.
This will help us in reducing the size of the release tarball
significantly. I have refrained from changing the `EMOJISETS`
constant in the `emoji_setup_utils.py` as that controls the
emojisets that we want to support. Since we want to re-enable
the feature of changing emojisets sometime again in the future
that variable should be kept as it is as it controls several
other things like emoji scripts that we use to generate emoji
names. Changing it might cause hard to catch bugs.
2018-08-13 12:09:06 -07:00
Harshit Bansal 99e1a81055 emoji: Fix some emoji images not loading in missed message emails.
`emoji-datasource` package v4.0.4 introduced the concept of qualified
and non-qualified emoji codes. As chat programs don't need to use
emoji representation selector, so we used migrated our infrastructure
to use non-qualified emoji codes. But we missed the fact that the
emoji file names in emoji farm are based on emoji data's 'unified'
field and the value of this field has changed. Consequently the image
file names must also have been changed. We used `emoji_code` while
converting the span tags to img tags while processing notifications.
But since now `emoji_code` refers to non-qualified code while image
file names are based on qualified code, we need to rename images
to correctly do the conversion. This commit just fixes this.
2018-08-13 12:03:16 -07:00
Tim Abbott 705c761980 version: Bump ZULIP_VERSION following release candidate 1.9.0-rc1. 2018-08-08 17:53:03 -07:00
Shubham Dhama 2dec30e4ab forms: Fix accounts listed in password_reset email to active accounts.
Previously we were listing both accounts, active as well as non-active.
Fixes: #10130.
2018-08-04 09:16:19 -07:00
Vishnu Ks c0ed2283d3 requirements: Upgrade django-auth-ldap to 1.6.0.
The autenticate function now follows the signature of
Django 2.0 https://github.com/django-auth-ldap/
django-auth-ldap/commit/27a8052b26f1d3a43cdbcdfc8e7dc0322580adae

Also AUTH_LDAP_CACHE_GROUPS is depricated in favor of
AUTH_LDAP_CACHE_TIMEOUT.
2018-08-02 15:53:11 -07:00
Vishnu Ks 32438cde9f version: Bump PROVISION_VERSION for py packages upgrade. 2018-07-30 10:09:19 -07:00
Rhea Parekh ddf011c915 requirements: Upgrade mypy to 0.620.
Fixes #9975.
2018-07-28 15:35:32 -07:00
Anupam Dagar c47e71abe0 emails: Fix preheader coming up inside email.
This makes the preheader correctly invisible inside the email.
2018-07-27 10:36:13 -07:00
Eeshan Garg 73e8daffbe pypi: Upgrade to release 0.5.2. 2018-07-27 00:11:52 -07:00
Harshit Bansal cf5b2b4815 emoji: Change emoticon mapping for `:)`, `(:` and `:(`.
See discussion on CZO:
https://chat.zulip.org/#narrow/stream/101-design/subject/emoji.20picker/near/617811
2018-07-26 11:17:03 -07:00
Aditya Bansal f2368bea22 stylelint: Add stylelint package. 2018-07-25 09:04:03 -07:00
Harshit Bansal c0b0fb7cce emoji: Move `EMOTICON_CONVERSIONS` mapping to build_emoji infra.
This commit closes a long pending issue which involved moving the
`EMOTICON_CONVERSION` mapping to build_emoji infrastructure so
that there is only one source of truth. This was pending from the
time when this feature was implemented.
2018-07-23 12:35:08 -07:00
Harshit Bansal bf70955c0c emoji: Update `emoji-datasource` packages.
This commit updates the `emoji-datasource` packages to version 4.0.4.
This update brings following changes to emoji infra:

1: Fix for the bleeding sprite sheets.

2: The category of some emojis has been changed. Categorywise breakup of
   net gain or loss is as follows:
    Travel & Places: 58 (gain)
    Symbols: 47 (loss)
    Smileys & People: 52 (gain)
    Objects: 11 (loss)
    Food & Drink: 3 (gain)
    Animals and Nature: 46 (gain)
    Activities: 9 (loss)

3: There were some changes in the image farm of the package which were
   breaking our old emoji farm. I fixed them by modifying the remapped
   emoji map.

Fixes: #8235.
2018-07-23 08:04:58 -07:00
Aditya Bansal 0e50afb599 thumbor: Add ldap and django as a dependancy.
We add this dependancy to thumbor for no use other than making an
import possible in one of the upcoming commits. Basically we wanted to
import LOCAL_UPLOADS_DIR from zproject.prod_settings or
zproject.dev_settings and prod_settings_template.py imports
django-auth-ldap (which depends on python-ldap and django).

This seems counterproductive, but it makes it possible for us to save
significant thumbor server startup time that would have been consumed
in `get-django-setting`, and once thumbor supports Python 3, we'll
probably be merging the virtualenvs anyway (in which case this change
would become a no-op).
2018-07-15 00:03:21 +05:30
Tim Abbott 52f5d83c90 thumbor: Add libthumbor dependency.
We need this for signing thumbnail requests from within Zulip.
2018-07-14 21:46:02 +05:30
Harshit Bansal f636882e04 build_emoji: Migrate to use `emoji_names.py` file.
This migrates Zulip to use a dramatically better set of names and
aliases for our emoji set, defined in emoji_names.py (which is in turn
manually generated from our hand-curated CSV file).

This should significantly improve the experience of using Zulip's
emoji picker and emoji typeahead for finding what one is looking for.
2018-07-13 21:18:02 +05:30
Cynthia Lin 7d9281fa6c simplebar: Add simplebar dependency for help page scrolling. 2018-07-11 20:04:55 +05:30
Anupam Dagar a1a36e7971 emails: Add preheader to "new login" emails.
This makes it easier to see what the new login was about in the previews.

Fixes: #9866.
2018-07-05 12:44:54 +02:00
Armaan Ahluwalia 54d3d8e8b3 webpack: Transition app.js to be compiled by webpack.
This commit moves all files previously under the 'app' bundle in
the Django pipeline to being compiled by webpack under the 'app'
entry point. In the process, it moves assets under the app entry
to a file called app.js that consumes all relevant css and js files.

This commit also edits the webpack config to be able to expose certain
variables for third party libraries that are currently required by
some modules. This is bad coding form and should be refactored to
requiring whatever dependencies a module may have; we're just
deferring that to the future to simplify the series of transitions we
need to do here. The variable exposure is done using expose-loader in
webpack.

The app/index.html template is edited to override the newly introduced
'commonjs' block in the base template. This is done as a temporary
measure so as not to disrupt other pages on the app during the transition.

It also fixes the value of the 'this' context that was being inferred
as window by third party libraries. This is done using imports-loader
in the webpack config.  This is also messy and probably isn't how we
want things to work long term.
2018-07-05 11:03:08 +02:00
Eeshan Garg 3f40240017 pypi_packages: Upgrade to release 0.5.1. 2018-07-01 08:17:17 -07:00
Yashashvi Dave 2b478268f3 third: Add sortablejs library. 2018-07-01 02:05:00 -07:00
Shubham Padia c72d38df56 yarn: Remove fuzzysearch from dependencies. 2018-06-26 20:38:47 -04:00
Yago González 93e75b9c84 requirements: Upgrade yamole to 2.1.5. 2018-06-23 12:42:11 +02:00
Aditya Bansal 998fb038a9 django-pylibmc: Add dependancy and use as cache backend for memcached. 2018-06-16 05:24:19 -07:00
Eeshan Garg 4446b76427 pypi: Upgrade to release 0.5.0. 2018-06-07 08:31:21 -07:00
Raymond Akornor 8f7af5fde5 install-node: Upgrade to yarn 1.7.0. 2018-06-05 08:52:55 -07:00
Eeshan Garg 2d6d8fe5d8 zulip_bots: Upgrade to latest commit.
This upgrade fixes a dependency conflict between one of the bots
in zulip_bots and the main repo.
2018-06-04 16:20:00 -07:00
Yago González 9f98002b76 tests: Allow testing our REST API against the OpenAPI docs. 2018-06-01 12:03:31 -07:00
Vishnu Ks 97e143e987 requirements: Upgrade Django to 1.11.13. 2018-05-28 09:06:10 -07:00
Yago González f84c9b919b api docs: Read parameters and response fixtures from OpenAPI files. 2018-05-26 22:49:55 +02:00
Vishnu Ks e458da5327 requirements: Upgrade PyJWT to 1.6.4. 2018-05-24 15:16:02 -07:00
Vishnu Ks d0037631e3 version: Bump PROVISION_VERSION for py packages upgrade. 2018-05-24 11:31:48 -07:00
Armaan Ahluwalia 1525e92058 logging/errors/webpack: Improve error stack traces with blueslip.
This commit improves the output that blueslip produces while
showing error stack traces on the front-end. This is done by
using a library called error-stack-parser to format the stack
traces.

This commit also edits the webpack config to use a different
devtool setting since the previous one did not support sourcemaps
within stack traces. It also removes a plugin that was obviated
by this change.
2018-05-22 14:36:41 -07:00
Priyank Patel d140838831 provision: Bump provision version after dependencies upgrade. 2018-05-20 11:18:59 -07:00
Tim Abbott 7208ad0ae0 version: Update for Zulip Server 1.8.1 release. 2018-05-07 16:01:41 -07:00
neiljp (Neil Pilgrim) 202063e030 requirements: Upgrade mypy to 0.600. 2018-05-06 20:55:21 -07:00
Tim Abbott d1b9e06cb4 version: Bump PROVISION_VERSION for Python package updates. 2018-05-03 20:44:00 -07:00
Aditya Bansal 9629be689b populate_db: Add a web public stream to dev database.
We flip the Stream "Rome" to be a web public stream. Also we add
attribute is_web_public in various stream dicts and in the
bulk_create_streams function of bulk_create.py responsible for
default stream creation in dev environment.
2018-05-02 15:23:33 -07:00
Vishnu Ks c80babdf95 provision: Increment version for generating email templates.
Email source files were modified for adding translation tags
but version was not incremented.
2018-05-01 17:57:46 +05:30
Eeshan Garg 7d14ce2cb6 pypi packaging: Upgrade to release 0.4.6.
As a part of the upgrade, we had to update our API tests in
zerver/lib/api_test_helpers.
2018-04-27 14:50:25 -07:00
Armaan Ahluwalia bda9f3e3ea CSS: Install dependencies for SCSS Integration.
This commit installs the dependencies required for SCSS compilation
by webpack.
2018-04-27 08:52:37 -07:00
neiljp (Neil Pilgrim) f5ec2639b7 requirements: Update mypy to 0.590. 2018-04-25 08:58:55 -07:00
Tim Abbott 47d53107a1 version: Bump PROVISION_VERSION for webpack upgrade. 2018-04-23 15:51:56 -07:00
Eeshan Garg 1e217ed4e4 pypi: Upgrade to release 0.4.5. 2018-04-23 15:20:02 -07:00
Priyank Patel 29f04511c0 node_tests: Use nyc instead of deceprated istanbul package. 2018-04-23 15:27:16 -04:00
Tim Abbott 1ec276b3a8 release: Update version strings following 1.8.0 release. 2018-04-17 17:12:20 -07:00
Tim Abbott a6a5636a32 Release Zulip server 1.8.0. 2018-04-17 16:59:07 -07:00
Eeshan Garg 6782f2b76a pypi: Upgrade to release 0.4.4.
This is the latest release after pip 10 was launched.
2018-04-16 16:04:46 -07:00
Tim Abbott 041fd802b7 Revert "yarn: Revert back to v0.27.5."
This reverts commit d4b88e86cc.
2018-04-12 11:37:00 -07:00
Tim Abbott fea65cbb01 Update changelog and versions following Zulip Server 1.7.2. 2018-04-12 10:20:49 -07:00
Priyank d4b88e86cc yarn: Revert back to v0.27.5.
Revert yarn version back due to some issue with new version that causes
permission issues in ~/.config/yarn directory.

Related discussion: https://chat.zulip.org/#narrow/stream/21-provision-help/topic/EACCES.3A.20permission.20denied.2C.20scandir.20'.2Fhome.2Fvagrant.2F.2Econfig.2Fya
2018-04-12 10:18:59 -07:00
Shubham Padia 55619cbe70 browser-support: Add string.prototype.endswith polyfill.
String.prototype.endsWith is not supported in ie11.
Adds string.prototype.endswith package to dependencies and places
it at `common` entry point in webpack.assets.json.
2018-04-11 15:40:57 -07:00
Vishnu Ks 132754f2ef requirements: Downgrade pika to 0.11.0.
Downgrading as issue #8466 is not fixed yet.
2018-04-11 09:31:10 -07:00
Priyank ee078c372f install-node: Upgrade node, yarn, and nvm.
node -> v8.9.4
yarn -> 1.5.1
nvm -> 0.33.8

Also updates a test in timerender.js which depends on time
provided by node which is now changed in newer release.

Some changes have been made in circeci script, we just create ~/.config
directory and chown it to circleci user so installing new version of yarn
does not cause any ci failure on circleci during provision.
2018-04-09 13:56:48 -07:00
Shubham Padia b906562f22 browser-support: Add string.prototype.startsWith polyfill.
Fixes #8944.
Adds string.prototype.startswith package to dependencies and places
it at `common` entry point in webpack.assets.json. As common.js is
loaded on all code paths first, there is no need to place this package
into other entry points.
2018-04-09 12:16:19 -07:00
Priyank 3413faee14 provision: Bump provision version after clipboard package update.
The provision wasn't bumped in https://github.com/zulip/zulip/pull/8984
which caused some js exceptions.
2018-04-06 11:12:35 -04:00
Eeshan Garg 96e01c0d27 pypi: Upgrade to release 0.4.3. 2018-04-03 18:11:25 -07:00