Commit Graph

3511 Commits

Author SHA1 Message Date
Mateusz Mandera 0c2cc41d2e CVE-2019-18933: Fix insecure account creation via social authentication.
A bug in Zulip's new user signup process meant that users who
registered their account using social authentication (e.g. GitHub or
Google SSO) in an organization that also allows password
authentication could have their personal API key stolen by an
unprivileged attacker, allowing nearly full access to the user's
account.

Zulip versions between 1.7.0 and 2.0.6 were affected.

This commit fixes the original bug and also contains a database
migration to fix any users with corrupt `password` fields in the
database as a result of the bug.

Out of an abundance of caution (and to protect the users of any
installations that delay applying this commit), the migration also
resets the API keys of any users where Zulip's logs cannot prove the
user's API key was not previously stolen via this bug.  Resetting
those API keys will be inconvenient for users:

* Users of the Zulip mobile and terminal apps whose API keys are reset
  will be logged out and need to login again.
* Users using their personal API keys for any other reason will need
  to re-fetch their personal API key.

We discovered this bug internally and don't believe it was disclosed
prior to our publishing it through this commit.  Because the algorithm
for determining which users might have been affected is very
conservative, many users who were never at risk will have their API
keys reset by this migration.

To avoid this on self-hosted installations that have always used
e.g. LDAP authentication, we skip resetting API keys on installations
that don't have password authentication enabled.  System
administrators on installations that used to have email authentication
enabled, but no longer do, should temporarily enable EmailAuthBackend
before applying this migration.

The migration also records which users had their passwords or API keys
reset in the usual RealmAuditLog table.
2019-11-21 10:23:37 -08:00
Anders Kaseorg f9f104a4f8 js: Automatically convert var to let and const in more files.
This commit was automatically generated by `tools/lint --only=eslint
--fix`, after an `.eslintrc.json` change.

A half dozen files were removed from the changes by tabbott pending
further work to ensure we avoid breaking valuable PRs with merge
conflicts.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-20 14:10:47 -08:00
Tim Abbott 4e421ebe12 scripts: Move inline-email-css from tools to scripts.
We'll be soon documenting a production workflow that involves using
it, and that means it needs to live under scripts/ (since tools/ isn't
present in release tarballs).
2019-11-15 17:39:42 -08:00
Vishnu Ks 00455df7f9 tests: Use admin client for curl examples test. 2019-11-15 15:53:31 -08:00
Vishnu Ks bb5463b29a tests: Run test_generated_curl_examples_for_success after test_the_api.
Upcoming changes in test_generated_curl_examples_for_success modifies
various data of iago user heavily. So it's much easier to run
test_the_api initially than making various changes in tests of
test_the_api function.
2019-11-15 15:53:31 -08:00
Anders Kaseorg 0ce4de0421 lint: Run mypy with --no-error-summary.
This suppresses the mypy message “Success: no issues found in 1085
source files” or “Found 1 error in 1 file (checked 1085 source files)”
in the output of lint.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 13:26:02 -08:00
Anders Kaseorg 2c4101dbc5 dependencies: Upgrade simplebar from 4.2.3 to 5.0.7.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 12:46:29 -08:00
Anders Kaseorg 40f4ead738 mypy: Upgrade from 0.720 to 0.730.
Fixes #13269.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 12:38:45 -08:00
Anders Kaseorg 43bba63549 upgrade-python-dependencies: Remove.
This tool doesn’t match our current workflow for Python requirements
upgrades as of commit ec9bf6576a (#13213).
It also has a type error with mypy 0.730, which would be easily fixable,
but removing it is easier.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 12:38:45 -08:00
Anders Kaseorg 1a622f91f9 list-outdated-packages: Remove.
This tool provides no value over `pip list --outdated`.  It also has a
type error with mypy 0.730, which would be easily fixable, but
removing it is easier.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 12:38:45 -08:00
Anders Kaseorg cafac83676 request: Tighten type checking on REQ.
Then, find and fix a predictable number of previous misuses.

With a small change by tabbott to preserve backwards compatibility for
sending `yes` for the `forged` field.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 12:35:55 -08:00
Anders Kaseorg 5e89a7d646 install-shellcheck: Upgrade ShellCheck from 0.6.0 to 0.7.0.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-11 16:26:31 -08:00
Anders Kaseorg fffef412bc dependencies: Upgrade to-markdown 3.1.1 to turndown 5.0.3.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-11 16:26:31 -08:00
David Rosa b041948132 docs: Reorganize auth and migrations subsystems.
- Moves "Authentication in the development environment" from subsystems
to "development/authentication.md".
- Moves "Renumbering migrations" to a section within "Schema migrations".
2019-11-07 09:42:36 -08:00
Anders Kaseorg 0584a7938f tsconfig: Move to top level.
This way, webpack.config.ts is type checked.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-04 18:12:11 -08:00
Anders Kaseorg de4685441c typescript: Type webpack.config.ts correctly.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-04 18:12:11 -08:00
Vishnu KS 8e9ccdf376 tests: Remove get-raw-message from curl test exclude_list. 2019-10-30 16:49:26 -07:00
Anders Kaseorg 042c558bb3 eslint: Enable sort-imports rule.
I figure we should enable this before we have lots of imports.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-30 13:10:25 -07:00
Anders Kaseorg 2bbcd6ab34 bundles: Factor out portico bundle.
This adds translations.js to the digest entrypoint.  Presumably that’s
fine.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:53:15 -07:00
Anders Kaseorg fda70a5458 templates: Use just one webpack entry point per page.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:53:15 -07:00
Anders Kaseorg f245fcf408 webpack: Enable code splitting and deduplication.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:53:15 -07:00
Anders Kaseorg d1a3bf424a requirements: Use webpack4 fork of django-webpack-loader.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:53:15 -07:00
Anders Kaseorg ee9a6071fd 5xx.html: Build with webpack.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:53:15 -07:00
Anders Kaseorg f077508c6d test-backend, run-casper: Remove proxy vars instead of setting to "".
Setting http_proxy to the empty string caused a spurious error from
xmlsec:

>>> import os
>>> os.environ["http_proxy"] = ""
>>> import xmlsec
error : Unknown IO error

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:47:45 -07:00
Anders Kaseorg c3bf7c6889 webpack: Consolidate production and development plugin configuration.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:39:34 -07:00
Anders Kaseorg 27fac76da8 styles: Move media queries into the files they override.
Webpack code splitting will make the inclusion order of CSS files less
obvious, and we need to guarantee that these rules follow the rules
they override.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:39:17 -07:00
Mateusz Mandera 28dd1b34f2 auth: Refactor social login rendering.
login_context now gets the social_backends list through
get_social_backend_dicts and we  move display_logo customization
to backend class definition.

This prepares for easily adding multiple IdP support in SAML
authentication - there will be a social_backend dict for each configured
IdP, also allowing display_name and icon customization per IdP.
2019-10-28 15:06:26 -07:00
Anders Kaseorg 453919bbc7 version: Only let `git describe` match tags beginning with a digit.
This will let us use other tags for things like `@zulip/shared`
versions.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-24 14:54:45 -07:00
David Rosa bd3a8f1e96 docs: Merge front-end-build-process with html-css.
This merges the "Static asset pipeline" article as a new section
at the bottom of the "HTML and CSS" article.
2019-10-23 13:08:36 -07:00
Vishnu KS ec955f8f78 support: Show confirmation links in search.
Fixes #13060 #12784
2019-10-21 16:56:50 -07:00
Vishnu KS 62a8e378a6 support: Refactor realm details into a seperate template. 2019-10-21 16:52:46 -07:00
Vishnu KS f2242c1487 docs: Make update-notification-settings use curl example system. 2019-10-21 17:10:35 +05:30
Greg Price a63786ac0d shared: Set up a way to share some frontend code with the mobile app.
This adds the general machinery required, and sets it up for the file
`typing_status.js` as a first use case.

Co-authored-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-17 16:48:23 -07:00
Anders Kaseorg 173c9cee42 frontend_tests: Switch from ts-node to Babel; add rewire-ts plugin.
This will let tests rewrite TypeScript/ES6 module bindings that would
otherwise be read-only.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-17 16:48:23 -07:00
Tim Abbott fa26475fcf docs: Deal with a few broken links. 2019-10-15 14:03:53 -07:00
Anders Kaseorg dfd9ace7fa requirements: Add back future.
It happens that commonmark, python-jose, and python-twitter don’t
actually use future on Python 3, and moto uses aws-xray-sdk in such a
way that it doesn’t use future, but this was a weird game to be
playing just to remove one dependency, and it caused CI failures after
new releases of future, so let’s just include it.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-11 14:04:35 -07:00
Vishnu KS 1585ad7bf4 mattermost: Add support for exporting DMs and huddles. 2019-10-10 16:37:03 -07:00
Tim Abbott f8928182cf run-dev: Set HTTP header to show we're proxing from port 9991.
Previously, while Django code that relied on EXTERNAL_HOST and other
settings would know the Zulip server is actually on port 9991, the
upcoming Django SAML code in python-social-auth would end up detecting
a port of 9992 (the one the Django server is actually listening on).
We fix this using X-Forwarded-Port.
2019-10-08 17:53:09 -07:00
Rishi Gupta 360cd7f147 remote data: Send RealmAuditLog data. 2019-10-08 17:27:29 -07:00
Ray Kraesig 1180299fe4 provisioning: fix up bash profile creation
The base Ubuntu image includes no `.bash_profile`. Creating one will
prevent the existing support scripts in `~/.profile` from loading.

Instead, append our setup code to [whichever script Bash will load
first][bash-startup].

[bash-startup]: https://www.gnu.org/software/bash/manual/html_node/Bash-Startup-Files.html#Bash-Startup-Files
2019-10-08 17:17:55 -07:00
Anders Kaseorg 3d876aacc6 requirements: Use pip-compile --generated-hashes for better security.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-06 15:21:18 -07:00
Anders Kaseorg 9f09d4fcfd update-locked-requirements: Print some progress information.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-06 15:21:18 -07:00
Anders Kaseorg 0af22dad18 flush-memcached: Respect MEMCACHED_LOCATION; handle errors.
Fixes #13238.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-01 16:05:55 -07:00
Tim Abbott 23c48764b5 README: Improve links to coverage/CI to point to master.
Apparently, the CircleCI and Codecov links (and the Codecov badge)
weren't pointing specifically at master, so they'd sometimes show
state from the lastest push to a pull request, which isn't a
reasonable way to advertise whether the project's build is passing.
2019-10-01 15:31:55 -07:00
Anders Kaseorg 8d38f0593b run-dev: Disable Tornado response decompression.
Apparently Tornado decompresses gzip responses by default.  Worse, it
fails to adjust the Content-Length header when it does.

https://github.com/tornadoweb/tornado/issues/2743

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-24 15:51:06 -07:00
Tim Abbott a84bb89bdc scripts: Move mobile i18n code out scripts/.
Like other code that is only used in the development environment, this
doesn't belong in scripts/.
2019-09-24 12:57:42 -07:00
Tim Abbott 27b3c1a312 provision: Move install-shellcheck to proper directory.
Scripts in scripts/ should be exclusively code that used in
production, and this isn't.
2019-09-24 12:54:33 -07:00
Anders Kaseorg d368aaad10 requirements: Generate pip.txt from pip.in like the other *.txt files.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 13:23:58 -07:00
Anders Kaseorg 92b42573fb requirements: Compile thumbor requirements with pip-tools on Python 2.
The reason that `pip-tools` running on Python 3 didn’t detect the
right requirements for `thumbor` on Python 2 is simply that some of
them are conditional on the Python version.

As for the requirements that had been manually added as a workaround:
`backports-abc` and `singledispatch` are now correctly detected, while
`backports.ssl-match-hostname` was vendored into `urllib3` some time
ago and `certifi` is no longer necessary.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 11:56:36 -07:00
Anders Kaseorg fc0554fcb8 emoji: Finish removing leftover code from banned Emoji sets.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-19 12:08:52 -07:00