Commit Graph

3520 Commits

Author SHA1 Message Date
Steve Howell ee3e488e02 js: Extract FoldDict class.
We have ~5 years of proof that we'll probably never
extend Dict with more options.

Breaking the classes into makes both a little faster
(no options to check), and we remove some options
in FoldDict that are never used (from/from_array).

A possible next step is to fine-tune the Dict to use
Map internally.

Note that the TypeScript types for FoldDict are now
more specific (requiring string keys).  Of course,
this isn't really enforced until we convert other
modules to TS.
2020-01-03 17:19:50 -08:00
Mateusz Mandera d691c249db api: Return a JsonableError if API key of invalid format is given. 2020-01-03 16:56:42 -08:00
Anders Kaseorg 1f8d21af74 test-install: Use lxc-destroy -f instead of lxc-stop.
Fixes this error after rebooting the host:

$ sudo ./destroy-all  -f
zulip-install-bionic-41MM2
lxc-stop: zulip-install-bionic-41MM2: tools/lxc_stop.c: main: 191 zulip-install-bionic-41MM2 is not running

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-12-18 03:48:39 -08:00
Anders Kaseorg 9b5f9858fb test-install: Run lxc-attach with --clear-env.
The host environment variables (especially PATH) should not be allowed
to pollute the test and could interfere with it.

This allows test-install to run on a NixOS host.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-12-18 03:48:39 -08:00
Anders Kaseorg ab211c7acf lint: Tell ShellCheck to look for sourced files at relative paths.
This uses the new -P option of ShellCheck 0.7.0.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-12-18 03:48:02 -08:00
Tim Abbott 3dc19344c0 renumber-migrations: Exclude migration 0209.
Otherwise, this tries to renumber the already merged duplicate 0209s,
which isn't helpful.
2019-12-11 16:08:32 -08:00
Mateusz Mandera 586a5facc9 models: Add is_realm_admin and is_guest setters.
Fixes #13452.

The migration from UserProfile.is_realm_admin/UserProfile.is_guest in
e10361a832 broke our LDAP-based support
for setting a user's role via LDAP properties, which relied on setting
those fields.  Because the django-auth-ldap feature powering that only
supports booleans (and in any case, we don't want to expose constants
like `ROLE_REALM_ADMINISTRATOR` to the LDAP configuration interface),
it makes sense to provide setters for these legacy fields for
backwards-compatibility.

We lint against using these setters directly in Zulip's codebase
directly.  The issue with using these is that when changing user's
.role we want to create appropriate RealmAuditLog entries and send
events. This isn't possible when using these setters - the log entries
and events should be created if the role change in the UserProfile is
actually save()-ed to the database - and on the level of the setter
function, it's not known whether the change will indeed be saved.

It would have to be somehow figured out on the level of post_save
signal handlers, but it doesn't seem like a good design to have such
complexity there, for the sake of setters that generally shouldn't be
used anyway - because we prefer the do_change_is_* functions.

The purpose of this change is narrowly to handle use cases like the
setattr on these boolean properties.
2019-12-09 11:54:01 -08:00
Anders Kaseorg f15cfc00ef test-documentation: Invoke sphinx-build via make instead of directly.
This lets us pick up the -W option from docs/Makefile.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-12-02 18:39:33 -08:00
Anders Kaseorg fdaca911e4 docs: Replace Makefile based on output of newer sphinx-quickstart.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-12-02 18:39:33 -08:00
Mateusz Mandera 0c2cc41d2e CVE-2019-18933: Fix insecure account creation via social authentication.
A bug in Zulip's new user signup process meant that users who
registered their account using social authentication (e.g. GitHub or
Google SSO) in an organization that also allows password
authentication could have their personal API key stolen by an
unprivileged attacker, allowing nearly full access to the user's
account.

Zulip versions between 1.7.0 and 2.0.6 were affected.

This commit fixes the original bug and also contains a database
migration to fix any users with corrupt `password` fields in the
database as a result of the bug.

Out of an abundance of caution (and to protect the users of any
installations that delay applying this commit), the migration also
resets the API keys of any users where Zulip's logs cannot prove the
user's API key was not previously stolen via this bug.  Resetting
those API keys will be inconvenient for users:

* Users of the Zulip mobile and terminal apps whose API keys are reset
  will be logged out and need to login again.
* Users using their personal API keys for any other reason will need
  to re-fetch their personal API key.

We discovered this bug internally and don't believe it was disclosed
prior to our publishing it through this commit.  Because the algorithm
for determining which users might have been affected is very
conservative, many users who were never at risk will have their API
keys reset by this migration.

To avoid this on self-hosted installations that have always used
e.g. LDAP authentication, we skip resetting API keys on installations
that don't have password authentication enabled.  System
administrators on installations that used to have email authentication
enabled, but no longer do, should temporarily enable EmailAuthBackend
before applying this migration.

The migration also records which users had their passwords or API keys
reset in the usual RealmAuditLog table.
2019-11-21 10:23:37 -08:00
Anders Kaseorg f9f104a4f8 js: Automatically convert var to let and const in more files.
This commit was automatically generated by `tools/lint --only=eslint
--fix`, after an `.eslintrc.json` change.

A half dozen files were removed from the changes by tabbott pending
further work to ensure we avoid breaking valuable PRs with merge
conflicts.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-20 14:10:47 -08:00
Tim Abbott 4e421ebe12 scripts: Move inline-email-css from tools to scripts.
We'll be soon documenting a production workflow that involves using
it, and that means it needs to live under scripts/ (since tools/ isn't
present in release tarballs).
2019-11-15 17:39:42 -08:00
Vishnu Ks 00455df7f9 tests: Use admin client for curl examples test. 2019-11-15 15:53:31 -08:00
Vishnu Ks bb5463b29a tests: Run test_generated_curl_examples_for_success after test_the_api.
Upcoming changes in test_generated_curl_examples_for_success modifies
various data of iago user heavily. So it's much easier to run
test_the_api initially than making various changes in tests of
test_the_api function.
2019-11-15 15:53:31 -08:00
Anders Kaseorg 0ce4de0421 lint: Run mypy with --no-error-summary.
This suppresses the mypy message “Success: no issues found in 1085
source files” or “Found 1 error in 1 file (checked 1085 source files)”
in the output of lint.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 13:26:02 -08:00
Anders Kaseorg 2c4101dbc5 dependencies: Upgrade simplebar from 4.2.3 to 5.0.7.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 12:46:29 -08:00
Anders Kaseorg 40f4ead738 mypy: Upgrade from 0.720 to 0.730.
Fixes #13269.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 12:38:45 -08:00
Anders Kaseorg 43bba63549 upgrade-python-dependencies: Remove.
This tool doesn’t match our current workflow for Python requirements
upgrades as of commit ec9bf6576a (#13213).
It also has a type error with mypy 0.730, which would be easily fixable,
but removing it is easier.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 12:38:45 -08:00
Anders Kaseorg 1a622f91f9 list-outdated-packages: Remove.
This tool provides no value over `pip list --outdated`.  It also has a
type error with mypy 0.730, which would be easily fixable, but
removing it is easier.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 12:38:45 -08:00
Anders Kaseorg cafac83676 request: Tighten type checking on REQ.
Then, find and fix a predictable number of previous misuses.

With a small change by tabbott to preserve backwards compatibility for
sending `yes` for the `forged` field.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 12:35:55 -08:00
Anders Kaseorg 5e89a7d646 install-shellcheck: Upgrade ShellCheck from 0.6.0 to 0.7.0.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-11 16:26:31 -08:00
Anders Kaseorg fffef412bc dependencies: Upgrade to-markdown 3.1.1 to turndown 5.0.3.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-11 16:26:31 -08:00
David Rosa b041948132 docs: Reorganize auth and migrations subsystems.
- Moves "Authentication in the development environment" from subsystems
to "development/authentication.md".
- Moves "Renumbering migrations" to a section within "Schema migrations".
2019-11-07 09:42:36 -08:00
Anders Kaseorg 0584a7938f tsconfig: Move to top level.
This way, webpack.config.ts is type checked.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-04 18:12:11 -08:00
Anders Kaseorg de4685441c typescript: Type webpack.config.ts correctly.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-04 18:12:11 -08:00
Vishnu KS 8e9ccdf376 tests: Remove get-raw-message from curl test exclude_list. 2019-10-30 16:49:26 -07:00
Anders Kaseorg 042c558bb3 eslint: Enable sort-imports rule.
I figure we should enable this before we have lots of imports.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-30 13:10:25 -07:00
Anders Kaseorg 2bbcd6ab34 bundles: Factor out portico bundle.
This adds translations.js to the digest entrypoint.  Presumably that’s
fine.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:53:15 -07:00
Anders Kaseorg fda70a5458 templates: Use just one webpack entry point per page.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:53:15 -07:00
Anders Kaseorg f245fcf408 webpack: Enable code splitting and deduplication.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:53:15 -07:00
Anders Kaseorg d1a3bf424a requirements: Use webpack4 fork of django-webpack-loader.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:53:15 -07:00
Anders Kaseorg ee9a6071fd 5xx.html: Build with webpack.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:53:15 -07:00
Anders Kaseorg f077508c6d test-backend, run-casper: Remove proxy vars instead of setting to "".
Setting http_proxy to the empty string caused a spurious error from
xmlsec:

>>> import os
>>> os.environ["http_proxy"] = ""
>>> import xmlsec
error : Unknown IO error

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:47:45 -07:00
Anders Kaseorg c3bf7c6889 webpack: Consolidate production and development plugin configuration.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:39:34 -07:00
Anders Kaseorg 27fac76da8 styles: Move media queries into the files they override.
Webpack code splitting will make the inclusion order of CSS files less
obvious, and we need to guarantee that these rules follow the rules
they override.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:39:17 -07:00
Mateusz Mandera 28dd1b34f2 auth: Refactor social login rendering.
login_context now gets the social_backends list through
get_social_backend_dicts and we  move display_logo customization
to backend class definition.

This prepares for easily adding multiple IdP support in SAML
authentication - there will be a social_backend dict for each configured
IdP, also allowing display_name and icon customization per IdP.
2019-10-28 15:06:26 -07:00
Anders Kaseorg 453919bbc7 version: Only let `git describe` match tags beginning with a digit.
This will let us use other tags for things like `@zulip/shared`
versions.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-24 14:54:45 -07:00
David Rosa bd3a8f1e96 docs: Merge front-end-build-process with html-css.
This merges the "Static asset pipeline" article as a new section
at the bottom of the "HTML and CSS" article.
2019-10-23 13:08:36 -07:00
Vishnu KS ec955f8f78 support: Show confirmation links in search.
Fixes #13060 #12784
2019-10-21 16:56:50 -07:00
Vishnu KS 62a8e378a6 support: Refactor realm details into a seperate template. 2019-10-21 16:52:46 -07:00
Vishnu KS f2242c1487 docs: Make update-notification-settings use curl example system. 2019-10-21 17:10:35 +05:30
Greg Price a63786ac0d shared: Set up a way to share some frontend code with the mobile app.
This adds the general machinery required, and sets it up for the file
`typing_status.js` as a first use case.

Co-authored-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-17 16:48:23 -07:00
Anders Kaseorg 173c9cee42 frontend_tests: Switch from ts-node to Babel; add rewire-ts plugin.
This will let tests rewrite TypeScript/ES6 module bindings that would
otherwise be read-only.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-17 16:48:23 -07:00
Tim Abbott fa26475fcf docs: Deal with a few broken links. 2019-10-15 14:03:53 -07:00
Anders Kaseorg dfd9ace7fa requirements: Add back future.
It happens that commonmark, python-jose, and python-twitter don’t
actually use future on Python 3, and moto uses aws-xray-sdk in such a
way that it doesn’t use future, but this was a weird game to be
playing just to remove one dependency, and it caused CI failures after
new releases of future, so let’s just include it.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-11 14:04:35 -07:00
Vishnu KS 1585ad7bf4 mattermost: Add support for exporting DMs and huddles. 2019-10-10 16:37:03 -07:00
Tim Abbott f8928182cf run-dev: Set HTTP header to show we're proxing from port 9991.
Previously, while Django code that relied on EXTERNAL_HOST and other
settings would know the Zulip server is actually on port 9991, the
upcoming Django SAML code in python-social-auth would end up detecting
a port of 9992 (the one the Django server is actually listening on).
We fix this using X-Forwarded-Port.
2019-10-08 17:53:09 -07:00
Rishi Gupta 360cd7f147 remote data: Send RealmAuditLog data. 2019-10-08 17:27:29 -07:00
Ray Kraesig 1180299fe4 provisioning: fix up bash profile creation
The base Ubuntu image includes no `.bash_profile`. Creating one will
prevent the existing support scripts in `~/.profile` from loading.

Instead, append our setup code to [whichever script Bash will load
first][bash-startup].

[bash-startup]: https://www.gnu.org/software/bash/manual/html_node/Bash-Startup-Files.html#Bash-Startup-Files
2019-10-08 17:17:55 -07:00
Anders Kaseorg 3d876aacc6 requirements: Use pip-compile --generated-hashes for better security.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-06 15:21:18 -07:00