Commit Graph

1885 Commits

Author SHA1 Message Date
Anders Kaseorg 74c17bf94a python: Convert more percent formatting to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus.

Now including %d, %i, %u, and multi-line strings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
Anders Kaseorg 1ed2d9b4a0 logging: Use logging.exception and exc_info for unexpected exceptions.
logging.exception() and logging.debug(exc_info=True),
etc. automatically include a traceback.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
Anders Kaseorg bff3dcadc8 email: Migrate to new Python ≥ 3.3 email API.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 20:24:06 -07:00
Dinesh d308c12ae2 auth: Add native flow support for Apple authentication.
Overrides some of internal functions of python-social-auth
to handle native flow.

Credits to Mateusz Mandera for the overridden functions.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2020-06-14 16:20:12 -07:00
Anders Kaseorg 6358339581 backends: Convert ExternalAuthDataDict to Python 3.6 style.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:37:25 -07:00
Anders Kaseorg 365fe0b3d5 python: Sort imports with isort.
Fixes #2665.

Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.

Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start.  I expect this change will increase pressure for us to split
those files, which isn't a bad thing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:45:32 -07:00
Anders Kaseorg 69730a78cc python: Use trailing commas consistently.
Automatically generated by the following script, based on the output
of lint with flake8-comma:

import re
import sys

last_filename = None
last_row = None
lines = []

for msg in sys.stdin:
    m = re.match(
        r"\x1b\[35mflake8    \|\x1b\[0m \x1b\[1;31m(.+):(\d+):(\d+): (\w+)", msg
    )
    if m:
        filename, row_str, col_str, err = m.groups()
        row, col = int(row_str), int(col_str)

        if filename == last_filename:
            assert last_row != row
        else:
            if last_filename is not None:
                with open(last_filename, "w") as f:
                    f.writelines(lines)

            with open(filename) as f:
                lines = f.readlines()
            last_filename = filename
        last_row = row

        line = lines[row - 1]
        if err in ["C812", "C815"]:
            lines[row - 1] = line[: col - 1] + "," + line[col - 1 :]
        elif err in ["C819"]:
            assert line[col - 2] == ","
            lines[row - 1] = line[: col - 2] + line[col - 1 :].lstrip(" ")

if last_filename is not None:
    with open(last_filename, "w") as f:
        f.writelines(lines)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 16:04:12 -07:00
Anders Kaseorg 0e5946ee5a python: Add noqa comments for the specific star imports we allow.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 15:36:43 -07:00
Anders Kaseorg 491fb3c3cf urls: Avoid bare tuples.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 15:36:43 -07:00
Kartik Srivastava 8c39ddfd28 api: Add GET /users/{user_id}/subscription/{stream_id} endpoint.
This new endpoint returns a 'user' dictionary which, as of now,
contains a single key 'is_subscribed' with a boolean value that
represents whether the user with the given 'user_id' is subscribed
to the stream with the given 'stream_id'.

Fixes #14966.
2020-06-10 17:59:14 -07:00
Anders Kaseorg 67e7a3631d python: Convert percent formatting to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 15:02:09 -07:00
Anders Kaseorg 6480deaf27 python: Convert more "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, with more
restrictions patched out.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 14:48:09 -07:00
Anders Kaseorg 5837560a82 dev_settings: Use Python 2-compatible annotation.
This file is imported from zthumbor.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 02:11:55 -07:00
Anders Kaseorg 3de3386c21 Revert "dev_settings: Deduplicate DEPLOY_ROOT definition."
This reverts commit b8acd82b72.
2020-06-10 02:11:00 -07:00
Anders Kaseorg 3a15e4b51d settings: Extract most of test_settings to test_extra_settings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:29:50 -07:00
Anders Kaseorg 5546762bd9 settings: Extract computed settings to computed_settings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:29:50 -07:00
Anders Kaseorg 826ca3bfdf settings: Move LDAP_DEACTIVATE_NON_MATCHING_USERS default to default_settings.
Tweaked by tabbott to fix an incorrect translation to ONLY_SSO.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:27:55 -07:00
Anders Kaseorg 24d320f245 dev_settings: Move prod_settings_template import to configured_settings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:20:42 -07:00
Anders Kaseorg 34b03dcf8a settings: Move DEBUG default to default_settings.
This is overridden in the prod_settings_template.py generated by
build-release-tarball.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:20:42 -07:00
Anders Kaseorg c45962785c settings: Group {default,prod,dev}_settings as configured_settings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:20:42 -07:00
Anders Kaseorg 18eba18df7 settings: Move bot email defaults to default_settings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:15:34 -07:00
Anders Kaseorg 8ba25960e7 settings: Move NAGIOS_BOT_HOST default to default_settings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:15:34 -07:00
Anders Kaseorg 92a58e9ce0 settings: Move IS_DEV_DROPLET default to default_settings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:15:34 -07:00
Anders Kaseorg 9824405842 dev_settings: Set ZULIP_ADMINISTRATOR.
Fixes this error in the dev environment:

$ ./manage.py checkconfig
Error: You must set ZULIP_ADMINISTRATOR in /etc/zulip/settings.py.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:15:34 -07:00
Anders Kaseorg b8acd82b72 dev_settings: Deduplicate DEPLOY_ROOT definition.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:15:34 -07:00
Anders Kaseorg 59dec63da3 settings: Remove unused slack_importer_test_settings.
Commit 3ebd30120c (#8345) removed the
corresponding test.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:15:34 -07:00
Mateusz Mandera 90f3882512 settings: Fix duplicate logging from zulip.auth logger.
The logger defines a full list of handlers, meaning propagate=False is
needed, to avoid the log line propagating further up the logging tree
and getting logged multiple times by the duplicated handlers.
2020-06-09 17:42:51 -07:00
Dinesh dc90d54b08 auth: Add Sign in with Apple support.
This implementation overrides some of PSA's internal backend
functions to handle `state` value with redis as the standard
way doesn't work because of apple sending required details
in the form of POST request.

Includes a mixin test class that'll be useful for testing
Native auth flow.

Thanks to Mateusz Mandera for the idea of using redis and
other important work on this.

Documentation rewritten by tabbott.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2020-06-09 17:29:35 -07:00
Tim Abbott 7b8ba5ebd9 docs: Update most remaining references to zulipchat.com.
In some cases, the cleanest tweak is to replace references to the
domain with Zulip Cloud, the product.
2020-06-08 18:10:45 -07:00
Tim Abbott 95be7dcbab portico: Remove mystery hunt landing page.
This was written by Rishi for a very brief purpose a few years ago,
and it doesn't serve much purpose now other than to be a place we
update in code sweeps.
2020-06-08 18:10:45 -07:00
Tim Abbott 71078adc50 docs: Update URLs to use https://zulip.com.
We're migrating to using the cleaner zulip.com domain, which involves
changing all of our links from ReadTheDocs and other places to point
to the cleaner URL.
2020-06-08 18:10:45 -07:00
Dinesh 04671f40ae auth: Add auth_name attribute for non external auth backends.
This commit adds `name` attribute for the backends that do not
have them.

This is just a kind of prep commit in case if we want to use
`self.logger.xxxx()` in the future which is dependent on the
`name` attribute. But right now these logging calls aren't used
anywhere in those backends.
2020-06-08 17:42:07 -07:00
Dinesh e22ca9483e logging: Log exception name also in auth_complete.
`HTTPError` has empty string for `str(HTTPError())`. Logging it
as it is would not be much helpful. So, this commits adds code
to log the name of error also.
2020-06-08 17:42:07 -07:00
Dinesh d30f11888a logging: Set up a different logger for each backend.
Adds a top-level logger in `settings.LOGGING` `zulip.auth`
with the default handlers `DEFAULT_ZULIP_HANDLERS` and
an extra hanlder that writes to `/var/log/zulip/auth.log`.

Each auth backend uses it's own logger, `self.logger` which
is in form 'zulip.auth.<backend name>'.

This way it's clear which auth backend generated the log
and is easier to look for all authentication logs in one file.

Besides the above mentioned changes, `name` attribute is added to
`ZulipAuthMixin` so that these logging kind of calls wouldn't raise
any issues when logging is tried in a class without `name` attribute.

Also in the tests we use a new way to check if logger calls are made
i.e. we use `assertLogs` to test if something is logged.

Thanks to Mateusz Mandera for the idea of having a seperate logger
for auth backends and suggestion of using `assertLogs`.
2020-06-08 17:42:07 -07:00
Anders Kaseorg 8e4f22c184 auth: Require algorithms setting for JWT auth.
Calling jwt.decode without an algorithms list raises a
DeprecationWarning.  This is for protecting against
symmetric/asymmetric key confusion attacks.

This is a backwards-incompatible configuration change.

Fixes #15207.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 16:22:25 -07:00
Anders Kaseorg 8dd83228e7 python: Convert "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, but with the
NamedTuple changes reverted (see commit
ba7906a3c6, #15132).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 15:31:20 -07:00
Anders Kaseorg 4d04fa3118 compose: Rewrite Zoom video call integration to use OAuth.
This reimplements our Zoom video call integration to use an OAuth
application.  In addition to providing a cleaner setup experience,
especially on zulipchat.com where the server administrators can have
done the app registration already, it also fixes the limitation of the
previous integration that it could only have one call active at a time
when set up with typical Zoom API keys.

Fixes #11672.

Co-authored-by: Marco Burstein <marco@marco.how>
Co-authored-by: Tim Abbott <tabbott@zulipchat.com>
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2020-06-03 16:39:12 -07:00
Anders Kaseorg 7a53da7526 capitalization: Fix OAuth capitalization.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-03 16:39:12 -07:00
Sahil Batra 77d4be56a4
users: Modify do_create_user and create_user to accept role.
We change do_create_user and create_user to accept
role as a parameter instead of 'is_realm_admin' and 'is_guest'.
These changes are done to minimize data conversions between
role and boolean fields.
2020-06-02 16:11:36 -07:00
Mateusz Mandera 3e7fc17788 auth: Delegate RemoteUser SSO to browser when using the desktop app. 2020-06-02 13:00:17 -07:00
Dinesh 9f2fb3a48a auth: Move `standard_relay_params` of SAMLAuthBackend to `SocialAuthMixin`.
Earlier this `standard_relay_params` was used only for SAML auth,
now "Sign in with Apple" also requires this to store those params
in session for reuse. So, this acts as a prep commit for "Sign in
with Apple" auth support.
2020-05-29 16:02:53 -07:00
Tim Abbott d32362e53e settings: Support free_trial_days in the development environment. 2020-05-26 21:31:07 -07:00
Vishnu KS 8784539d53 free trial: Send users to /upgrade after realm creation. 2020-05-26 17:01:32 -07:00
Anders Kaseorg cf923b49d3 python: Remove extra pass statements with autoflake.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-26 11:43:40 -07:00
Anders Kaseorg 8bcdf4ca97 python: Convert TypedDict declarations to Python 3.6 style.
A subset of the diff generated by pyupgrade --py36-plus
--keep-percent-format.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-26 11:43:40 -07:00
Mateusz Mandera b66dc9de50 saml: Support IdP-initiated SSO. 2020-05-25 16:09:30 -07:00
Mateusz Mandera 13c3eaf086 rate_limit: Restrict tornado backend to explicitly specified domains.
This will protect us in case of some kinds of bugs that could allow
making requests such as password authentication attempts to tornado.
Without restricting the domains to which the in-memory backend can
be applied, such bugs would lead to attackers having multiple times
larger rate limits for these sensitive requests.
2020-05-25 15:40:00 -07:00
Vishnu KS 8b3eb9a846 landing: Add option to show custom navbar message. 2020-05-24 17:34:03 -07:00
Dinesh 288921d425 auth: Log when a user tries to login with deactivated account.
Helps to see if users are often trying to login with deactived
accounts.
A use case: Trackdown whether any deactivated bot users are still
trying to access the API.

This implementation adds a new key `inactive_user_id`
to `return_data` in the function `is_user_active` which
check if a `user_profile` is active. This reduces the effort
of getting `user_id` just before logging.

Modified tests for line coverage.
2020-05-24 17:27:19 -07:00
Mateusz Mandera dac4a7a70b saml: Figure out the idp from SAMLResponse.
Instead of plumbing the idp to /complete/saml/ through redis, it's much
more natural to just figure it out from the SAMLResponse, because the
information is there.
This is also a preparatory step for adding IdP-initiated sign in, for
which it is important for /complete/saml/ to be able to figure out which
IdP the request is coming from.
2020-05-24 16:40:28 -07:00