Commit Graph

287 Commits

Author SHA1 Message Date
Stefan Weil d2fa058cc1
text: Fix some typos (most of them found and fixed by codespell).
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2020-03-27 17:25:56 -07:00
Mateusz Mandera 8069133f88 rate_limit: Remove __str__ methods of RateLimitedObjects.
These were clunky from the start and are no longer used, as keys are now
used directly for logging purposes.
2020-03-22 18:42:35 -07:00
Mateusz Mandera 4e9f77a6c4 rate_limit: Adjust keys() of some RateLimitedObjects.
type().__name__ is sufficient, and much readable than type(), so it's
better to use the former for keys.
We also make the classes consistent in forming the keys in the format
type(self).__name__:identifier and adjust logger.warning and statsd to
take advantage of that and simply log the key().
2020-03-22 18:42:35 -07:00
Mateusz Mandera 2c6b1fd575 rate_limit: Rename key_fragment() method to key(). 2020-03-22 18:42:35 -07:00
Mateusz Mandera 9c9f8100e7 rate_limit: Add the concept of RateLimiterBackend.
This will allow easily swapping and using various implementations of
rate-limiting, and separate the implementation logic from
RateLimitedObjects.
2020-03-22 18:42:35 -07:00
Mateusz Mandera 85df6201f6 rate_limit: Move functions called by external code to RateLimitedObject. 2020-03-22 18:42:35 -07:00
Dinesh 3de646d2cf auth: Improve GitHub auth with multiple verified emails.
The previous model for GitHub authentication was as follows:

* If the user has only one verified email address, we'll generally just log them in to that account
* If the user has multiple verified email addresses, we will always
  prompt them to pick which one to use, with the one registered as
  "primary" in GitHub listed at the top.

This change fixes the situation for users going through a "login" flow
(not registration) where exactly one of the emails has an account in
the Zulip oragnization -- they should just be logged in.

Fixes part of #12638.
2020-03-22 17:31:01 -07:00
Mateusz Mandera f5e95c4fc1 requirements: Bump python-social-auth version.
We had a bunch of ugly hacks to monkey patch things due to upstream
being temporarily unmaintained and not merging PRs. Now the project is
active again and the fixes have been merged and included in the latest
version - so we clean up all that code.
2020-03-18 12:14:31 -07:00
Steve Howell 62fb3ad801 refactor: Move validate_email_not_already_in_realm.
We move this to email_validation.py.
2020-03-06 11:53:22 -08:00
Steve Howell 4f5b07a7e6 refactor: Extract zerver/lib/email_validation.py. 2020-03-06 11:53:22 -08:00
Steve Howell 57f1aa722c refactor: Rename validate_email_for_realm.
Now called:

    validate_email_not_already_in_realm

We have a separate validation function that
makes sure that the email fits into a realm's
domain scheme, and we want to avoid naming
confusion here.
2020-03-06 11:53:22 -08:00
Mateusz Mandera e506dbcdad auth: Monkey patch a fix for Github deprecation notice spam.
This is a way to monkey-patch a fix for
https://github.com/python-social-auth/social-core/issues/430
Changes from this commit should be reverted once the issue is fixed
upstream.
2020-03-03 15:51:40 -08:00
Mateusz Mandera efb3065158 social_auth: Take user to find_account if invalid subdomain is given.
This allows to also clean up some code that's not really useful.
2020-02-27 17:27:55 -08:00
Mateusz Mandera 98a7cd85a2 auth: Fix return type annotations on social auth pipeline functions. 2020-02-27 17:27:55 -08:00
Mateusz Mandera 98ae2fb940 auth: Remove redundant realm argument to finish_desktop_flow.
finish_desktop_flow is called with the assumption that the request
successfully proved control over the user_profile and generates a
special link to log into the user_profile account. There's no reason to
pass the realm param, as user_profile.realm can be assumed.
2020-02-24 12:39:48 -08:00
Mateusz Mandera bf0f1274fa saml: Make the bad idp param KeyError log message more verbose.
Original idea was that KeyError was only going to happen there in case
of user passing bad input params to the endpoint, so logging a generic
message seemed sufficient. But this can also happen in case of
misconfiguration, so it's worth logging more info as it may help in
debugging the configuration.
2020-02-20 14:49:27 -08:00
Mateusz Mandera bde495db87 registration: Add support for mobile and desktop flows.
This makes it possible to create a Zulip account from the mobile or
desktop apps and have the end result be that the user is logged in on
their mobile device.

We may need small changes in the desktop and/or mobile apps to support
this.

Closes #10859.
2020-02-12 11:22:16 -08:00
Dinesh 4304d5f8db auth: Add support for GitLab authentication.
With some tweaks by tabbott to the documentation and comments.

Fixes #13694.
2020-02-11 13:54:17 -08:00
Mateusz Mandera bc062e1c4d auth: Give all backend authenticate() optional request argument.
This is required for our migration to Django 2.2. authenticate()
definitions need to have that starting with Django 2.1.
rate_limit_auth needs to be adjusted to expect the request in the first
positional argument instead of a kwarg.
2020-02-04 12:46:53 -08:00
Mateusz Mandera 7b34853328 rate_limiter: Rename authenticate domain to authenticate_by_username.
This prepares for adding authenticate_by_ip_address.
2020-02-02 19:15:13 -08:00
Mateusz Mandera 5f94ea3d54 auth: Rate limit username+password authenticate() calls.
This applies rate limiting (through a decorator) of authenticate()
functions in the Email and LDAP backends - because those are the ones
where we check user's password.
The limiting is based on the username that the authentication is
attempted for - more than X attempts in Y minutes to a username is not
permitted.

If the limit is exceeded, RateLimited exception will be raised - this
can be either handled in a custom way by the code that calls
authenticate(), or it will be handled by RateLimitMiddleware and return
a json_error as the response.
2020-02-02 19:15:13 -08:00
Mateusz Mandera d5786ee67a auth: Ensure only one of mobile and desktop otps in validate_otp_params.
validate_otp_params needs to be moved to backends.py, because as of this
commit it'll be used both there and in views.auth - and import from
views.auth to backends.py causes circular import issue.
2020-02-02 19:14:40 -08:00
Mateusz Mandera 92c16996fc redis_utils: Require key_format argument in get_dict_from_redis. 2020-01-26 21:40:15 -08:00
Mateusz Mandera 859bde482d auth: Implement server side of desktop_flow_otp. 2020-01-26 21:40:15 -08:00
Mateusz Mandera af2c4a9735 redis: Extract put_dict_in_redis and get_dict_from_redis helpers. 2020-01-23 16:24:07 -08:00
Mateusz Mandera 27b9eafcac social_auth: Set is_signup=False if the user is already signed up.
Because of how login_or_register_remote_user code is structured, this
doesn't change how the flow will go, but it's not a clean use of
login_or_register_remote_user to call it with is_signup=True if sign up
shouldn't actually happen - and may be fragile when refactoring
login_or_register_remote_user.
2020-01-23 16:24:07 -08:00
Tim Abbott 4562949f43 default stream groups: Fix buggy LDAP behavior.
With LDAP authentication, we don't currently have a good way to
support the default stream groups feature.

The old behavior was just to assume a user select every default stream
group, which seems wrong; since we didn't prompt the user about these,
we should just ignore the feature.
2020-01-14 14:50:18 -08:00
Mateusz Mandera e559447f83 ldap: Improve logging.
Our ldap integration is quite sensitive to misconfigurations, so more
logging is better than less to help debug those issues.
Despite the following docstring on ZulipLDAPException:

"Since this inherits from _LDAPUser.AuthenticationFailed, these will
be caught and logged at debug level inside django-auth-ldap's
authenticate()"

We weren't actually logging anything, because debug level messages were
ignored due to our general logging settings. It is however desirable to
log these errors, as they can prove useful in debugging configuration
problems. The django_auth_ldap logger can get fairly spammy on debug
level, so we delegate ldap logging to a separate file
/var/log/zulip/ldap.log to avoid spamming server.log too much.
2019-12-28 10:47:08 -08:00
Mateusz Mandera a180f01e6b ldap: Use a cleaner super().authenticate() call in ZulipLDAPAuthBackend. 2019-12-28 10:47:08 -08:00
Tim Abbott 02169c48cf ldap: Fix bad interaction between EMAIL_ADDRESS_VISIBILITY and LDAP sync.
A block of LDAP integration code related to data synchronization did
not correctly handle EMAIL_ADDRESS_VISIBILITY_ADMINS, as it was
accessing .email, not .delivery_email, both for logging and doing the
mapping between email addresses and LDAP users.

Fixes #13539.
2019-12-15 22:59:02 -08:00
Mateusz Mandera 6dbd2b5fc3 auth: Merge RemoteUserBackend into external_authentication_methods.
We register ZulipRemoteUserBackend as an external_authentication_method
to make it show up in the corresponding field in the /server_settings
endpoint.

This also allows rendering its login button together with
Google/Github/etc. leading to us being able to get rid of some of the
code that was handling it as a special case - the js code for plumbing
the "next" value and the special {% if only_sso %} block in login.html.
An additional consequence of the login.html change is that now the
backend will have it button rendered even if it isn't the only backend
enabled on the server.
2019-12-10 20:16:21 +01:00
Mateusz Mandera a842968090 auth: Expand on the external_auth_method abstraction.
This commit builds a more complete concept of an "external
authentication method". Our social backends become a special case of an
external authentication method - but these changes don't change the
actual behavior of social backends, they allow having other backends
(that come from python-social-auth and don't use the social backend
pipeline) share useful code that so far only serviced social backends.
Most importantly, this allows having other backends show up in the
external_authentication_methods field of the /server_settings endpoint,
as well as rendering buttons through the same mechanism as we already
did for social backends.

This moves the creation of dictonaries describing the backend for the
API and button rendering code away into a method, that each backend in
this category is responsible for defining.

To register a backend as an external_authentication_method, it should
subclass ExternalAuthMethod and define its dict_representation
classmethod, and finally use the external_auth_method class decorator to
get added to the EXTERNAL_AUTH_METHODS list.
2019-12-10 20:16:21 +01:00
Mateusz Mandera 67b6179df2 ldap: Fix error while updating a user registered in multiple realms.
Previously, the LDAP code for syncing user data was not
multiple-realm-aware, resulting in errors trying to sync data for an
LDAP user present in multiple realms.

Tweaked by tabbott to add some extended comments.

Fixes #11520.
2019-11-21 11:13:31 -08:00
Mateusz Mandera 06c2161f7e auth: Use zxcvbn to ensure password strength on server side.
For a long time, we've been only doing the zxcvbn password strength
checks on the browser, which is helpful, but means users could through
hackery (or a bug in the frontend validation code) manage to set a
too-weak password.  We fix this by running our password strength
validation on the backend as well, using python-zxcvbn.

In theory, a bug in python-zxcvbn could result in it producing a
different opinion than the frontend version; if so, it'd be a pretty
bad bug in the library, and hopefully we'd hear about it from users,
report upstream, and get it fixed that way. Alternatively, we can
switch to shelling out to node like we do for KaTeX.

Fixes #6880.
2019-11-21 10:23:37 -08:00
Mateusz Mandera 0c2cc41d2e CVE-2019-18933: Fix insecure account creation via social authentication.
A bug in Zulip's new user signup process meant that users who
registered their account using social authentication (e.g. GitHub or
Google SSO) in an organization that also allows password
authentication could have their personal API key stolen by an
unprivileged attacker, allowing nearly full access to the user's
account.

Zulip versions between 1.7.0 and 2.0.6 were affected.

This commit fixes the original bug and also contains a database
migration to fix any users with corrupt `password` fields in the
database as a result of the bug.

Out of an abundance of caution (and to protect the users of any
installations that delay applying this commit), the migration also
resets the API keys of any users where Zulip's logs cannot prove the
user's API key was not previously stolen via this bug.  Resetting
those API keys will be inconvenient for users:

* Users of the Zulip mobile and terminal apps whose API keys are reset
  will be logged out and need to login again.
* Users using their personal API keys for any other reason will need
  to re-fetch their personal API key.

We discovered this bug internally and don't believe it was disclosed
prior to our publishing it through this commit.  Because the algorithm
for determining which users might have been affected is very
conservative, many users who were never at risk will have their API
keys reset by this migration.

To avoid this on self-hosted installations that have always used
e.g. LDAP authentication, we skip resetting API keys on installations
that don't have password authentication enabled.  System
administrators on installations that used to have email authentication
enabled, but no longer do, should temporarily enable EmailAuthBackend
before applying this migration.

The migration also records which users had their passwords or API keys
reset in the usual RealmAuditLog table.
2019-11-21 10:23:37 -08:00
Mateusz Mandera ed40d37e44 ldap: Fix realm_creation=True registration flow.
When creating realm with the ldap backend, the registration flow didn't
properly handle some things - the user wouldn't be set as realm admin,
initial subscriptions and messages weren't created, and the redirect
wasn't happening properly in the case of subdomains.
2019-11-08 14:01:45 -08:00
David Rosa b041948132 docs: Reorganize auth and migrations subsystems.
- Moves "Authentication in the development environment" from subsystems
to "development/authentication.md".
- Moves "Renumbering migrations" to a section within "Schema migrations".
2019-11-07 09:42:36 -08:00
Mateusz Mandera b05a0d0177 social_backends: If no icon is to be displayed, set display_icon to None. 2019-11-05 15:44:07 -08:00
Mateusz Mandera 8edbbe7b3c ldap: Make email search config obligatory without LDAP_APPEND_DOMAIN.
Having to account everywhere for both cases of having and not
having email search configured makes things needlessly complicated.
It's better to make the setting obligatory in configurations other than
LDAP_APPEND_DOMAIN.
2019-11-05 15:25:58 -08:00
Mateusz Mandera 9f330fba8e ldap: Move ZulipLDAPAuthBackendBase.get_or_build_user definition.
This function is inherited by ZulipLDAPUserPopulator and overriden by
ZulipLDAPAuthBackend, so it's more clear to have it simply defined in
ZulipLDAPUserPopulator directly.
2019-11-05 15:25:58 -08:00
Mateusz Mandera 5aded51b73 register: Pre-populate Name in social backend flow.
By adding some additional plumbing (through PreregistrationUser) of the
full_name and an additional full_name_validated option, we
pre-populate the Full Name field in the registration form when coming
through a social backend (google/github/saml/etc.) and potentially skip
the registration form (if the user would have nothing to do there other
than clicking the Confirm button) and just create the account and log
the user in.
2019-11-03 16:15:48 -08:00
Mateusz Mandera a62d084247 social_backends: Rename display_logo to display_icon. 2019-11-03 15:54:05 -08:00
Mateusz Mandera 5a39e70bce social_backends: Remove sort_order from social backend dicts.
These are returned through the API, at the /server_settings
endpoint. It's better to just return the list of dicts with a guarantee
of being sorted in the correct order, than to clutter things with the
sort_order field.
2019-11-03 15:51:49 -08:00
Mateusz Mandera 9d14b50186 auth: Support not using an icon when rendering social login buttons.
Since we were using a placeholder emote for SAML, we change the
defaults to no icon now that it's possible.
2019-10-28 15:14:57 -07:00
Mateusz Mandera b870816a75 saml: Sanity-check configuration in both login and signup codepaths. 2019-10-28 15:11:19 -07:00
Mateusz Mandera 892d25faa1 auth: Change SAML login url scheme, enabling multiple IdP support.
The url scheme is now /accounts/login/social/saml/{idp_name} to initiate
login using the IdP configured under "idp_name" name.

display_name and display_logo (the name and icon to show on the "Log in
with" button) can be customized by adding the apprioprate settings in
the configured IdP dictionaries.
2019-10-28 15:09:42 -07:00
Mateusz Mandera 28dd1b34f2 auth: Refactor social login rendering.
login_context now gets the social_backends list through
get_social_backend_dicts and we  move display_logo customization
to backend class definition.

This prepares for easily adding multiple IdP support in SAML
authentication - there will be a social_backend dict for each configured
IdP, also allowing display_name and icon customization per IdP.
2019-10-28 15:06:26 -07:00
Mateusz Mandera 9532e99800 saml: Give SAMLAuthBackend highest sort_order. 2019-10-28 15:06:26 -07:00
Mateusz Mandera 8c065d1fcd ldap: Ensure django_to_ldap_username returns username that is in ldap.
This changes the way django_to_ldap_username works to make sure the ldap
username it returns actually has a corresponding ldap entry and raise an
exception if that's not possible. It seems to be a more sound approach
than just having it return its best guess - which was the case so far.
Now there is a guarantee that what it returns is the username of an
actual ldap user.

This allows communicating to the registration flow when the email being
registered doesn't belong to ldap, which then will proceed to register
it via the normal email backend flow - finally fixing the bug where you
couldn't register a non-ldap email even with the email backend enabled.

These changes to the behavior of django_to_ldap_username require small
refactorings in a couple of other functions that call it, as well as
adapting some tests to these changes. Finally, additional tests are
added for the above-mentioned registration flow behavior and some
related corner-cases.
2019-10-25 12:14:51 -07:00
Mateusz Mandera 68f4cd1e94 ldap: Extract ldap user -> django username mapping logic to a function.
Fixes #11878

Instead of a confusing mix of django_auth_backed applying
ldap_to_django_username in its internals for one part of the
translation, and then custom logic for grabbing it from the email
attribute of the ldapuser in ZulipLDAPAuthBackend.get_or_build_user
for the second part of the translation,
we put all the logic in a single function user_email_from_ldapuser
which will be used by get_or_build of both ZulipLDAPUserPopulator and
ZulipLDAPAuthBackend.

This, building on the previous commits with the email search feature,
fixes the ldap sync bug from issue #11878.

If we can get upstream django-auth-ldap to merge
https://github.com/django-auth-ldap/django-auth-ldap/pull/154, we'll
be able to go back to using the version of ldap_to_django_username
that accepts a _LDAPUser object.
2019-10-22 16:02:23 -07:00