Commit Graph

4113 Commits

Author SHA1 Message Date
Tim Abbott 6407d0b1f9 push_notifications: Clear PushDeviceToken on API key change.
This includes adding a new endpoint to the push notification bouncer
interface, and code to call it appropriately after resetting a user's
personal API key.

When we add support for a user having multiple API keys, we may need
to add an additional key here to support removing keys associated with
just one client.
2019-11-19 15:37:43 -08:00
Anders Kaseorg 6717daf4a6 test_events: Fix apparent typo.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-19 15:34:05 -08:00
Matheus Melo 39bd565941 test_realm: Automate tests for invalid integer values in Realm.
Fixes #13362.
2019-11-18 15:21:11 -08:00
Matheus Melo 678c3a89d0 validators: Add check_int_in validator.
This is a useful helper for our enum format fields where we want to
only allow a fixed list of integer values.
2019-11-18 15:14:56 -08:00
Tim Abbott 993ed9c2b1 tornado: Remove stale user_profile_email field.
Since years ago, this field hasn't been used for anything other than
some logging that would be better off logging the user ID anyway.

It existed in the first place simply because we weren't passing the
user_profile_id to Tornado at all.
2019-11-15 17:07:52 -08:00
Tim Abbott df90533eef home: Switch invalid pointer logging to by user ID. 2019-11-15 17:07:52 -08:00
Anders Kaseorg 0d20145b93 mypy: Upgrade from 0.730 to 0.740.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 12:38:45 -08:00
Anders Kaseorg 40f4ead738 mypy: Upgrade from 0.720 to 0.730.
Fixes #13269.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 12:38:45 -08:00
Anders Kaseorg cafac83676 request: Tighten type checking on REQ.
Then, find and fix a predictable number of previous misuses.

With a small change by tabbott to preserve backwards compatibility for
sending `yes` for the `forged` field.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 12:35:55 -08:00
Gloria Elston f8855ca179 api: Remove legacy emoji reactions endpoint.
The original/legacy emoji reactions endpoints made use of HTTP PUT and
didn't have an API that could correctly handle situations where the
emoji names change over time.  We stopped using the legacy endpoints
some time ago, so we can remove them now.

This requires straightforward updates to older tests that were still
written against the legacy API.

Fixes #12940.
2019-11-12 13:07:06 -08:00
Tim Abbott ddd1a0eb00 actions: Convert do_delete_messages to take a Realm.
The function only used the user's realm anyway, so this is a cleaner
API.

This should also make it more convenient to permanently delete
messages manually, since one doesn't have to fetch a random user in
the realm in order to delete a message using the management shell.

No functional change.
2019-11-12 12:20:31 -08:00
Tlazypanda 6945ced76f slack import: Map Slack guest users to Zulip guests.
Slack's Single-User Guest and Multi-User Guest users should be
imported as Zulip guests during data import.

Fixes #13255.
2019-11-12 12:12:59 -08:00
Anders Kaseorg cce85f6ec7 dependencies: Upgrade katex from 0.10.2 to 0.11.1.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-11 16:26:31 -08:00
Mateusz Mandera ed40d37e44 ldap: Fix realm_creation=True registration flow.
When creating realm with the ldap backend, the registration flow didn't
properly handle some things - the user wouldn't be set as realm admin,
initial subscriptions and messages weren't created, and the redirect
wasn't happening properly in the case of subdomains.
2019-11-08 14:01:45 -08:00
Mateusz Mandera 071038f111 test_signup: Check that user is admin in RealmCreationTest. 2019-11-08 14:01:45 -08:00
Mateusz Mandera 3daec7783a ldap: Fix development environment configuration.
The state of the FAKELDAP setup for the dev env has fallen behind the
backend changes and updates to fakeldap (which implemented
SCOPE_ONELEVEL searches), as well as having some other minor issues.
This commit restore it to a working state and now all three config modes
work properly.
2019-11-08 14:00:24 -08:00
Tim Abbott 43a965ff19 test_event_queue: Remove a stray print statement. 2019-11-06 16:25:34 -08:00
Mateusz Mandera c228f7ebcb email_notifs: Take full advantage of django_to_ldap_username.
django_to_ldap_username is now able to find the correct ldap username in
every supported type of configuration, so we can remove these
conditionals and use django_to_ldap_username in a straight-forward
manner.
2019-11-05 15:32:00 -08:00
Mateusz Mandera 8edbbe7b3c ldap: Make email search config obligatory without LDAP_APPEND_DOMAIN.
Having to account everywhere for both cases of having and not
having email search configured makes things needlessly complicated.
It's better to make the setting obligatory in configurations other than
LDAP_APPEND_DOMAIN.
2019-11-05 15:25:58 -08:00
Mateusz Mandera bfe800b11a ldap tests: Move test_ldap into test_auth_backends.
These tests belong more in test_auth_backends rather than deserving
their own separate file.
2019-11-05 15:25:58 -08:00
Mateusz Mandera bb3ddb9576 ldap tests: Put django_to_ldap tests in DjangoToLDAPUsernameTests.
test_auth_backends had a few random django_to_ldap_username tests laying
around, they belong in DjangoToLDAPUsernameTests.
2019-11-05 15:25:58 -08:00
Tim Abbott 54e357e154 avatars: Fix gravatar URLs with EMAIL_ADDRESS_VISIBILITY_ADMINS.
Previously, we were using user_profile.email rather than
user_profile.delivery_email in all calculations involving Gravatar
URLs, which meant that all organizations with the new
EMAIL_ADDRESS_VISIBILITY_ADMINS setting enabled had useless gravatars
not based on the `user15@host.domain` type fake email addresses we
generate for the API to refer to users.

The fix is to convert these calculations to use the user's
delivery_email.  Some refactoring is required to ensure the data is
passed through to the parts of the codebase that do the check;
fortunately, our automated tests of schemas are effective in verifying
that the new `sender_delivery_email` field isn't visible to the API.

Fixes #13369.
2019-11-05 12:33:20 -08:00
Tim Abbott fb59f98304 test_events: Fix handling of client_gravatar flag.
Previously, we weren't properly passing through the value of the
client_gravatar flag from the caller, resulting in buggy results if
the caller passed client_gravatar=False to do_test().

We happened to not have any uses of this before, but we're about to
add one.
2019-11-05 12:33:20 -08:00
Tim Abbott 9bf947846e tests: Improve outgoing webhook testing strategy. 2019-11-05 11:49:00 -08:00
Tim Abbott 47cf99a970 realm: Prevent setting invalid enum policy values.
As discussed in the comment, ideally these checks should be added
completely automatically, rather than needing to be manually added
every time we add a new setting.  But hopefully the example code for
all of the similar enums that this provides will at least provide some
help.
2019-11-03 17:11:56 -08:00
Matheus Melo c96762b7a9 settings: Add setting for who can edit user groups.
Fixes #12380.
2019-11-03 16:45:13 -08:00
Mateusz Mandera 849c925dc9 social_auth tests: Test registration with ldap populator enabled.
Finishes testing for the last remaining case and thus closes #13298.
2019-11-03 16:15:49 -08:00
Mateusz Mandera c1095474c6 social_auth tests: Extract some common logic in the long signup tests. 2019-11-03 16:15:49 -08:00
Mateusz Mandera 5aded51b73 register: Pre-populate Name in social backend flow.
By adding some additional plumbing (through PreregistrationUser) of the
full_name and an additional full_name_validated option, we
pre-populate the Full Name field in the registration form when coming
through a social backend (google/github/saml/etc.) and potentially skip
the registration form (if the user would have nothing to do there other
than clicking the Confirm button) and just create the account and log
the user in.
2019-11-03 16:15:48 -08:00
Mateusz Mandera 34a540bacb context: Rename social_backends to external_authentication_methods.
The main purpose of this is to make that name change happen in
/server_settings. external_authentication_methods is a much better, more
descriptive name than social_backends from API perspective.
2019-11-03 15:55:44 -08:00
Mateusz Mandera 5a39e70bce social_backends: Remove sort_order from social backend dicts.
These are returned through the API, at the /server_settings
endpoint. It's better to just return the list of dicts with a guarantee
of being sorted in the correct order, than to clutter things with the
sort_order field.
2019-11-03 15:51:49 -08:00
Mateusz Mandera 339f742578 test_signup: Add test for a non-covered case in accounts_register. 2019-11-03 15:51:19 -08:00
Mateusz Mandera 29314f3195 api: Remove unused /get_auth_backends endpoint.
This legacy endpoint was designed for the original native Zulip mobile
apps, which were deprecated years ago in favor of the React Native
app.

It was replaced by /server_settings for active use years ago, so it's
safe to remove it now.
2019-11-01 12:30:59 -07:00
Rafid Aslam 447f74ae63 Upgrade pika to 1.1.*.
Upgrade pika to 1.1.* and make some changes accordingly
to comply with the new version.

Fixes #12899.
2019-10-29 17:01:12 -07:00
Hemanth V. Alluri c1370547d5 events: Only send bot_type for bots and thus remove the for_api param. 2019-10-29 15:41:35 -07:00
Hemanth V. Alluri 5b3e346369 users: Send custom profile fields with the /profile endpoint. 2019-10-29 15:41:35 -07:00
Mateusz Mandera b870816a75 saml: Sanity-check configuration in both login and signup codepaths. 2019-10-28 15:11:19 -07:00
Mateusz Mandera db29fcbbc4 auth: Add social_backends to /server_settings. 2019-10-28 15:11:19 -07:00
Mateusz Mandera 892d25faa1 auth: Change SAML login url scheme, enabling multiple IdP support.
The url scheme is now /accounts/login/social/saml/{idp_name} to initiate
login using the IdP configured under "idp_name" name.

display_name and display_logo (the name and icon to show on the "Log in
with" button) can be customized by adding the apprioprate settings in
the configured IdP dictionaries.
2019-10-28 15:09:42 -07:00
Mateusz Mandera 8c065d1fcd ldap: Ensure django_to_ldap_username returns username that is in ldap.
This changes the way django_to_ldap_username works to make sure the ldap
username it returns actually has a corresponding ldap entry and raise an
exception if that's not possible. It seems to be a more sound approach
than just having it return its best guess - which was the case so far.
Now there is a guarantee that what it returns is the username of an
actual ldap user.

This allows communicating to the registration flow when the email being
registered doesn't belong to ldap, which then will proceed to register
it via the normal email backend flow - finally fixing the bug where you
couldn't register a non-ldap email even with the email backend enabled.

These changes to the behavior of django_to_ldap_username require small
refactorings in a couple of other functions that call it, as well as
adapting some tests to these changes. Finally, additional tests are
added for the above-mentioned registration flow behavior and some
related corner-cases.
2019-10-25 12:14:51 -07:00
Hemanth V. Alluri 1946692f9a users: Refactor get_members_backend endpoint to use get_raw_user_data.
Modify the get_raw_user_data method for use by the /users API endpoint
and then modify the /users endpoint to use it.
2019-10-23 14:50:26 -07:00
Hemanth V. Alluri 34849d7355 test_events: Fix a small typo for test_realm_bots_admin. 2019-10-23 14:35:32 -07:00
Mateusz Mandera 869b57b7f7 ldap tests: Change TestQueryLDAP tests to use the test ldap directory.
Instead of mocking the _LDAPUser class, these tests can now take
advantage of the test directory that other ldap are using. After these
changes, test_query_email_attr also verifies that query_ldap can
successfully be used to query by user email, if email search is
configured.
2019-10-22 16:03:58 -07:00
Mateusz Mandera 1be2779515 tests: Add ldap_username() and ldap_password() method. 2019-10-22 16:03:58 -07:00
Mateusz Mandera 68f4cd1e94 ldap: Extract ldap user -> django username mapping logic to a function.
Fixes #11878

Instead of a confusing mix of django_auth_backed applying
ldap_to_django_username in its internals for one part of the
translation, and then custom logic for grabbing it from the email
attribute of the ldapuser in ZulipLDAPAuthBackend.get_or_build_user
for the second part of the translation,
we put all the logic in a single function user_email_from_ldapuser
which will be used by get_or_build of both ZulipLDAPUserPopulator and
ZulipLDAPAuthBackend.

This, building on the previous commits with the email search feature,
fixes the ldap sync bug from issue #11878.

If we can get upstream django-auth-ldap to merge
https://github.com/django-auth-ldap/django-auth-ldap/pull/154, we'll
be able to go back to using the version of ldap_to_django_username
that accepts a _LDAPUser object.
2019-10-22 16:02:23 -07:00
Mateusz Mandera 3699fe28f8 ldap: Use email search in django_to_ldap_username.
With this, django_to_ldap_username can take an email and find the ldap
username of the ldap user who has this email - if email search is
configured.

This allows successful authenticate() with ldap email and ldap password,
instead of ldap username. This is especially useful because when
a user wants to fetch their api key, the server attempts authenticate
with user_profile.email - and this used to fail if the user was an ldap
user (because the ldap username was required to authenticate
succesfully). See issue #9277.
2019-10-22 15:57:52 -07:00
Mateusz Mandera fea4d0b2be ldap: Do a proper search for email in email_belongs_to_ldap.
This fixes a collection of bugs surrounding LDAP configurations A and
C (i.e. LDAP_APPEND_DOMAIN=None) with EmailAuthBackend also enabled.

The core problem was that our desired security model in that setting
of requiring LDAP authentication for accounts managed by LDAP was not
implementable without a way to

Now admins can configure an LDAPSearch query that will find if there
are users in LDAP that have the email address and
email_belongs_to_ldap() will take advantage of that - no longer
returning True in response to all requests and thus blocking email
backend authentication.

In the documentation, we describe this as mandatory configuration for
users (and likely will make it so soon in the code) because the
failure modes for this not being configured are confusing.

But making that change is pending work to improve the relevant error
messages.

Fixes #11715.
2019-10-22 15:53:39 -07:00
Vishnu KS e080b42fe5 tests: Move get_stream_id function to ZulipTestCase. 2019-10-21 16:56:48 -07:00
Vishnu KS 23036a9f40 confirmation: Set confirmation object realm attribute in realm reactivation.
The value of realm attribute in confirmation object used to be empty
before. We are not currently using the realm attribute of reactivation
links anywhere. The value of realm stored in content_object is currently
used.
2019-10-21 16:52:46 -07:00
Vishnu KS 139ebf387b support: Pass various realm functions as template context.
We currently have code to calculate the value of realm_icon_url,
admin_emails and default_discount in two diffrent places. With
the addition of showing confirmation links it would become three.
The easiest way to deduplicate the code and make the view cleaner
is by doing the calculations in template. Alternatively one can
write a function that takes users, realms and confirmations as
arguments and sets the value of realm_icon_url, admin_emails and
default_discount appropriately in realm object according to the
type of the confirmation. But that seems more messy than passing
the functions directly to template approach.
2019-10-21 16:52:46 -07:00