Commit Graph

652 Commits

Author SHA1 Message Date
Anders Kaseorg 3ec58fd3d5 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-10-20 18:16:27 -07:00
Lauryn Menard 70ab893d34 urls: Generate narrow links in backend with "channel" operator. 2024-10-11 17:00:23 -07:00
Shubham Padia 12ebd97f1f settings: Add group_creator as default for can_manage_group.
We create an unnamed user group with just the group creator as it's
member when trying to set the default. The pattern I've followed across
most of the acting_user additions is to just put the user declared
somewhere before the check_add_user_group and see if the test passes.
If it does not, then I'll look at what kind of user it needs to be set
to `acting_user`.
2024-10-01 17:35:14 -07:00
Mateusz Mandera 7e1f468f04 saml: Fix exception when syncing missing value to custom profile field.
There was a bug here that would trigger an exception inside
`sync_user_profile_custom_fields`, causing it to get logged with
logging.warning, when an attribute configured for SAML custom profile
field sync was missing from a SAMLResponse or had an empty value.
`sync_user_profile_custom_fields` expects valid values, and None is not
valid.

We could consider a slightly different behavior here instead - when an
attribute is sent with no value in the SAMLResponse, that means the attr
has no value in the IdP's user directory - so perhaps a better behavior
would be to also remove the custom profile field value in Zulip. However
there are two issues with that:

1. It's not necessarily the best behavior, because an organization might
want the "user doesn't have this attribute set at the IdP level" state
to just mean that the user should be free to set the value manually in
Zulip if they wish. And having that value get reset on every login would
then be an issue. The implementation in this commit is consistent with
this philosophy.

2. There's some implementation difficulty - upstream
`self.get_attr(...)`, which we use for reading the attr value from the
SAMLResponse, doesn't distinguish between an attribute being sent with
no value and the attribute not being sent at all - in both cases it
returns None. So we'd need some extra work here with parsing the
SAMLResponse properly, to be able to know when the custom profile field
should get cleared.
2024-08-20 13:57:30 -07:00
Mateusz Mandera 833dce8a13 saml: Add support for syncing user role.
Replace the SOCIAL_AUTH_SYNC_CUSTOM_ATTRS_DICT with
SOCIAL_AUTH_SYNC_ATTRS_DICT, designed to support also regular user attrs
like role or full name (in the future).

Custom attributes can stay configured as they were and will get merged
into SOCIAL_AUTH_SYNC_ATTRS_DICT in computed_settings, or can be
specified in SOCIAL_AUTH_SYNC_ATTRS_DICT directly with "custom__"
prefix.

The role sync is plumbed through to user creation, so users can
immediately be created with their intended role as provided by the IdP
when they're creating their account, even when doing this flow without
an invitiation.
2024-08-20 11:53:24 -07:00
Mateusz Mandera 114f13e0ee auth: Fix re-enabling of SAML/AzureAD in organization settings UI.
This bug was introduced in da9e4e6e54.
validate validate_plan_for_authentication_methods is already called
inside validate_authentication_methods_dict_from_api, conditionally on
settings.BILLING_ENABLED. This additional, redundant call runs
regardless of BILLING_ENABLED, and thus prevents a self-hosted server
from enabling certain backends in the organization settings UI.

The impact of this is limited - in order to encounter this bug, a
self-hosted server would have to first disable the backend in the UI, as
self-hosted realms are created with all backend flags enabled. A backend
doesn't show up in the org settings UI until it is first enabled in
AUTHENTICATION_BACKENDS in settings.py - that's why this is a rare
state. A sequence of steps like this has to be followed to reproduce:
1. Add the backend to AUTHENTICATION_BACKENDS in settings.py.
2. Disable the backend in the org settings UI.
3. Now try to re-enable it, which fails due to the bug.
2024-08-05 21:14:43 -07:00
Mateusz Mandera 3f472ec664 test_auth_backends: Remove copied-and-pasted comments.
These makes no sense and were copied from the block at the top of the
test.
2024-08-05 21:14:43 -07:00
Anders Kaseorg 8843f9f62a tests: Remove deprecated SHA1PasswordHasher.
SHA1PasswordHasher will be removed in Django 5.1.  MD5PasswordHasher
will remain for the purpose of speeding up tests.

Followup to commit ac5161f439 (#29620).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-16 13:06:31 -07:00
Anders Kaseorg e3a191b99b ruff: Fix FURB154 Use of repeated consecutive `global`, `nonlocal`.
This is a preview rule, not yet enabled by default.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-14 13:53:18 -07:00
Anders Kaseorg b96feb34f6 ruff: Fix SIM117 Use a single `with` statement with multiple contexts.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-14 13:48:32 -07:00
Anders Kaseorg 48202389b8 ruff: Bump target-version from py38 to py310.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 0fa5e7f629 ruff: Fix UP035 Import from `collections.abc`, `typing` instead.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 531b34cb4c ruff: Fix UP007 Use `X | Y` for type annotations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg e08a24e47f ruff: Fix UP006 Use `list` instead of `List` for type annotation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Lauryn Menard 673a01ea0c realm-deactivation: Send email to owners as part of deactivation.
Creates a new "realm_deactivated" email that can be sent to realm
owners as part of `do_deactivate_realm`, via a boolean flag,
`email_owners`.

This flag is set to `False` when `do_deactivate_realm` is used for
realm exports or changing a realm's subdomain, so that the active
organization owners are not emailed in those cases.

This flag is optional for the `deactivate_realm` management command,
but as there is no active user passed in that case, then the email
is sent without referencing who deactivated the realm.

It is passed as `True` for the support analytics view, but the email
that is generated does not include information about the support
admin user who completed the request for organization deactivation.

When an active organization owner deactivates the organization, then
the flag is `True` and an email is sent to them as well as any other
active organization owners, with a slight variation in the email text
for those two cases.

Adds specific tests for when `email_owners` is passed as `True`. All
existing tests for other functionality of `do_deactivate_user` pass
the flag as `False`.

Adds `localize` from django.util.formats as a jinja env filter so
that the dates in these emails are internationlized for the owner's
default language setting in the "realm_deactivated" email templates.

Fixes #24685.
2024-06-26 16:48:18 -07:00
Alex Vandiver 0153d6dbcd thumbnailing: Move resizing functions into zerver.lib.thumbnail. 2024-06-20 23:06:08 -04:00
Mateusz Mandera 27c4e46b30 do_deactivate_realm: Add deactivation_reason kwarg.
It's going to be helpful in the future to record the reason for realm
deactivation.
- For information tracking
- For making a distinction between cases where we can allow realm owners
  to reactivate their realm via a self-serve flow (e.g.
  "owner_request") vs where we can't (ToS abuse).
2024-05-19 23:07:28 -07:00
Sahil Batra 7b42c802b1 invites: Add include_realm_default_subscriptions parameter.
This commit adds include_realm_default_subscriptions parameter
to the invite endpoints and the corresponding field in
PreregistrationUser and MultiuseInvite objects. This field will
be used to subscribe the new users to the default streams at the
time of account creation and not to the streams that were default
when sending the invite.
2024-05-14 14:20:07 -07:00
Alya Abbott 5aeeafd39c portico: Improve message for deactivated, invalid and moved orgs. 2024-05-13 12:44:20 -07:00
Vector73 ac4dde24ae realm: Add an alias `realm_url` and deprecate `realm_uri` in the API.
The naming `uri` is deprecated while `url` should be used in order to
satisfy URL standards. For this reason, four endpoints are affected:

* The response content of three endpoints `/server_settings`,
`/register` and `/realm` that contain a field `realm_uri` is
changed to `realm_url`.

* In one of the common fields for all mobile push notifications payloads,
`realm_url` field is now added as an alias to `realm_uri`.

For backwards compatibility, we keep the field `realm_uri` and add
an alias `realm_url`.

Co-authored-by: Junyao Chen <junyao.chen@socitydao.org>
2024-05-08 17:39:15 -07:00
Vector73 8ab526a25a models: Replace realm.uri with realm.url.
In #23380, we are changing all occurrences of uri with url in order to
follow the latest URL standard. Previous PRs #25038 and #25045 has
replaced the occurences of uri that has no direct relation with realm.

This commit changes just the model property, which has no API
compatibility concerns.
2024-05-08 11:12:43 -07:00
Alex Vandiver d863aa56de invites: Lock the realm when determining invitation counts.
This prevents users from hammering the invitation endpoint, causing
races, and inviting more users than they should otherwise be allowed
to.

Doing this requires that we not raise InvitationError when we have
partially succeeded; that behaviour is left to the one callsite of
do_invite_users.

Reported by Lakshit Agarwal (@chiekosec).
2024-05-02 14:23:04 -07:00
Sahil Batra e78d0aacaf tests: Use NamedUserGroup for queries. 2024-04-26 17:03:09 -07:00
Sahil Batra 0ff9aacb96 audit_logs: Set modified_user_group field to NamedUserGroup. 2024-04-26 17:03:09 -07:00
Anders Kaseorg 96fbe060a6 python: Mark regexes as raw strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-26 12:30:31 -07:00
Mateusz Mandera da9e4e6e54 backends: Implementation of restricting certain backends by plan.
Only affects zulipchat, by being based on the BILLING_ENABLED setting.

The restricted backends in this commit are
- AzureAD - restricted to Standard plan
- SAML - restricted to Plus plan, although it was already practically
  restricted due to requiring server-side configuration to be done by us

This restriction is placed upon **enabling** a backend - so
organizations that already have a backend enabled, will continue to be
able to use it. This allows us to make exceptions and enable a backend
for an org manually via the shell, and to grandfather organizations into
keeping the backend they have been relying on.
2024-03-05 11:48:58 -08:00
Anders Kaseorg 570f3dd447 python: Reformat with Ruff formatter.
https://docs.astral.sh/ruff/formatter/

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-02-29 17:07:16 -08:00
Mateusz Mandera 6dd6fc045f realm_settings: Improve authentication_methods param validation.
The endpoint was lacking validation that the authentication_methods dict
submitted by the user made sense. So e.g. it allowed submitting a
nonsense key like NoSuchBackend or modifying the realm's configured
authentication methods for a backend that's not enabled on the server,
which should not be allowed.

Both were ultimately harmless, because:
1. Submitting NoSuchBackend would luckily just trigger a KeyError inside
   the transaction.atomic() block in do_set_realm_authentication_methods
   so it would actually roll back the database changes it was trying to
   make. So this couldn't actually create some weird
   RealmAuthenticationMethod entries.
2. Silently enabling or disabling e.g. GitHub for a realm when GitHub
   isn't enabled on the server doesn't really change anything. And this
   action is only available to the realm's admins to begin with, so
   there's no attack vector here.

test_supported_backends_only_updated wasn't actually testing anything,
because the state it was asserting:
```
        self.assertFalse(github_auth_enabled(realm))
        self.assertTrue(dev_auth_enabled(realm))
        self.assertFalse(password_auth_enabled(realm))
```

matched the desired state submitted to the API...
```
        result = self.client_patch(
            "/json/realm",
            {
                "authentication_methods": orjson.dumps(
                    {"Email": False, "Dev": True, "GitHub": False}
                ).decode()
            },
        )
```

so we just replace it with a new test that tests the param validation.
2024-02-02 17:26:32 -08:00
Mateusz Mandera 80f5963bbc auth: Add a configurable wrapper around authenticate calls. 2024-01-15 12:18:48 -08:00
Mateusz Mandera d3b4cbd182 auth: Add hardening authenticate(use_dummy_backend=True) in do_login.
As explained in the comment, this is to prevent bugs where some strange
combination of codepaths could end up calling do_login without basic
validation of e.g. the subdomain. The usefulness of this will be
extended with the upcoming commit to add the ability to configure custom
code to wrap authenticate() calls in. This will help ensure that some
codepaths don't slip by the mechanism, ending up logging in a user
without the chance for the custom wrapper to run its code.
2024-01-15 12:18:48 -08:00
Mateusz Mandera 56ca307bc1 tests: Delete test_social_auth_backends.
This test is ancient and patches so much that it's almost unreadable,
while being redundant considering we have comprehensive tests via the
SocialAuthBase subclasses. The one missing case was the one with the
backend we disabled. We replace that with a proper
test_social_auth_backend_disabled test in SocialAuthBase.
2024-01-15 12:18:48 -08:00
Anders Kaseorg cd96193768 models: Extract zerver.models.realms.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Anders Kaseorg 45bb8d2580 models: Extract zerver.models.users.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Anders Kaseorg 223b626256 python: Use urlsplit instead of urlparse.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-05 13:03:07 -08:00
Anders Kaseorg 3853fa875a python: Consistently use from…import for urllib.parse.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-05 13:03:07 -08:00
Anders Kaseorg 8a7916f21a python: Consistently use from…import for datetime.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-05 12:01:18 -08:00
Prakhar Pratyush b32950d790 notifications: Revert API changes for push_notifications_enabled.
This commit reverts the API changes in 56ec1c2.
2023-12-01 08:14:14 -08:00
Tim Abbott b71c5746ab notifications: Rename push_notifications_enabled for clarity.
This doesn't actually check if push notifications are working, just
whether there is configuration for them.
2023-11-29 15:06:46 -08:00
Alex Vandiver 61fc838179 python: Switch mocking of timezone_now to time_machine. 2023-11-28 15:10:39 -08:00
Mateusz Mandera 2a65183991 tests: Add test for nocoverage userAccountControl case in ldap auth.
This logic has been #nocoverage since its implementation, but since this
is an authentication codepath, it seems important for it to have a test.
2023-11-24 13:38:23 -08:00
Mateusz Mandera 1800b2c797 ldap: Tweak AUTH_LDAP_ADVANCED_REALM_ACCESS_CONTROL behavior.
The original behavior of this setting was to disable LDAP
authentication for any realms not configured to use it. This was an
arbitrary choice, and its only value was to potentially help catch
typos for users who are lazy about testing their configuration.

Since it makes it a very inconvenient to potentially host multiple
organizations with different LDAP configurations, remove that
behavior.
2023-11-17 14:40:26 -08:00
Sahil Batra deb8431a47 users: Rename get_raw_user_data to get_users_for_api.
This is a prep commit for adding feature of restricting
user access to guests such that we can keep the code
easy to read and understand when that feature is added.
2023-11-08 09:22:26 -08:00
Anders Kaseorg a50eb2e809 mypy: Enable new error explicit-override.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-12 12:28:41 -07:00
Tim Abbott 07e6bcda7c tests: Fix missing setUp calls in ZulipTestCase subclasses.
This may save us from bugs like the ones fixed in
78683c1b9c in the future.
2023-10-12 10:28:04 -07:00
Alex Vandiver d98aff26c9 config_error: Return status code 500. 2023-10-11 17:13:01 -07:00
Alex Vandiver 5506dffa36 config_error: Coalesce redundant dicts. 2023-10-11 17:13:01 -07:00
Simon Michalke b6a25840a1 zerver: Implement ldap group synchronization.
Fixes #9957.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2023-10-05 16:12:52 -07:00
Tim Abbott c6fe799369 i18n: Fix default language for users created via API/LDAP.
This fixes a regression introduced in
9954db4b59, where the realm's default
language would be ignored for users created via API/LDAP/SAML,
resulting in all such users having English as their default language.

The API/LDAP/SAML account creation code paths don't have a request,
and thus cannot pull default language from the user's browser.

We have the `realm.default_language` field intended for this use case,
but it was not being passed through the system.

Rather than pass `realm.default_language` through from each caller, we
make the low-level user creation code set this field, as that seems
more robust to the creation of future callers.
2023-10-01 21:10:13 +02:00
Mateusz Mandera a095c34503 saml: Add OneLogin_Saml2_ValidationError to caught parsing exceptions.
This is an exception that we should be generally catching like the
others, which will give our standard /login/ redirect and proper logging
- as opposed to a 500 if we don't catch.

Addresses directly a bug we occurred in the wild, where a SAMLResponse
was submitted without issuers specified in a valid way, causing this
exception. The added test tests this specific type of scenario.
2023-09-27 10:32:18 -07:00
Anders Kaseorg 2665a3ce2b python: Elide unnecessary list wrappers.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-13 12:41:23 -07:00