user_groups: Make locks required for updating user group memberships.
**Background**
User groups are expected to comply with the DAG constraint for the
many-to-many inter-group membership. The check for this constraint has
to be performed recursively so that we can find all direct and indirect
subgroups of the user group to be added.
This kind of check is vulnerable to phantom reads which is possible at
the default read committed isolation level because we cannot guarantee
that the check is still valid when we are adding the subgroups to the
user group.
**Solution**
To avoid having another transaction concurrently update one of the
to-be-subgroup after the recursive check is done, and before the subgroup
is added, we use SELECT FOR UPDATE to lock the user group rows.
The lock needs to be acquired before a group membership change is about
to occur before any check has been conducted.
Suppose that we are adding subgroup B to supergroup A, the locking protocol
is specified as follows:
1. Acquire a lock for B and all its direct and indirect subgroups.
2. Acquire a lock for A.
For the removal of user groups, we acquire a lock for the user group to
be removed with all its direct and indirect subgroups. This is the special
case A=B, which is still complaint with the protocol.
**Error handling**
We currently rely on Postgres' deadlock detection to abort transactions
and show an error for the users. In the future, we might need some
recovery mechanism or at least better error handling.
**Notes**
An important note is that we need to reuse the recursive CTE query that
finds the direct and indirect subgroups when applying the lock on the
rows. And the lock needs to be acquired the same way for the addition and
removal of direct subgroups.
User membership change (as opposed to user group membership) is not
affected. Read-only queries aren't either. The locks only protect
critical regions where the user group dependency graph might violate
the DAG constraint, where users are not participating.
**Testing**
We implement a transaction test case targeting some typical scenarios
when an internal server error is expected to happen (this means that the
user group view makes the correct decision to abort the transaction when
something goes wrong with locks).
To achieve this, we add a development view intended only for unit tests.
It has a global BARRIER that can be shared across threads, so that we
can synchronize them to consistently reproduce certain potential race
conditions prevented by the database locks.
The transaction test case lanuches pairs of threads initiating possibly
conflicting requests at the same time. The tests are set up such that exactly N
of them are expected to succeed with a certain error message (while we don't
know each one).
**Security notes**
get_recursive_subgroups_for_groups will no longer fetch user groups from
other realms. As a result, trying to add/remove a subgroup from another
realm results in a UserGroup not found error response.
We also implement subgroup-specific checks in has_user_group_access to
keep permission managing in a single place. Do note that the API
currently don't have a way to violate that check because we are only
checking the realm ID now.
2023-06-17 04:39:52 +02:00
|
|
|
from contextlib import contextmanager
|
2024-05-24 11:24:18 +02:00
|
|
|
from dataclasses import asdict, dataclass
|
2024-04-30 13:15:46 +02:00
|
|
|
from typing import Collection, Dict, Iterable, Iterator, List, Mapping, Optional, TypedDict, Union
|
2020-06-11 00:54:34 +02:00
|
|
|
|
2024-05-30 05:45:38 +02:00
|
|
|
from django.conf import settings
|
2024-04-16 16:05:43 +02:00
|
|
|
from django.db import connection, transaction
|
2024-05-02 05:52:37 +02:00
|
|
|
from django.db.models import F, Prefetch, QuerySet
|
2022-11-21 05:43:03 +01:00
|
|
|
from django.utils.timezone import now as timezone_now
|
2021-04-16 00:57:30 +02:00
|
|
|
from django.utils.translation import gettext as _
|
2021-09-29 02:46:57 +02:00
|
|
|
from django_cte import With
|
2022-09-19 21:48:53 +02:00
|
|
|
from django_stubs_ext import ValuesQuerySet
|
2024-04-16 16:05:43 +02:00
|
|
|
from psycopg2.sql import SQL, Literal
|
2020-06-11 00:54:34 +02:00
|
|
|
|
2024-05-16 17:58:43 +02:00
|
|
|
from zerver.lib.exceptions import JsonableError, PreviousSettingValueMismatchedError
|
2023-10-19 16:50:26 +02:00
|
|
|
from zerver.lib.types import GroupPermissionSetting, ServerSupportedPermissionSettings
|
2022-11-21 05:43:03 +01:00
|
|
|
from zerver.models import (
|
|
|
|
GroupGroupMembership,
|
2024-04-16 16:05:43 +02:00
|
|
|
NamedUserGroup,
|
2022-11-21 05:43:03 +01:00
|
|
|
Realm,
|
|
|
|
RealmAuditLog,
|
2023-10-19 16:50:26 +02:00
|
|
|
Stream,
|
2022-11-21 05:43:03 +01:00
|
|
|
UserGroup,
|
|
|
|
UserGroupMembership,
|
|
|
|
UserProfile,
|
|
|
|
)
|
2023-12-15 01:55:59 +01:00
|
|
|
from zerver.models.groups import SystemGroups
|
2020-06-11 00:54:34 +02:00
|
|
|
|
2017-09-25 09:47:15 +02:00
|
|
|
|
2024-04-30 13:15:46 +02:00
|
|
|
@dataclass
|
|
|
|
class AnonymousSettingGroupDict:
|
|
|
|
direct_members: List[int]
|
|
|
|
direct_subgroups: List[int]
|
|
|
|
|
|
|
|
|
2024-05-16 17:58:43 +02:00
|
|
|
@dataclass
|
|
|
|
class GroupSettingChangeRequest:
|
|
|
|
new: Union[int, AnonymousSettingGroupDict]
|
|
|
|
old: Optional[Union[int, AnonymousSettingGroupDict]] = None
|
|
|
|
|
|
|
|
|
2022-06-20 23:18:50 +02:00
|
|
|
class UserGroupDict(TypedDict):
|
|
|
|
id: int
|
|
|
|
name: str
|
|
|
|
description: str
|
|
|
|
members: List[int]
|
|
|
|
direct_subgroup_ids: List[int]
|
|
|
|
is_system_group: bool
|
2024-05-02 05:52:37 +02:00
|
|
|
can_mention_group: Union[int, AnonymousSettingGroupDict]
|
2022-06-20 23:18:50 +02:00
|
|
|
|
|
|
|
|
user_groups: Make locks required for updating user group memberships.
**Background**
User groups are expected to comply with the DAG constraint for the
many-to-many inter-group membership. The check for this constraint has
to be performed recursively so that we can find all direct and indirect
subgroups of the user group to be added.
This kind of check is vulnerable to phantom reads which is possible at
the default read committed isolation level because we cannot guarantee
that the check is still valid when we are adding the subgroups to the
user group.
**Solution**
To avoid having another transaction concurrently update one of the
to-be-subgroup after the recursive check is done, and before the subgroup
is added, we use SELECT FOR UPDATE to lock the user group rows.
The lock needs to be acquired before a group membership change is about
to occur before any check has been conducted.
Suppose that we are adding subgroup B to supergroup A, the locking protocol
is specified as follows:
1. Acquire a lock for B and all its direct and indirect subgroups.
2. Acquire a lock for A.
For the removal of user groups, we acquire a lock for the user group to
be removed with all its direct and indirect subgroups. This is the special
case A=B, which is still complaint with the protocol.
**Error handling**
We currently rely on Postgres' deadlock detection to abort transactions
and show an error for the users. In the future, we might need some
recovery mechanism or at least better error handling.
**Notes**
An important note is that we need to reuse the recursive CTE query that
finds the direct and indirect subgroups when applying the lock on the
rows. And the lock needs to be acquired the same way for the addition and
removal of direct subgroups.
User membership change (as opposed to user group membership) is not
affected. Read-only queries aren't either. The locks only protect
critical regions where the user group dependency graph might violate
the DAG constraint, where users are not participating.
**Testing**
We implement a transaction test case targeting some typical scenarios
when an internal server error is expected to happen (this means that the
user group view makes the correct decision to abort the transaction when
something goes wrong with locks).
To achieve this, we add a development view intended only for unit tests.
It has a global BARRIER that can be shared across threads, so that we
can synchronize them to consistently reproduce certain potential race
conditions prevented by the database locks.
The transaction test case lanuches pairs of threads initiating possibly
conflicting requests at the same time. The tests are set up such that exactly N
of them are expected to succeed with a certain error message (while we don't
know each one).
**Security notes**
get_recursive_subgroups_for_groups will no longer fetch user groups from
other realms. As a result, trying to add/remove a subgroup from another
realm results in a UserGroup not found error response.
We also implement subgroup-specific checks in has_user_group_access to
keep permission managing in a single place. Do note that the API
currently don't have a way to violate that check because we are only
checking the realm ID now.
2023-06-17 04:39:52 +02:00
|
|
|
@dataclass
|
|
|
|
class LockedUserGroupContext:
|
|
|
|
"""User groups in this dataclass are guaranteeed to be locked until the
|
|
|
|
end of the current transaction.
|
|
|
|
|
|
|
|
supergroup is the user group to have subgroups added or removed;
|
|
|
|
direct_subgroups are user groups that are recursively queried for subgroups;
|
|
|
|
recursive_subgroups include direct_subgroups and their descendants.
|
|
|
|
"""
|
|
|
|
|
2024-04-17 05:45:32 +02:00
|
|
|
supergroup: NamedUserGroup
|
|
|
|
direct_subgroups: List[NamedUserGroup]
|
|
|
|
recursive_subgroups: List[NamedUserGroup]
|
user_groups: Make locks required for updating user group memberships.
**Background**
User groups are expected to comply with the DAG constraint for the
many-to-many inter-group membership. The check for this constraint has
to be performed recursively so that we can find all direct and indirect
subgroups of the user group to be added.
This kind of check is vulnerable to phantom reads which is possible at
the default read committed isolation level because we cannot guarantee
that the check is still valid when we are adding the subgroups to the
user group.
**Solution**
To avoid having another transaction concurrently update one of the
to-be-subgroup after the recursive check is done, and before the subgroup
is added, we use SELECT FOR UPDATE to lock the user group rows.
The lock needs to be acquired before a group membership change is about
to occur before any check has been conducted.
Suppose that we are adding subgroup B to supergroup A, the locking protocol
is specified as follows:
1. Acquire a lock for B and all its direct and indirect subgroups.
2. Acquire a lock for A.
For the removal of user groups, we acquire a lock for the user group to
be removed with all its direct and indirect subgroups. This is the special
case A=B, which is still complaint with the protocol.
**Error handling**
We currently rely on Postgres' deadlock detection to abort transactions
and show an error for the users. In the future, we might need some
recovery mechanism or at least better error handling.
**Notes**
An important note is that we need to reuse the recursive CTE query that
finds the direct and indirect subgroups when applying the lock on the
rows. And the lock needs to be acquired the same way for the addition and
removal of direct subgroups.
User membership change (as opposed to user group membership) is not
affected. Read-only queries aren't either. The locks only protect
critical regions where the user group dependency graph might violate
the DAG constraint, where users are not participating.
**Testing**
We implement a transaction test case targeting some typical scenarios
when an internal server error is expected to happen (this means that the
user group view makes the correct decision to abort the transaction when
something goes wrong with locks).
To achieve this, we add a development view intended only for unit tests.
It has a global BARRIER that can be shared across threads, so that we
can synchronize them to consistently reproduce certain potential race
conditions prevented by the database locks.
The transaction test case lanuches pairs of threads initiating possibly
conflicting requests at the same time. The tests are set up such that exactly N
of them are expected to succeed with a certain error message (while we don't
know each one).
**Security notes**
get_recursive_subgroups_for_groups will no longer fetch user groups from
other realms. As a result, trying to add/remove a subgroup from another
realm results in a UserGroup not found error response.
We also implement subgroup-specific checks in has_user_group_access to
keep permission managing in a single place. Do note that the API
currently don't have a way to violate that check because we are only
checking the realm ID now.
2023-06-17 04:39:52 +02:00
|
|
|
|
|
|
|
|
2023-08-23 00:11:09 +02:00
|
|
|
def has_user_group_access(
|
2024-04-17 05:45:32 +02:00
|
|
|
user_group: NamedUserGroup, user_profile: UserProfile, *, for_read: bool, as_subgroup: bool
|
2023-08-23 00:11:09 +02:00
|
|
|
) -> bool:
|
user_groups: Make locks required for updating user group memberships.
**Background**
User groups are expected to comply with the DAG constraint for the
many-to-many inter-group membership. The check for this constraint has
to be performed recursively so that we can find all direct and indirect
subgroups of the user group to be added.
This kind of check is vulnerable to phantom reads which is possible at
the default read committed isolation level because we cannot guarantee
that the check is still valid when we are adding the subgroups to the
user group.
**Solution**
To avoid having another transaction concurrently update one of the
to-be-subgroup after the recursive check is done, and before the subgroup
is added, we use SELECT FOR UPDATE to lock the user group rows.
The lock needs to be acquired before a group membership change is about
to occur before any check has been conducted.
Suppose that we are adding subgroup B to supergroup A, the locking protocol
is specified as follows:
1. Acquire a lock for B and all its direct and indirect subgroups.
2. Acquire a lock for A.
For the removal of user groups, we acquire a lock for the user group to
be removed with all its direct and indirect subgroups. This is the special
case A=B, which is still complaint with the protocol.
**Error handling**
We currently rely on Postgres' deadlock detection to abort transactions
and show an error for the users. In the future, we might need some
recovery mechanism or at least better error handling.
**Notes**
An important note is that we need to reuse the recursive CTE query that
finds the direct and indirect subgroups when applying the lock on the
rows. And the lock needs to be acquired the same way for the addition and
removal of direct subgroups.
User membership change (as opposed to user group membership) is not
affected. Read-only queries aren't either. The locks only protect
critical regions where the user group dependency graph might violate
the DAG constraint, where users are not participating.
**Testing**
We implement a transaction test case targeting some typical scenarios
when an internal server error is expected to happen (this means that the
user group view makes the correct decision to abort the transaction when
something goes wrong with locks).
To achieve this, we add a development view intended only for unit tests.
It has a global BARRIER that can be shared across threads, so that we
can synchronize them to consistently reproduce certain potential race
conditions prevented by the database locks.
The transaction test case lanuches pairs of threads initiating possibly
conflicting requests at the same time. The tests are set up such that exactly N
of them are expected to succeed with a certain error message (while we don't
know each one).
**Security notes**
get_recursive_subgroups_for_groups will no longer fetch user groups from
other realms. As a result, trying to add/remove a subgroup from another
realm results in a UserGroup not found error response.
We also implement subgroup-specific checks in has_user_group_access to
keep permission managing in a single place. Do note that the API
currently don't have a way to violate that check because we are only
checking the realm ID now.
2023-06-17 04:39:52 +02:00
|
|
|
if user_group.realm_id != user_profile.realm_id:
|
|
|
|
return False
|
|
|
|
|
|
|
|
if as_subgroup:
|
|
|
|
# At this time, we only check for realm ID of a potential subgroup.
|
|
|
|
return True
|
|
|
|
|
2023-06-17 00:25:05 +02:00
|
|
|
if for_read and not user_profile.is_guest:
|
|
|
|
# Everyone is allowed to read a user group and check who
|
|
|
|
# are its members. Guests should be unable to reach this
|
|
|
|
# code path, since they can't access user groups API
|
|
|
|
# endpoints, but we check for guests here for defense in
|
|
|
|
# depth.
|
2023-08-23 00:11:09 +02:00
|
|
|
return True
|
|
|
|
|
2023-06-17 00:25:05 +02:00
|
|
|
if user_group.is_system_group:
|
2023-08-23 00:11:09 +02:00
|
|
|
return False
|
|
|
|
|
2023-06-17 00:25:05 +02:00
|
|
|
group_member_ids = get_user_group_direct_member_ids(user_group)
|
|
|
|
if (
|
|
|
|
not user_profile.is_realm_admin
|
|
|
|
and not user_profile.is_moderator
|
|
|
|
and user_profile.id not in group_member_ids
|
|
|
|
):
|
2023-08-23 00:11:09 +02:00
|
|
|
return False
|
|
|
|
|
|
|
|
return True
|
|
|
|
|
|
|
|
|
|
|
|
def access_user_group_by_id(
|
|
|
|
user_group_id: int, user_profile: UserProfile, *, for_read: bool
|
2024-04-17 05:45:32 +02:00
|
|
|
) -> NamedUserGroup:
|
2023-08-23 00:11:09 +02:00
|
|
|
try:
|
user_groups: Make locks required for updating user group memberships.
**Background**
User groups are expected to comply with the DAG constraint for the
many-to-many inter-group membership. The check for this constraint has
to be performed recursively so that we can find all direct and indirect
subgroups of the user group to be added.
This kind of check is vulnerable to phantom reads which is possible at
the default read committed isolation level because we cannot guarantee
that the check is still valid when we are adding the subgroups to the
user group.
**Solution**
To avoid having another transaction concurrently update one of the
to-be-subgroup after the recursive check is done, and before the subgroup
is added, we use SELECT FOR UPDATE to lock the user group rows.
The lock needs to be acquired before a group membership change is about
to occur before any check has been conducted.
Suppose that we are adding subgroup B to supergroup A, the locking protocol
is specified as follows:
1. Acquire a lock for B and all its direct and indirect subgroups.
2. Acquire a lock for A.
For the removal of user groups, we acquire a lock for the user group to
be removed with all its direct and indirect subgroups. This is the special
case A=B, which is still complaint with the protocol.
**Error handling**
We currently rely on Postgres' deadlock detection to abort transactions
and show an error for the users. In the future, we might need some
recovery mechanism or at least better error handling.
**Notes**
An important note is that we need to reuse the recursive CTE query that
finds the direct and indirect subgroups when applying the lock on the
rows. And the lock needs to be acquired the same way for the addition and
removal of direct subgroups.
User membership change (as opposed to user group membership) is not
affected. Read-only queries aren't either. The locks only protect
critical regions where the user group dependency graph might violate
the DAG constraint, where users are not participating.
**Testing**
We implement a transaction test case targeting some typical scenarios
when an internal server error is expected to happen (this means that the
user group view makes the correct decision to abort the transaction when
something goes wrong with locks).
To achieve this, we add a development view intended only for unit tests.
It has a global BARRIER that can be shared across threads, so that we
can synchronize them to consistently reproduce certain potential race
conditions prevented by the database locks.
The transaction test case lanuches pairs of threads initiating possibly
conflicting requests at the same time. The tests are set up such that exactly N
of them are expected to succeed with a certain error message (while we don't
know each one).
**Security notes**
get_recursive_subgroups_for_groups will no longer fetch user groups from
other realms. As a result, trying to add/remove a subgroup from another
realm results in a UserGroup not found error response.
We also implement subgroup-specific checks in has_user_group_access to
keep permission managing in a single place. Do note that the API
currently don't have a way to violate that check because we are only
checking the realm ID now.
2023-06-17 04:39:52 +02:00
|
|
|
if for_read:
|
2024-04-17 05:45:32 +02:00
|
|
|
user_group = NamedUserGroup.objects.get(id=user_group_id, realm=user_profile.realm)
|
user_groups: Make locks required for updating user group memberships.
**Background**
User groups are expected to comply with the DAG constraint for the
many-to-many inter-group membership. The check for this constraint has
to be performed recursively so that we can find all direct and indirect
subgroups of the user group to be added.
This kind of check is vulnerable to phantom reads which is possible at
the default read committed isolation level because we cannot guarantee
that the check is still valid when we are adding the subgroups to the
user group.
**Solution**
To avoid having another transaction concurrently update one of the
to-be-subgroup after the recursive check is done, and before the subgroup
is added, we use SELECT FOR UPDATE to lock the user group rows.
The lock needs to be acquired before a group membership change is about
to occur before any check has been conducted.
Suppose that we are adding subgroup B to supergroup A, the locking protocol
is specified as follows:
1. Acquire a lock for B and all its direct and indirect subgroups.
2. Acquire a lock for A.
For the removal of user groups, we acquire a lock for the user group to
be removed with all its direct and indirect subgroups. This is the special
case A=B, which is still complaint with the protocol.
**Error handling**
We currently rely on Postgres' deadlock detection to abort transactions
and show an error for the users. In the future, we might need some
recovery mechanism or at least better error handling.
**Notes**
An important note is that we need to reuse the recursive CTE query that
finds the direct and indirect subgroups when applying the lock on the
rows. And the lock needs to be acquired the same way for the addition and
removal of direct subgroups.
User membership change (as opposed to user group membership) is not
affected. Read-only queries aren't either. The locks only protect
critical regions where the user group dependency graph might violate
the DAG constraint, where users are not participating.
**Testing**
We implement a transaction test case targeting some typical scenarios
when an internal server error is expected to happen (this means that the
user group view makes the correct decision to abort the transaction when
something goes wrong with locks).
To achieve this, we add a development view intended only for unit tests.
It has a global BARRIER that can be shared across threads, so that we
can synchronize them to consistently reproduce certain potential race
conditions prevented by the database locks.
The transaction test case lanuches pairs of threads initiating possibly
conflicting requests at the same time. The tests are set up such that exactly N
of them are expected to succeed with a certain error message (while we don't
know each one).
**Security notes**
get_recursive_subgroups_for_groups will no longer fetch user groups from
other realms. As a result, trying to add/remove a subgroup from another
realm results in a UserGroup not found error response.
We also implement subgroup-specific checks in has_user_group_access to
keep permission managing in a single place. Do note that the API
currently don't have a way to violate that check because we are only
checking the realm ID now.
2023-06-17 04:39:52 +02:00
|
|
|
else:
|
2024-04-17 05:45:32 +02:00
|
|
|
user_group = NamedUserGroup.objects.select_for_update().get(
|
user_groups: Make locks required for updating user group memberships.
**Background**
User groups are expected to comply with the DAG constraint for the
many-to-many inter-group membership. The check for this constraint has
to be performed recursively so that we can find all direct and indirect
subgroups of the user group to be added.
This kind of check is vulnerable to phantom reads which is possible at
the default read committed isolation level because we cannot guarantee
that the check is still valid when we are adding the subgroups to the
user group.
**Solution**
To avoid having another transaction concurrently update one of the
to-be-subgroup after the recursive check is done, and before the subgroup
is added, we use SELECT FOR UPDATE to lock the user group rows.
The lock needs to be acquired before a group membership change is about
to occur before any check has been conducted.
Suppose that we are adding subgroup B to supergroup A, the locking protocol
is specified as follows:
1. Acquire a lock for B and all its direct and indirect subgroups.
2. Acquire a lock for A.
For the removal of user groups, we acquire a lock for the user group to
be removed with all its direct and indirect subgroups. This is the special
case A=B, which is still complaint with the protocol.
**Error handling**
We currently rely on Postgres' deadlock detection to abort transactions
and show an error for the users. In the future, we might need some
recovery mechanism or at least better error handling.
**Notes**
An important note is that we need to reuse the recursive CTE query that
finds the direct and indirect subgroups when applying the lock on the
rows. And the lock needs to be acquired the same way for the addition and
removal of direct subgroups.
User membership change (as opposed to user group membership) is not
affected. Read-only queries aren't either. The locks only protect
critical regions where the user group dependency graph might violate
the DAG constraint, where users are not participating.
**Testing**
We implement a transaction test case targeting some typical scenarios
when an internal server error is expected to happen (this means that the
user group view makes the correct decision to abort the transaction when
something goes wrong with locks).
To achieve this, we add a development view intended only for unit tests.
It has a global BARRIER that can be shared across threads, so that we
can synchronize them to consistently reproduce certain potential race
conditions prevented by the database locks.
The transaction test case lanuches pairs of threads initiating possibly
conflicting requests at the same time. The tests are set up such that exactly N
of them are expected to succeed with a certain error message (while we don't
know each one).
**Security notes**
get_recursive_subgroups_for_groups will no longer fetch user groups from
other realms. As a result, trying to add/remove a subgroup from another
realm results in a UserGroup not found error response.
We also implement subgroup-specific checks in has_user_group_access to
keep permission managing in a single place. Do note that the API
currently don't have a way to violate that check because we are only
checking the realm ID now.
2023-06-17 04:39:52 +02:00
|
|
|
id=user_group_id, realm=user_profile.realm
|
|
|
|
)
|
2024-04-17 05:45:32 +02:00
|
|
|
except NamedUserGroup.DoesNotExist:
|
2023-08-23 00:11:09 +02:00
|
|
|
raise JsonableError(_("Invalid user group"))
|
|
|
|
|
user_groups: Make locks required for updating user group memberships.
**Background**
User groups are expected to comply with the DAG constraint for the
many-to-many inter-group membership. The check for this constraint has
to be performed recursively so that we can find all direct and indirect
subgroups of the user group to be added.
This kind of check is vulnerable to phantom reads which is possible at
the default read committed isolation level because we cannot guarantee
that the check is still valid when we are adding the subgroups to the
user group.
**Solution**
To avoid having another transaction concurrently update one of the
to-be-subgroup after the recursive check is done, and before the subgroup
is added, we use SELECT FOR UPDATE to lock the user group rows.
The lock needs to be acquired before a group membership change is about
to occur before any check has been conducted.
Suppose that we are adding subgroup B to supergroup A, the locking protocol
is specified as follows:
1. Acquire a lock for B and all its direct and indirect subgroups.
2. Acquire a lock for A.
For the removal of user groups, we acquire a lock for the user group to
be removed with all its direct and indirect subgroups. This is the special
case A=B, which is still complaint with the protocol.
**Error handling**
We currently rely on Postgres' deadlock detection to abort transactions
and show an error for the users. In the future, we might need some
recovery mechanism or at least better error handling.
**Notes**
An important note is that we need to reuse the recursive CTE query that
finds the direct and indirect subgroups when applying the lock on the
rows. And the lock needs to be acquired the same way for the addition and
removal of direct subgroups.
User membership change (as opposed to user group membership) is not
affected. Read-only queries aren't either. The locks only protect
critical regions where the user group dependency graph might violate
the DAG constraint, where users are not participating.
**Testing**
We implement a transaction test case targeting some typical scenarios
when an internal server error is expected to happen (this means that the
user group view makes the correct decision to abort the transaction when
something goes wrong with locks).
To achieve this, we add a development view intended only for unit tests.
It has a global BARRIER that can be shared across threads, so that we
can synchronize them to consistently reproduce certain potential race
conditions prevented by the database locks.
The transaction test case lanuches pairs of threads initiating possibly
conflicting requests at the same time. The tests are set up such that exactly N
of them are expected to succeed with a certain error message (while we don't
know each one).
**Security notes**
get_recursive_subgroups_for_groups will no longer fetch user groups from
other realms. As a result, trying to add/remove a subgroup from another
realm results in a UserGroup not found error response.
We also implement subgroup-specific checks in has_user_group_access to
keep permission managing in a single place. Do note that the API
currently don't have a way to violate that check because we are only
checking the realm ID now.
2023-06-17 04:39:52 +02:00
|
|
|
if not has_user_group_access(user_group, user_profile, for_read=for_read, as_subgroup=False):
|
2023-06-17 00:25:05 +02:00
|
|
|
raise JsonableError(_("Insufficient permission"))
|
2023-08-23 00:11:09 +02:00
|
|
|
|
2017-11-01 10:04:16 +01:00
|
|
|
return user_group
|
|
|
|
|
2021-02-12 08:19:30 +01:00
|
|
|
|
user_groups: Make locks required for updating user group memberships.
**Background**
User groups are expected to comply with the DAG constraint for the
many-to-many inter-group membership. The check for this constraint has
to be performed recursively so that we can find all direct and indirect
subgroups of the user group to be added.
This kind of check is vulnerable to phantom reads which is possible at
the default read committed isolation level because we cannot guarantee
that the check is still valid when we are adding the subgroups to the
user group.
**Solution**
To avoid having another transaction concurrently update one of the
to-be-subgroup after the recursive check is done, and before the subgroup
is added, we use SELECT FOR UPDATE to lock the user group rows.
The lock needs to be acquired before a group membership change is about
to occur before any check has been conducted.
Suppose that we are adding subgroup B to supergroup A, the locking protocol
is specified as follows:
1. Acquire a lock for B and all its direct and indirect subgroups.
2. Acquire a lock for A.
For the removal of user groups, we acquire a lock for the user group to
be removed with all its direct and indirect subgroups. This is the special
case A=B, which is still complaint with the protocol.
**Error handling**
We currently rely on Postgres' deadlock detection to abort transactions
and show an error for the users. In the future, we might need some
recovery mechanism or at least better error handling.
**Notes**
An important note is that we need to reuse the recursive CTE query that
finds the direct and indirect subgroups when applying the lock on the
rows. And the lock needs to be acquired the same way for the addition and
removal of direct subgroups.
User membership change (as opposed to user group membership) is not
affected. Read-only queries aren't either. The locks only protect
critical regions where the user group dependency graph might violate
the DAG constraint, where users are not participating.
**Testing**
We implement a transaction test case targeting some typical scenarios
when an internal server error is expected to happen (this means that the
user group view makes the correct decision to abort the transaction when
something goes wrong with locks).
To achieve this, we add a development view intended only for unit tests.
It has a global BARRIER that can be shared across threads, so that we
can synchronize them to consistently reproduce certain potential race
conditions prevented by the database locks.
The transaction test case lanuches pairs of threads initiating possibly
conflicting requests at the same time. The tests are set up such that exactly N
of them are expected to succeed with a certain error message (while we don't
know each one).
**Security notes**
get_recursive_subgroups_for_groups will no longer fetch user groups from
other realms. As a result, trying to add/remove a subgroup from another
realm results in a UserGroup not found error response.
We also implement subgroup-specific checks in has_user_group_access to
keep permission managing in a single place. Do note that the API
currently don't have a way to violate that check because we are only
checking the realm ID now.
2023-06-17 04:39:52 +02:00
|
|
|
@contextmanager
|
|
|
|
def lock_subgroups_with_respect_to_supergroup(
|
|
|
|
potential_subgroup_ids: Collection[int], potential_supergroup_id: int, acting_user: UserProfile
|
|
|
|
) -> Iterator[LockedUserGroupContext]:
|
|
|
|
"""This locks the user groups with the given potential_subgroup_ids, as well
|
|
|
|
as their indirect subgroups, followed by the potential supergroup. It
|
|
|
|
ensures that we lock the user groups in a consistent order topologically to
|
|
|
|
avoid unnecessary deadlocks on non-conflicting queries.
|
2022-03-02 11:58:37 +01:00
|
|
|
|
user_groups: Make locks required for updating user group memberships.
**Background**
User groups are expected to comply with the DAG constraint for the
many-to-many inter-group membership. The check for this constraint has
to be performed recursively so that we can find all direct and indirect
subgroups of the user group to be added.
This kind of check is vulnerable to phantom reads which is possible at
the default read committed isolation level because we cannot guarantee
that the check is still valid when we are adding the subgroups to the
user group.
**Solution**
To avoid having another transaction concurrently update one of the
to-be-subgroup after the recursive check is done, and before the subgroup
is added, we use SELECT FOR UPDATE to lock the user group rows.
The lock needs to be acquired before a group membership change is about
to occur before any check has been conducted.
Suppose that we are adding subgroup B to supergroup A, the locking protocol
is specified as follows:
1. Acquire a lock for B and all its direct and indirect subgroups.
2. Acquire a lock for A.
For the removal of user groups, we acquire a lock for the user group to
be removed with all its direct and indirect subgroups. This is the special
case A=B, which is still complaint with the protocol.
**Error handling**
We currently rely on Postgres' deadlock detection to abort transactions
and show an error for the users. In the future, we might need some
recovery mechanism or at least better error handling.
**Notes**
An important note is that we need to reuse the recursive CTE query that
finds the direct and indirect subgroups when applying the lock on the
rows. And the lock needs to be acquired the same way for the addition and
removal of direct subgroups.
User membership change (as opposed to user group membership) is not
affected. Read-only queries aren't either. The locks only protect
critical regions where the user group dependency graph might violate
the DAG constraint, where users are not participating.
**Testing**
We implement a transaction test case targeting some typical scenarios
when an internal server error is expected to happen (this means that the
user group view makes the correct decision to abort the transaction when
something goes wrong with locks).
To achieve this, we add a development view intended only for unit tests.
It has a global BARRIER that can be shared across threads, so that we
can synchronize them to consistently reproduce certain potential race
conditions prevented by the database locks.
The transaction test case lanuches pairs of threads initiating possibly
conflicting requests at the same time. The tests are set up such that exactly N
of them are expected to succeed with a certain error message (while we don't
know each one).
**Security notes**
get_recursive_subgroups_for_groups will no longer fetch user groups from
other realms. As a result, trying to add/remove a subgroup from another
realm results in a UserGroup not found error response.
We also implement subgroup-specific checks in has_user_group_access to
keep permission managing in a single place. Do note that the API
currently don't have a way to violate that check because we are only
checking the realm ID now.
2023-06-17 04:39:52 +02:00
|
|
|
Regardless of whether the user groups returned are used, always call this
|
|
|
|
helper before making changes to subgroup memberships. This avoids
|
|
|
|
introducing cycles among user groups when there is a race condition in
|
|
|
|
which one of these subgroups become an ancestor of the parent user group in
|
|
|
|
another transaction.
|
|
|
|
|
|
|
|
Note that it only does a permission check on the potential supergroup,
|
|
|
|
not the potential subgroups or their recursive subgroups.
|
|
|
|
"""
|
|
|
|
with transaction.atomic(savepoint=False):
|
|
|
|
# Calling list with the QuerySet forces its evaluation putting a lock on
|
|
|
|
# the queried rows.
|
|
|
|
recursive_subgroups = list(
|
|
|
|
get_recursive_subgroups_for_groups(
|
|
|
|
potential_subgroup_ids, acting_user.realm
|
|
|
|
).select_for_update(nowait=True)
|
2023-07-17 22:40:33 +02:00
|
|
|
)
|
user_groups: Make locks required for updating user group memberships.
**Background**
User groups are expected to comply with the DAG constraint for the
many-to-many inter-group membership. The check for this constraint has
to be performed recursively so that we can find all direct and indirect
subgroups of the user group to be added.
This kind of check is vulnerable to phantom reads which is possible at
the default read committed isolation level because we cannot guarantee
that the check is still valid when we are adding the subgroups to the
user group.
**Solution**
To avoid having another transaction concurrently update one of the
to-be-subgroup after the recursive check is done, and before the subgroup
is added, we use SELECT FOR UPDATE to lock the user group rows.
The lock needs to be acquired before a group membership change is about
to occur before any check has been conducted.
Suppose that we are adding subgroup B to supergroup A, the locking protocol
is specified as follows:
1. Acquire a lock for B and all its direct and indirect subgroups.
2. Acquire a lock for A.
For the removal of user groups, we acquire a lock for the user group to
be removed with all its direct and indirect subgroups. This is the special
case A=B, which is still complaint with the protocol.
**Error handling**
We currently rely on Postgres' deadlock detection to abort transactions
and show an error for the users. In the future, we might need some
recovery mechanism or at least better error handling.
**Notes**
An important note is that we need to reuse the recursive CTE query that
finds the direct and indirect subgroups when applying the lock on the
rows. And the lock needs to be acquired the same way for the addition and
removal of direct subgroups.
User membership change (as opposed to user group membership) is not
affected. Read-only queries aren't either. The locks only protect
critical regions where the user group dependency graph might violate
the DAG constraint, where users are not participating.
**Testing**
We implement a transaction test case targeting some typical scenarios
when an internal server error is expected to happen (this means that the
user group view makes the correct decision to abort the transaction when
something goes wrong with locks).
To achieve this, we add a development view intended only for unit tests.
It has a global BARRIER that can be shared across threads, so that we
can synchronize them to consistently reproduce certain potential race
conditions prevented by the database locks.
The transaction test case lanuches pairs of threads initiating possibly
conflicting requests at the same time. The tests are set up such that exactly N
of them are expected to succeed with a certain error message (while we don't
know each one).
**Security notes**
get_recursive_subgroups_for_groups will no longer fetch user groups from
other realms. As a result, trying to add/remove a subgroup from another
realm results in a UserGroup not found error response.
We also implement subgroup-specific checks in has_user_group_access to
keep permission managing in a single place. Do note that the API
currently don't have a way to violate that check because we are only
checking the realm ID now.
2023-06-17 04:39:52 +02:00
|
|
|
# TODO: This select_for_update query is subject to deadlocking, and
|
|
|
|
# better error handling is needed. We may use
|
|
|
|
# select_for_update(nowait=True) and release the locks held by ending
|
|
|
|
# the transaction with a JsonableError by handling the DatabaseError.
|
|
|
|
# But at the current scale of concurrent requests, we rely on
|
|
|
|
# Postgres's deadlock detection when it occurs.
|
|
|
|
potential_supergroup = access_user_group_by_id(
|
|
|
|
potential_supergroup_id, acting_user, for_read=False
|
|
|
|
)
|
|
|
|
# We avoid making a separate query for user_group_ids because the
|
|
|
|
# recursive query already returns those user groups.
|
|
|
|
potential_subgroups = [
|
|
|
|
user_group
|
|
|
|
for user_group in recursive_subgroups
|
|
|
|
if user_group.id in potential_subgroup_ids
|
|
|
|
]
|
|
|
|
|
|
|
|
# We expect that the passed user_group_ids each corresponds to an
|
|
|
|
# existing user group.
|
|
|
|
group_ids_found = [group.id for group in potential_subgroups]
|
|
|
|
group_ids_not_found = [
|
|
|
|
group_id for group_id in potential_subgroup_ids if group_id not in group_ids_found
|
|
|
|
]
|
|
|
|
if group_ids_not_found:
|
|
|
|
raise JsonableError(
|
|
|
|
_("Invalid user group ID: {group_id}").format(group_id=group_ids_not_found[0])
|
|
|
|
)
|
2022-03-02 11:58:37 +01:00
|
|
|
|
user_groups: Make locks required for updating user group memberships.
**Background**
User groups are expected to comply with the DAG constraint for the
many-to-many inter-group membership. The check for this constraint has
to be performed recursively so that we can find all direct and indirect
subgroups of the user group to be added.
This kind of check is vulnerable to phantom reads which is possible at
the default read committed isolation level because we cannot guarantee
that the check is still valid when we are adding the subgroups to the
user group.
**Solution**
To avoid having another transaction concurrently update one of the
to-be-subgroup after the recursive check is done, and before the subgroup
is added, we use SELECT FOR UPDATE to lock the user group rows.
The lock needs to be acquired before a group membership change is about
to occur before any check has been conducted.
Suppose that we are adding subgroup B to supergroup A, the locking protocol
is specified as follows:
1. Acquire a lock for B and all its direct and indirect subgroups.
2. Acquire a lock for A.
For the removal of user groups, we acquire a lock for the user group to
be removed with all its direct and indirect subgroups. This is the special
case A=B, which is still complaint with the protocol.
**Error handling**
We currently rely on Postgres' deadlock detection to abort transactions
and show an error for the users. In the future, we might need some
recovery mechanism or at least better error handling.
**Notes**
An important note is that we need to reuse the recursive CTE query that
finds the direct and indirect subgroups when applying the lock on the
rows. And the lock needs to be acquired the same way for the addition and
removal of direct subgroups.
User membership change (as opposed to user group membership) is not
affected. Read-only queries aren't either. The locks only protect
critical regions where the user group dependency graph might violate
the DAG constraint, where users are not participating.
**Testing**
We implement a transaction test case targeting some typical scenarios
when an internal server error is expected to happen (this means that the
user group view makes the correct decision to abort the transaction when
something goes wrong with locks).
To achieve this, we add a development view intended only for unit tests.
It has a global BARRIER that can be shared across threads, so that we
can synchronize them to consistently reproduce certain potential race
conditions prevented by the database locks.
The transaction test case lanuches pairs of threads initiating possibly
conflicting requests at the same time. The tests are set up such that exactly N
of them are expected to succeed with a certain error message (while we don't
know each one).
**Security notes**
get_recursive_subgroups_for_groups will no longer fetch user groups from
other realms. As a result, trying to add/remove a subgroup from another
realm results in a UserGroup not found error response.
We also implement subgroup-specific checks in has_user_group_access to
keep permission managing in a single place. Do note that the API
currently don't have a way to violate that check because we are only
checking the realm ID now.
2023-06-17 04:39:52 +02:00
|
|
|
for subgroup in potential_subgroups:
|
|
|
|
# At this time, we only do a check on the realm ID of the fetched
|
|
|
|
# subgroup. This would be caught by the check earlier, so there is
|
|
|
|
# no coverage here.
|
|
|
|
if not has_user_group_access(subgroup, acting_user, for_read=False, as_subgroup=True):
|
|
|
|
raise JsonableError(_("Insufficient permission")) # nocoverage
|
|
|
|
|
|
|
|
yield LockedUserGroupContext(
|
|
|
|
direct_subgroups=potential_subgroups,
|
|
|
|
recursive_subgroups=recursive_subgroups,
|
|
|
|
supergroup=potential_supergroup,
|
|
|
|
)
|
2022-03-02 11:58:37 +01:00
|
|
|
|
|
|
|
|
2024-04-30 13:15:46 +02:00
|
|
|
def check_setting_configuration_for_system_groups(
|
|
|
|
setting_group: NamedUserGroup,
|
2022-09-16 14:27:38 +02:00
|
|
|
setting_name: str,
|
2023-09-18 16:55:58 +02:00
|
|
|
permission_configuration: GroupPermissionSetting,
|
2024-04-30 13:15:46 +02:00
|
|
|
) -> None:
|
|
|
|
if permission_configuration.require_system_group and not setting_group.is_system_group:
|
2023-07-17 22:40:33 +02:00
|
|
|
raise JsonableError(
|
|
|
|
_("'{setting_name}' must be a system user group.").format(setting_name=setting_name)
|
|
|
|
)
|
2022-09-16 14:27:38 +02:00
|
|
|
|
2023-09-18 16:55:58 +02:00
|
|
|
if (
|
|
|
|
not permission_configuration.allow_internet_group
|
2024-04-30 13:15:46 +02:00
|
|
|
and setting_group.name == SystemGroups.EVERYONE_ON_INTERNET
|
2023-09-18 16:55:58 +02:00
|
|
|
):
|
2022-09-16 14:27:38 +02:00
|
|
|
raise JsonableError(
|
2023-07-17 22:40:33 +02:00
|
|
|
_("'{setting_name}' setting cannot be set to 'role:internet' group.").format(
|
|
|
|
setting_name=setting_name
|
|
|
|
)
|
2022-09-16 14:27:38 +02:00
|
|
|
)
|
|
|
|
|
2024-04-30 13:15:46 +02:00
|
|
|
if (
|
|
|
|
not permission_configuration.allow_owners_group
|
|
|
|
and setting_group.name == SystemGroups.OWNERS
|
|
|
|
):
|
2022-09-16 14:27:38 +02:00
|
|
|
raise JsonableError(
|
2023-07-17 22:40:33 +02:00
|
|
|
_("'{setting_name}' setting cannot be set to 'role:owners' group.").format(
|
|
|
|
setting_name=setting_name
|
|
|
|
)
|
2022-09-16 14:27:38 +02:00
|
|
|
)
|
|
|
|
|
2024-04-30 13:15:46 +02:00
|
|
|
if (
|
|
|
|
not permission_configuration.allow_nobody_group
|
|
|
|
and setting_group.name == SystemGroups.NOBODY
|
|
|
|
):
|
2023-04-06 08:13:16 +02:00
|
|
|
raise JsonableError(
|
2023-07-17 22:40:33 +02:00
|
|
|
_("'{setting_name}' setting cannot be set to 'role:nobody' group.").format(
|
|
|
|
setting_name=setting_name
|
|
|
|
)
|
2023-04-06 08:13:16 +02:00
|
|
|
)
|
|
|
|
|
2023-09-18 16:55:58 +02:00
|
|
|
if (
|
|
|
|
not permission_configuration.allow_everyone_group
|
2024-04-30 13:15:46 +02:00
|
|
|
and setting_group.name == SystemGroups.EVERYONE
|
2023-09-18 16:55:58 +02:00
|
|
|
):
|
2023-09-07 02:06:51 +02:00
|
|
|
raise JsonableError(
|
|
|
|
_("'{setting_name}' setting cannot be set to 'role:everyone' group.").format(
|
|
|
|
setting_name=setting_name
|
|
|
|
)
|
|
|
|
)
|
|
|
|
|
2023-03-23 15:42:00 +01:00
|
|
|
if (
|
|
|
|
permission_configuration.allowed_system_groups
|
2024-04-30 13:15:46 +02:00
|
|
|
and setting_group.name not in permission_configuration.allowed_system_groups
|
2023-03-23 15:42:00 +01:00
|
|
|
):
|
|
|
|
raise JsonableError(
|
|
|
|
_("'{setting_name}' setting cannot be set to '{group_name}' group.").format(
|
2024-04-30 13:15:46 +02:00
|
|
|
setting_name=setting_name, group_name=setting_group.name
|
2023-03-23 15:42:00 +01:00
|
|
|
)
|
|
|
|
)
|
|
|
|
|
2024-04-30 13:15:46 +02:00
|
|
|
|
|
|
|
def update_or_create_user_group_for_setting(
|
|
|
|
realm: Realm,
|
|
|
|
direct_members: List[int],
|
|
|
|
direct_subgroups: List[int],
|
|
|
|
current_setting_value: Optional[UserGroup],
|
2024-04-29 05:51:48 +02:00
|
|
|
) -> UserGroup:
|
2024-04-30 15:16:58 +02:00
|
|
|
if current_setting_value is not None and not hasattr(current_setting_value, "named_user_group"):
|
2024-04-30 13:15:46 +02:00
|
|
|
# We do not create a new group if the setting was already set
|
|
|
|
# to an anonymous group. The memberships of existing group
|
|
|
|
# itself are updated.
|
|
|
|
user_group = current_setting_value
|
|
|
|
else:
|
|
|
|
user_group = UserGroup.objects.create(realm=realm)
|
|
|
|
|
|
|
|
from zerver.lib.users import user_ids_to_users
|
|
|
|
|
|
|
|
member_users = user_ids_to_users(direct_members, realm)
|
|
|
|
user_group.direct_members.set(member_users)
|
|
|
|
|
|
|
|
potential_subgroups = NamedUserGroup.objects.filter(realm=realm, id__in=direct_subgroups)
|
|
|
|
group_ids_found = [group.id for group in potential_subgroups]
|
|
|
|
group_ids_not_found = [
|
|
|
|
group_id for group_id in direct_subgroups if group_id not in group_ids_found
|
|
|
|
]
|
|
|
|
if group_ids_not_found:
|
|
|
|
raise JsonableError(
|
|
|
|
_("Invalid user group ID: {group_id}").format(group_id=group_ids_not_found[0])
|
|
|
|
)
|
|
|
|
|
|
|
|
user_group.direct_subgroups.set(group_ids_found)
|
|
|
|
|
2022-09-16 14:27:38 +02:00
|
|
|
return user_group
|
|
|
|
|
|
|
|
|
2024-04-30 13:15:46 +02:00
|
|
|
def access_user_group_for_setting(
|
|
|
|
setting_user_group: Union[int, AnonymousSettingGroupDict],
|
|
|
|
user_profile: UserProfile,
|
|
|
|
*,
|
|
|
|
setting_name: str,
|
|
|
|
permission_configuration: GroupPermissionSetting,
|
|
|
|
current_setting_value: Optional[UserGroup] = None,
|
|
|
|
) -> UserGroup:
|
|
|
|
if isinstance(setting_user_group, int):
|
|
|
|
named_user_group = access_user_group_by_id(setting_user_group, user_profile, for_read=True)
|
|
|
|
check_setting_configuration_for_system_groups(
|
|
|
|
named_user_group, setting_name, permission_configuration
|
|
|
|
)
|
|
|
|
return named_user_group.usergroup_ptr
|
|
|
|
|
|
|
|
# The API would not allow passing the setting parameter as a Dict
|
|
|
|
# if require_system_group is true for a setting.
|
2024-04-29 05:51:48 +02:00
|
|
|
assert permission_configuration.require_system_group is False
|
2024-04-30 13:15:46 +02:00
|
|
|
|
|
|
|
user_group = update_or_create_user_group_for_setting(
|
|
|
|
user_profile.realm,
|
|
|
|
setting_user_group.direct_members,
|
|
|
|
setting_user_group.direct_subgroups,
|
|
|
|
current_setting_value,
|
2024-04-29 05:51:48 +02:00
|
|
|
)
|
2024-04-30 13:15:46 +02:00
|
|
|
|
2024-04-29 05:51:48 +02:00
|
|
|
return user_group
|
2024-04-30 13:15:46 +02:00
|
|
|
|
|
|
|
|
2023-07-03 08:01:01 +02:00
|
|
|
def check_user_group_name(group_name: str) -> str:
|
2023-09-20 11:46:52 +02:00
|
|
|
if group_name.strip() == "":
|
|
|
|
raise JsonableError(_("User group name can't be empty!"))
|
|
|
|
|
2024-04-18 10:50:51 +02:00
|
|
|
if len(group_name) > NamedUserGroup.MAX_NAME_LENGTH:
|
2023-07-03 08:01:01 +02:00
|
|
|
raise JsonableError(
|
2023-07-17 22:40:33 +02:00
|
|
|
_("User group name cannot exceed {max_length} characters.").format(
|
2024-04-18 10:50:51 +02:00
|
|
|
max_length=NamedUserGroup.MAX_NAME_LENGTH
|
2023-07-17 22:40:33 +02:00
|
|
|
)
|
2023-07-03 08:01:01 +02:00
|
|
|
)
|
|
|
|
|
2024-04-18 10:50:51 +02:00
|
|
|
for invalid_prefix in NamedUserGroup.INVALID_NAME_PREFIXES:
|
2023-07-03 08:20:48 +02:00
|
|
|
if group_name.startswith(invalid_prefix):
|
2023-07-17 22:40:33 +02:00
|
|
|
raise JsonableError(
|
|
|
|
_("User group name cannot start with '{prefix}'.").format(prefix=invalid_prefix)
|
|
|
|
)
|
2023-07-03 08:20:48 +02:00
|
|
|
|
2023-07-03 08:01:01 +02:00
|
|
|
return group_name
|
|
|
|
|
|
|
|
|
2024-05-02 05:52:37 +02:00
|
|
|
def get_group_setting_value_for_api(
|
|
|
|
setting_value_group: UserGroup,
|
|
|
|
) -> Union[int, AnonymousSettingGroupDict]:
|
|
|
|
if hasattr(setting_value_group, "named_user_group"):
|
|
|
|
return setting_value_group.id
|
|
|
|
|
|
|
|
return AnonymousSettingGroupDict(
|
|
|
|
direct_members=[member.id for member in setting_value_group.direct_members.all()],
|
|
|
|
direct_subgroups=[subgroup.id for subgroup in setting_value_group.direct_subgroups.all()],
|
|
|
|
)
|
|
|
|
|
|
|
|
|
2022-06-20 23:18:50 +02:00
|
|
|
def user_groups_in_realm_serialized(realm: Realm) -> List[UserGroupDict]:
|
2017-11-30 01:09:23 +01:00
|
|
|
"""This function is used in do_events_register code path so this code
|
|
|
|
should be performant. We need to do 2 database queries because
|
|
|
|
Django's ORM doesn't properly support the left join between
|
|
|
|
UserGroup and UserGroupMembership that we need.
|
2017-11-07 07:56:26 +01:00
|
|
|
"""
|
2024-05-02 05:52:37 +02:00
|
|
|
realm_groups = (
|
|
|
|
NamedUserGroup.objects.select_related(
|
|
|
|
"can_mention_group", "can_mention_group__named_user_group"
|
|
|
|
)
|
|
|
|
# Using prefetch_related results in one query for each field. This is fine
|
|
|
|
# for now but would be problematic when more settings would be added.
|
|
|
|
#
|
|
|
|
# TODO: We should refactor it such that we only make two queries - one
|
|
|
|
# to fetch all the realm groups and the second to fetch all the groups
|
|
|
|
# that they point to and then set the setting fields for realm groups
|
|
|
|
# accordingly in Python.
|
|
|
|
.prefetch_related(
|
|
|
|
Prefetch("can_mention_group__direct_members", queryset=UserProfile.objects.only("id")),
|
|
|
|
Prefetch(
|
|
|
|
"can_mention_group__direct_subgroups", queryset=NamedUserGroup.objects.only("id")
|
|
|
|
),
|
|
|
|
)
|
|
|
|
.filter(realm=realm)
|
|
|
|
)
|
|
|
|
|
2022-06-20 23:18:50 +02:00
|
|
|
group_dicts: Dict[int, UserGroupDict] = {}
|
2017-11-30 01:09:23 +01:00
|
|
|
for user_group in realm_groups:
|
|
|
|
group_dicts[user_group.id] = dict(
|
|
|
|
id=user_group.id,
|
|
|
|
name=user_group.name,
|
|
|
|
description=user_group.description,
|
|
|
|
members=[],
|
2022-05-16 17:02:44 +02:00
|
|
|
direct_subgroup_ids=[],
|
2021-08-06 19:31:00 +02:00
|
|
|
is_system_group=user_group.is_system_group,
|
2024-05-02 05:52:37 +02:00
|
|
|
can_mention_group=get_group_setting_value_for_api(user_group.can_mention_group),
|
2017-11-30 01:09:23 +01:00
|
|
|
)
|
|
|
|
|
2024-05-02 05:46:09 +02:00
|
|
|
membership = (
|
|
|
|
UserGroupMembership.objects.filter(user_group__realm=realm)
|
|
|
|
.exclude(user_group__named_user_group=None)
|
|
|
|
.values_list("user_group_id", "user_profile_id")
|
2021-02-12 08:19:30 +01:00
|
|
|
)
|
2023-02-02 04:35:24 +01:00
|
|
|
for user_group_id, user_profile_id in membership:
|
2021-02-12 08:20:45 +01:00
|
|
|
group_dicts[user_group_id]["members"].append(user_profile_id)
|
2022-02-28 11:50:33 +01:00
|
|
|
|
2024-05-02 05:46:09 +02:00
|
|
|
group_membership = (
|
|
|
|
GroupGroupMembership.objects.filter(subgroup__realm=realm)
|
|
|
|
.exclude(supergroup__named_user_group=None)
|
|
|
|
.values_list("subgroup_id", "supergroup_id")
|
2022-02-28 11:50:33 +01:00
|
|
|
)
|
2023-02-02 04:35:24 +01:00
|
|
|
for subgroup_id, supergroup_id in group_membership:
|
2022-05-16 17:02:44 +02:00
|
|
|
group_dicts[supergroup_id]["direct_subgroup_ids"].append(subgroup_id)
|
2022-02-28 11:50:33 +01:00
|
|
|
|
2017-11-30 01:09:23 +01:00
|
|
|
for group_dict in group_dicts.values():
|
2021-02-12 08:20:45 +01:00
|
|
|
group_dict["members"] = sorted(group_dict["members"])
|
2022-05-16 17:02:44 +02:00
|
|
|
group_dict["direct_subgroup_ids"] = sorted(group_dict["direct_subgroup_ids"])
|
2017-11-30 01:09:23 +01:00
|
|
|
|
2021-02-12 08:20:45 +01:00
|
|
|
return sorted(group_dicts.values(), key=lambda group_dict: group_dict["id"])
|
2017-11-07 07:56:26 +01:00
|
|
|
|
2021-02-12 08:19:30 +01:00
|
|
|
|
2021-10-09 19:53:03 +02:00
|
|
|
def get_direct_user_groups(user_profile: UserProfile) -> List[UserGroup]:
|
2021-10-11 08:37:15 +02:00
|
|
|
return list(user_profile.direct_groups.all())
|
2017-09-25 09:47:15 +02:00
|
|
|
|
2021-02-12 08:19:30 +01:00
|
|
|
|
2022-06-23 19:53:41 +02:00
|
|
|
def get_user_group_direct_member_ids(
|
|
|
|
user_group: UserGroup,
|
2022-09-19 21:48:53 +02:00
|
|
|
) -> ValuesQuerySet[UserGroupMembership, int]:
|
2021-10-12 11:47:36 +02:00
|
|
|
return UserGroupMembership.objects.filter(user_group=user_group).values_list(
|
|
|
|
"user_profile_id", flat=True
|
|
|
|
)
|
2018-02-19 13:38:18 +01:00
|
|
|
|
2021-02-12 08:19:30 +01:00
|
|
|
|
2022-06-23 19:53:41 +02:00
|
|
|
def get_user_group_direct_members(user_group: UserGroup) -> QuerySet[UserProfile]:
|
2022-04-27 12:38:19 +02:00
|
|
|
return user_group.direct_members.all()
|
|
|
|
|
|
|
|
|
2021-10-09 20:02:39 +02:00
|
|
|
def get_direct_memberships_of_users(user_group: UserGroup, members: List[UserProfile]) -> List[int]:
|
2021-02-12 08:19:30 +01:00
|
|
|
return list(
|
|
|
|
UserGroupMembership.objects.filter(
|
|
|
|
user_group=user_group, user_profile__in=members
|
2021-02-12 08:20:45 +01:00
|
|
|
).values_list("user_profile_id", flat=True)
|
2021-02-12 08:19:30 +01:00
|
|
|
)
|
2021-09-29 02:46:57 +02:00
|
|
|
|
|
|
|
|
|
|
|
# These recursive lookups use standard PostgreSQL common table
|
|
|
|
# expression (CTE) queries. These queries use the django-cte library,
|
|
|
|
# because upstream Django does not yet support CTE.
|
|
|
|
#
|
|
|
|
# https://www.postgresql.org/docs/current/queries-with.html
|
|
|
|
# https://pypi.org/project/django-cte/
|
|
|
|
# https://code.djangoproject.com/ticket/28919
|
|
|
|
|
|
|
|
|
2022-06-26 10:03:34 +02:00
|
|
|
def get_recursive_subgroups(user_group: UserGroup) -> QuerySet[UserGroup]:
|
2021-09-29 02:46:57 +02:00
|
|
|
cte = With.recursive(
|
|
|
|
lambda cte: UserGroup.objects.filter(id=user_group.id)
|
2023-04-07 02:03:03 +02:00
|
|
|
.values(group_id=F("id"))
|
2024-04-20 17:03:33 +02:00
|
|
|
.union(
|
|
|
|
cte.join(NamedUserGroup, direct_supergroups=cte.col.group_id).values(group_id=F("id"))
|
|
|
|
)
|
2021-09-29 02:46:57 +02:00
|
|
|
)
|
2023-04-07 02:03:03 +02:00
|
|
|
return cte.join(UserGroup, id=cte.col.group_id).with_cte(cte)
|
2021-09-29 02:46:57 +02:00
|
|
|
|
|
|
|
|
2024-04-20 17:03:33 +02:00
|
|
|
def get_recursive_strict_subgroups(user_group: UserGroup) -> QuerySet[NamedUserGroup]:
|
2024-04-24 11:26:50 +02:00
|
|
|
# Same as get_recursive_subgroups but does not include the
|
|
|
|
# user_group passed.
|
|
|
|
direct_subgroup_ids = user_group.direct_subgroups.all().values("id")
|
|
|
|
cte = With.recursive(
|
2024-04-20 17:03:33 +02:00
|
|
|
lambda cte: NamedUserGroup.objects.filter(id__in=direct_subgroup_ids)
|
2024-04-24 11:26:50 +02:00
|
|
|
.values(group_id=F("id"))
|
2024-04-20 17:03:33 +02:00
|
|
|
.union(
|
|
|
|
cte.join(NamedUserGroup, direct_supergroups=cte.col.group_id).values(group_id=F("id"))
|
|
|
|
)
|
2024-04-24 11:26:50 +02:00
|
|
|
)
|
2024-04-20 17:03:33 +02:00
|
|
|
return cte.join(NamedUserGroup, id=cte.col.group_id).with_cte(cte)
|
2024-04-24 11:26:50 +02:00
|
|
|
|
|
|
|
|
2022-06-26 10:03:34 +02:00
|
|
|
def get_recursive_group_members(user_group: UserGroup) -> QuerySet[UserProfile]:
|
2021-09-29 02:46:57 +02:00
|
|
|
return UserProfile.objects.filter(direct_groups__in=get_recursive_subgroups(user_group))
|
|
|
|
|
|
|
|
|
2022-06-26 10:03:34 +02:00
|
|
|
def get_recursive_membership_groups(user_profile: UserProfile) -> QuerySet[UserGroup]:
|
2021-09-29 02:46:57 +02:00
|
|
|
cte = With.recursive(
|
2023-04-07 02:03:03 +02:00
|
|
|
lambda cte: user_profile.direct_groups.values(group_id=F("id")).union(
|
|
|
|
cte.join(UserGroup, direct_subgroups=cte.col.group_id).values(group_id=F("id"))
|
2021-09-29 02:46:57 +02:00
|
|
|
)
|
|
|
|
)
|
2023-04-07 02:03:03 +02:00
|
|
|
return cte.join(UserGroup, id=cte.col.group_id).with_cte(cte)
|
2021-08-11 15:10:17 +02:00
|
|
|
|
|
|
|
|
2022-03-28 15:55:51 +02:00
|
|
|
def is_user_in_group(
|
|
|
|
user_group: UserGroup, user: UserProfile, *, direct_member_only: bool = False
|
|
|
|
) -> bool:
|
|
|
|
if direct_member_only:
|
2022-04-27 12:38:19 +02:00
|
|
|
return get_user_group_direct_members(user_group=user_group).filter(id=user.id).exists()
|
2022-03-28 15:55:51 +02:00
|
|
|
|
|
|
|
return get_recursive_group_members(user_group=user_group).filter(id=user.id).exists()
|
|
|
|
|
|
|
|
|
2022-03-24 11:39:57 +01:00
|
|
|
def get_user_group_member_ids(
|
|
|
|
user_group: UserGroup, *, direct_member_only: bool = False
|
|
|
|
) -> List[int]:
|
|
|
|
if direct_member_only:
|
2022-06-23 19:53:41 +02:00
|
|
|
member_ids: Iterable[int] = get_user_group_direct_member_ids(user_group)
|
2022-03-24 11:39:57 +01:00
|
|
|
else:
|
|
|
|
member_ids = get_recursive_group_members(user_group).values_list("id", flat=True)
|
|
|
|
|
|
|
|
return list(member_ids)
|
|
|
|
|
|
|
|
|
2022-04-04 13:59:25 +02:00
|
|
|
def get_subgroup_ids(user_group: UserGroup, *, direct_subgroup_only: bool = False) -> List[int]:
|
|
|
|
if direct_subgroup_only:
|
|
|
|
subgroup_ids = user_group.direct_subgroups.all().values_list("id", flat=True)
|
|
|
|
else:
|
2024-04-24 11:26:50 +02:00
|
|
|
subgroup_ids = get_recursive_strict_subgroups(user_group).values_list("id", flat=True)
|
2022-04-04 13:59:25 +02:00
|
|
|
|
|
|
|
return list(subgroup_ids)
|
|
|
|
|
|
|
|
|
user_groups: Make locks required for updating user group memberships.
**Background**
User groups are expected to comply with the DAG constraint for the
many-to-many inter-group membership. The check for this constraint has
to be performed recursively so that we can find all direct and indirect
subgroups of the user group to be added.
This kind of check is vulnerable to phantom reads which is possible at
the default read committed isolation level because we cannot guarantee
that the check is still valid when we are adding the subgroups to the
user group.
**Solution**
To avoid having another transaction concurrently update one of the
to-be-subgroup after the recursive check is done, and before the subgroup
is added, we use SELECT FOR UPDATE to lock the user group rows.
The lock needs to be acquired before a group membership change is about
to occur before any check has been conducted.
Suppose that we are adding subgroup B to supergroup A, the locking protocol
is specified as follows:
1. Acquire a lock for B and all its direct and indirect subgroups.
2. Acquire a lock for A.
For the removal of user groups, we acquire a lock for the user group to
be removed with all its direct and indirect subgroups. This is the special
case A=B, which is still complaint with the protocol.
**Error handling**
We currently rely on Postgres' deadlock detection to abort transactions
and show an error for the users. In the future, we might need some
recovery mechanism or at least better error handling.
**Notes**
An important note is that we need to reuse the recursive CTE query that
finds the direct and indirect subgroups when applying the lock on the
rows. And the lock needs to be acquired the same way for the addition and
removal of direct subgroups.
User membership change (as opposed to user group membership) is not
affected. Read-only queries aren't either. The locks only protect
critical regions where the user group dependency graph might violate
the DAG constraint, where users are not participating.
**Testing**
We implement a transaction test case targeting some typical scenarios
when an internal server error is expected to happen (this means that the
user group view makes the correct decision to abort the transaction when
something goes wrong with locks).
To achieve this, we add a development view intended only for unit tests.
It has a global BARRIER that can be shared across threads, so that we
can synchronize them to consistently reproduce certain potential race
conditions prevented by the database locks.
The transaction test case lanuches pairs of threads initiating possibly
conflicting requests at the same time. The tests are set up such that exactly N
of them are expected to succeed with a certain error message (while we don't
know each one).
**Security notes**
get_recursive_subgroups_for_groups will no longer fetch user groups from
other realms. As a result, trying to add/remove a subgroup from another
realm results in a UserGroup not found error response.
We also implement subgroup-specific checks in has_user_group_access to
keep permission managing in a single place. Do note that the API
currently don't have a way to violate that check because we are only
checking the realm ID now.
2023-06-17 04:39:52 +02:00
|
|
|
def get_recursive_subgroups_for_groups(
|
|
|
|
user_group_ids: Iterable[int], realm: Realm
|
2024-04-17 05:45:32 +02:00
|
|
|
) -> QuerySet[NamedUserGroup]:
|
2023-06-10 10:00:56 +02:00
|
|
|
cte = With.recursive(
|
2024-04-17 05:45:32 +02:00
|
|
|
lambda cte: NamedUserGroup.objects.filter(id__in=user_group_ids, realm=realm)
|
2023-06-10 10:00:56 +02:00
|
|
|
.values(group_id=F("id"))
|
2024-04-17 05:45:32 +02:00
|
|
|
.union(
|
|
|
|
cte.join(NamedUserGroup, direct_supergroups=cte.col.group_id).values(group_id=F("id"))
|
|
|
|
)
|
2023-06-10 10:00:56 +02:00
|
|
|
)
|
2024-04-17 05:45:32 +02:00
|
|
|
recursive_subgroups = cte.join(NamedUserGroup, id=cte.col.group_id).with_cte(cte)
|
2023-06-17 03:59:22 +02:00
|
|
|
return recursive_subgroups
|
2023-06-10 10:00:56 +02:00
|
|
|
|
|
|
|
|
2024-04-16 16:05:43 +02:00
|
|
|
def get_role_based_system_groups_dict(realm: Realm) -> Dict[str, NamedUserGroup]:
|
2024-04-02 18:39:18 +02:00
|
|
|
system_groups = NamedUserGroup.objects.filter(realm=realm, is_system_group=True).select_related(
|
|
|
|
"usergroup_ptr"
|
|
|
|
)
|
2023-06-12 13:27:47 +02:00
|
|
|
system_groups_name_dict = {}
|
|
|
|
for group in system_groups:
|
|
|
|
system_groups_name_dict[group.name] = group
|
|
|
|
|
|
|
|
return system_groups_name_dict
|
|
|
|
|
|
|
|
|
|
|
|
def set_defaults_for_group_settings(
|
2024-04-16 16:05:43 +02:00
|
|
|
user_group: NamedUserGroup,
|
2023-06-14 16:48:58 +02:00
|
|
|
group_settings_map: Mapping[str, UserGroup],
|
2024-04-16 16:05:43 +02:00
|
|
|
system_groups_name_dict: Dict[str, NamedUserGroup],
|
|
|
|
) -> NamedUserGroup:
|
2024-04-18 10:50:51 +02:00
|
|
|
for setting_name, permission_config in NamedUserGroup.GROUP_PERMISSION_SETTINGS.items():
|
2023-06-14 16:48:58 +02:00
|
|
|
if setting_name in group_settings_map:
|
|
|
|
# We skip the settings for which a value is passed
|
|
|
|
# in user group creation API request.
|
|
|
|
continue
|
|
|
|
|
2023-06-12 13:27:47 +02:00
|
|
|
if user_group.is_system_group and permission_config.default_for_system_groups is not None:
|
|
|
|
default_group_name = permission_config.default_for_system_groups
|
|
|
|
else:
|
|
|
|
default_group_name = permission_config.default_group_name
|
|
|
|
|
2024-04-16 16:05:43 +02:00
|
|
|
default_group = system_groups_name_dict[default_group_name].usergroup_ptr
|
2023-06-12 13:27:47 +02:00
|
|
|
setattr(user_group, setting_name, default_group)
|
|
|
|
|
|
|
|
return user_group
|
|
|
|
|
|
|
|
|
2024-04-16 16:05:43 +02:00
|
|
|
def bulk_create_system_user_groups(groups: List[Dict[str, str]], realm: Realm) -> None:
|
|
|
|
# This value will be used to set the temporary initial value for different
|
|
|
|
# settings since we can only set them to the correct values after the groups
|
|
|
|
# are created.
|
|
|
|
initial_group_setting_value = -1
|
|
|
|
|
2024-04-18 18:59:50 +02:00
|
|
|
rows = [SQL("({})").format(Literal(realm.id))] * len(groups)
|
2024-04-16 16:05:43 +02:00
|
|
|
query = SQL(
|
|
|
|
"""
|
2024-04-18 18:59:50 +02:00
|
|
|
INSERT INTO zerver_usergroup (realm_id)
|
2024-04-16 16:05:43 +02:00
|
|
|
VALUES {rows}
|
|
|
|
RETURNING id
|
|
|
|
"""
|
|
|
|
).format(rows=SQL(", ").join(rows))
|
|
|
|
with connection.cursor() as cursor:
|
|
|
|
cursor.execute(query)
|
|
|
|
user_group_ids = [id for (id,) in cursor.fetchall()]
|
|
|
|
|
|
|
|
rows = [
|
|
|
|
SQL("({},{},{},{},{},{})").format(
|
|
|
|
Literal(user_group_ids[idx]),
|
|
|
|
Literal(realm.id),
|
|
|
|
Literal(group["name"]),
|
|
|
|
Literal(group["description"]),
|
|
|
|
Literal(True),
|
|
|
|
Literal(initial_group_setting_value),
|
|
|
|
)
|
|
|
|
for idx, group in enumerate(groups)
|
|
|
|
]
|
|
|
|
query = SQL(
|
|
|
|
"""
|
|
|
|
INSERT INTO zerver_namedusergroup (usergroup_ptr_id, realm_id, name, description, is_system_group, can_mention_group_id)
|
|
|
|
VALUES {rows}
|
|
|
|
"""
|
|
|
|
).format(rows=SQL(", ").join(rows))
|
|
|
|
with connection.cursor() as cursor:
|
|
|
|
cursor.execute(query)
|
|
|
|
|
|
|
|
|
2023-06-05 23:31:10 +02:00
|
|
|
@transaction.atomic(savepoint=False)
|
2024-04-16 16:05:43 +02:00
|
|
|
def create_system_user_groups_for_realm(realm: Realm) -> Dict[int, NamedUserGroup]:
|
2021-08-12 12:15:06 +02:00
|
|
|
"""Any changes to this function likely require a migration to adjust
|
2022-08-01 18:19:31 +02:00
|
|
|
existing realms. See e.g. migration 0382_create_role_based_system_groups.py,
|
2021-08-12 12:15:06 +02:00
|
|
|
which is a copy of this function from when we introduced system groups.
|
|
|
|
"""
|
2024-04-16 16:05:43 +02:00
|
|
|
role_system_groups_dict: Dict[int, NamedUserGroup] = {}
|
|
|
|
|
|
|
|
system_groups_info_list: List[Dict[str, str]] = []
|
|
|
|
|
|
|
|
nobody_group_info = {
|
|
|
|
"name": SystemGroups.NOBODY,
|
|
|
|
"description": "Nobody",
|
|
|
|
}
|
|
|
|
|
|
|
|
full_members_group_info = {
|
|
|
|
"name": SystemGroups.FULL_MEMBERS,
|
|
|
|
"description": "Members of this organization, not including new accounts and guests",
|
|
|
|
}
|
|
|
|
|
|
|
|
everyone_on_internet_group_info = {
|
|
|
|
"name": SystemGroups.EVERYONE_ON_INTERNET,
|
|
|
|
"description": "Everyone on the Internet",
|
|
|
|
}
|
|
|
|
|
|
|
|
system_groups_info_list = [
|
|
|
|
nobody_group_info,
|
2024-04-18 10:50:51 +02:00
|
|
|
NamedUserGroup.SYSTEM_USER_GROUP_ROLE_MAP[UserProfile.ROLE_REALM_OWNER],
|
|
|
|
NamedUserGroup.SYSTEM_USER_GROUP_ROLE_MAP[UserProfile.ROLE_REALM_ADMINISTRATOR],
|
|
|
|
NamedUserGroup.SYSTEM_USER_GROUP_ROLE_MAP[UserProfile.ROLE_MODERATOR],
|
2024-04-16 16:05:43 +02:00
|
|
|
full_members_group_info,
|
2024-04-18 10:50:51 +02:00
|
|
|
NamedUserGroup.SYSTEM_USER_GROUP_ROLE_MAP[UserProfile.ROLE_MEMBER],
|
|
|
|
NamedUserGroup.SYSTEM_USER_GROUP_ROLE_MAP[UserProfile.ROLE_GUEST],
|
2024-04-16 16:05:43 +02:00
|
|
|
everyone_on_internet_group_info,
|
|
|
|
]
|
2023-06-12 13:27:47 +02:00
|
|
|
|
2024-04-16 16:05:43 +02:00
|
|
|
bulk_create_system_user_groups(system_groups_info_list, realm)
|
2023-06-12 13:27:47 +02:00
|
|
|
|
2024-04-16 16:05:43 +02:00
|
|
|
system_groups_name_dict: Dict[str, NamedUserGroup] = get_role_based_system_groups_dict(realm)
|
2024-04-18 10:50:51 +02:00
|
|
|
for role in NamedUserGroup.SYSTEM_USER_GROUP_ROLE_MAP:
|
|
|
|
group_name = NamedUserGroup.SYSTEM_USER_GROUP_ROLE_MAP[role]["name"]
|
2024-04-16 16:05:43 +02:00
|
|
|
role_system_groups_dict[role] = system_groups_name_dict[group_name]
|
|
|
|
|
2021-08-11 15:10:17 +02:00
|
|
|
# Order of this list here is important to create correct GroupGroupMembership objects
|
2022-11-21 07:26:34 +01:00
|
|
|
# Note that because we do not create user memberships here, no audit log entries for
|
|
|
|
# user memberships are populated either.
|
2021-08-11 15:10:17 +02:00
|
|
|
system_user_groups_list = [
|
2024-04-16 16:05:43 +02:00
|
|
|
system_groups_name_dict[SystemGroups.NOBODY],
|
|
|
|
system_groups_name_dict[SystemGroups.OWNERS],
|
|
|
|
system_groups_name_dict[SystemGroups.ADMINISTRATORS],
|
|
|
|
system_groups_name_dict[SystemGroups.MODERATORS],
|
|
|
|
system_groups_name_dict[SystemGroups.FULL_MEMBERS],
|
|
|
|
system_groups_name_dict[SystemGroups.MEMBERS],
|
|
|
|
system_groups_name_dict[SystemGroups.EVERYONE],
|
|
|
|
system_groups_name_dict[SystemGroups.EVERYONE_ON_INTERNET],
|
2021-08-11 15:10:17 +02:00
|
|
|
]
|
|
|
|
|
2022-11-21 05:43:03 +01:00
|
|
|
creation_time = timezone_now()
|
2022-12-12 03:29:10 +01:00
|
|
|
realmauditlog_objects = [
|
2022-11-21 05:43:03 +01:00
|
|
|
RealmAuditLog(
|
|
|
|
realm=realm,
|
|
|
|
acting_user=None,
|
|
|
|
event_type=RealmAuditLog.USER_GROUP_CREATED,
|
|
|
|
event_time=creation_time,
|
|
|
|
modified_user_group=user_group,
|
|
|
|
)
|
|
|
|
for user_group in system_user_groups_list
|
2022-12-12 03:29:10 +01:00
|
|
|
]
|
2021-08-11 15:10:17 +02:00
|
|
|
|
2023-06-12 13:27:47 +02:00
|
|
|
groups_with_updated_settings = []
|
|
|
|
for group in system_user_groups_list:
|
2023-06-14 16:48:58 +02:00
|
|
|
user_group = set_defaults_for_group_settings(group, {}, system_groups_name_dict)
|
2023-09-03 13:41:31 +02:00
|
|
|
groups_with_updated_settings.append(user_group)
|
2024-04-18 18:59:50 +02:00
|
|
|
NamedUserGroup.objects.bulk_update(groups_with_updated_settings, ["can_mention_group"])
|
2023-06-12 13:27:47 +02:00
|
|
|
|
2022-12-12 03:29:10 +01:00
|
|
|
subgroup_objects: List[GroupGroupMembership] = []
|
2023-03-27 05:28:12 +02:00
|
|
|
# "Nobody" system group is not a subgroup of any user group, since it is already empty.
|
|
|
|
subgroup, remaining_groups = system_user_groups_list[1], system_user_groups_list[2:]
|
2021-08-11 15:10:17 +02:00
|
|
|
for supergroup in remaining_groups:
|
|
|
|
subgroup_objects.append(GroupGroupMembership(subgroup=subgroup, supergroup=supergroup))
|
2022-12-12 03:29:10 +01:00
|
|
|
now = timezone_now()
|
|
|
|
realmauditlog_objects.extend(
|
|
|
|
[
|
|
|
|
RealmAuditLog(
|
|
|
|
realm=realm,
|
|
|
|
modified_user_group=supergroup,
|
|
|
|
event_type=RealmAuditLog.USER_GROUP_DIRECT_SUBGROUP_MEMBERSHIP_ADDED,
|
|
|
|
event_time=now,
|
|
|
|
acting_user=None,
|
2023-07-13 19:46:06 +02:00
|
|
|
extra_data={"subgroup_ids": [subgroup.id]},
|
2022-12-12 03:29:10 +01:00
|
|
|
),
|
|
|
|
RealmAuditLog(
|
|
|
|
realm=realm,
|
|
|
|
modified_user_group=subgroup,
|
|
|
|
event_type=RealmAuditLog.USER_GROUP_DIRECT_SUPERGROUP_MEMBERSHIP_ADDED,
|
|
|
|
event_time=now,
|
|
|
|
acting_user=None,
|
2023-07-13 19:46:06 +02:00
|
|
|
extra_data={"supergroup_ids": [supergroup.id]},
|
2022-12-12 03:29:10 +01:00
|
|
|
),
|
|
|
|
]
|
2022-12-12 03:29:10 +01:00
|
|
|
)
|
2021-08-11 15:10:17 +02:00
|
|
|
subgroup = supergroup
|
|
|
|
|
|
|
|
GroupGroupMembership.objects.bulk_create(subgroup_objects)
|
2022-12-12 03:29:10 +01:00
|
|
|
RealmAuditLog.objects.bulk_create(realmauditlog_objects)
|
2021-08-11 15:10:17 +02:00
|
|
|
|
|
|
|
return role_system_groups_dict
|
2021-08-12 12:15:06 +02:00
|
|
|
|
|
|
|
|
2024-04-17 05:45:32 +02:00
|
|
|
def get_system_user_group_for_user(user_profile: UserProfile) -> NamedUserGroup:
|
2024-04-18 10:50:51 +02:00
|
|
|
system_user_group_name = NamedUserGroup.SYSTEM_USER_GROUP_ROLE_MAP[user_profile.role]["name"]
|
2021-08-12 12:15:06 +02:00
|
|
|
|
2024-04-17 05:45:32 +02:00
|
|
|
system_user_group = NamedUserGroup.objects.get(
|
2021-08-12 12:15:06 +02:00
|
|
|
name=system_user_group_name, realm=user_profile.realm, is_system_group=True
|
|
|
|
)
|
|
|
|
return system_user_group
|
2023-10-19 16:50:26 +02:00
|
|
|
|
|
|
|
|
|
|
|
def get_server_supported_permission_settings() -> ServerSupportedPermissionSettings:
|
|
|
|
return ServerSupportedPermissionSettings(
|
2024-03-01 03:02:52 +01:00
|
|
|
realm=Realm.REALM_PERMISSION_GROUP_SETTINGS,
|
|
|
|
stream=Stream.stream_permission_group_settings,
|
2024-04-18 10:50:51 +02:00
|
|
|
group=NamedUserGroup.GROUP_PERMISSION_SETTINGS,
|
2023-10-19 16:50:26 +02:00
|
|
|
)
|
2024-05-16 17:58:43 +02:00
|
|
|
|
|
|
|
|
|
|
|
def parse_group_setting_value(
|
|
|
|
setting_value: Union[int, AnonymousSettingGroupDict],
|
2024-05-30 05:45:38 +02:00
|
|
|
setting_name: str,
|
2024-05-16 17:58:43 +02:00
|
|
|
) -> Union[int, AnonymousSettingGroupDict]:
|
|
|
|
if isinstance(setting_value, int):
|
|
|
|
return setting_value
|
|
|
|
|
|
|
|
if len(setting_value.direct_members) == 0 and len(setting_value.direct_subgroups) == 1:
|
|
|
|
return setting_value.direct_subgroups[0]
|
|
|
|
|
2024-05-30 05:45:38 +02:00
|
|
|
if not settings.ALLOW_ANONYMOUS_GROUP_VALUED_SETTINGS:
|
|
|
|
raise JsonableError(
|
|
|
|
_("{setting_name} can only be set to a single named user group.").format(
|
|
|
|
setting_name=setting_name
|
|
|
|
)
|
|
|
|
)
|
|
|
|
|
2024-05-16 17:58:43 +02:00
|
|
|
return setting_value
|
|
|
|
|
|
|
|
|
2024-05-16 18:10:01 +02:00
|
|
|
def are_both_group_setting_values_equal(
|
2024-05-16 17:58:43 +02:00
|
|
|
first_setting_value: Union[int, AnonymousSettingGroupDict],
|
|
|
|
second_setting_value: Union[int, AnonymousSettingGroupDict],
|
|
|
|
) -> bool:
|
|
|
|
if isinstance(first_setting_value, int) and isinstance(second_setting_value, int):
|
|
|
|
return first_setting_value == second_setting_value
|
|
|
|
|
|
|
|
if isinstance(first_setting_value, AnonymousSettingGroupDict) and isinstance(
|
|
|
|
second_setting_value, AnonymousSettingGroupDict
|
|
|
|
):
|
|
|
|
return set(first_setting_value.direct_members) == set(
|
|
|
|
second_setting_value.direct_members
|
|
|
|
) and set(first_setting_value.direct_subgroups) == set(
|
|
|
|
second_setting_value.direct_subgroups
|
|
|
|
)
|
|
|
|
|
|
|
|
return False
|
|
|
|
|
|
|
|
|
|
|
|
def validate_group_setting_value_change(
|
2024-05-28 11:22:42 +02:00
|
|
|
current_setting_api_value: Union[int, AnonymousSettingGroupDict],
|
2024-05-16 17:58:43 +02:00
|
|
|
new_setting_value: Union[int, AnonymousSettingGroupDict],
|
|
|
|
expected_current_setting_value: Optional[Union[int, AnonymousSettingGroupDict]],
|
|
|
|
) -> bool:
|
2024-05-16 18:10:01 +02:00
|
|
|
if expected_current_setting_value is not None and not are_both_group_setting_values_equal(
|
2024-05-16 17:58:43 +02:00
|
|
|
expected_current_setting_value,
|
|
|
|
current_setting_api_value,
|
|
|
|
):
|
|
|
|
# This check is here to help prevent races, by refusing to
|
|
|
|
# change a setting where the client (and thus the UI presented
|
|
|
|
# to user) showed a different existing state.
|
|
|
|
raise PreviousSettingValueMismatchedError
|
|
|
|
|
2024-05-16 18:10:01 +02:00
|
|
|
return not are_both_group_setting_values_equal(current_setting_api_value, new_setting_value)
|
2024-05-24 11:24:18 +02:00
|
|
|
|
|
|
|
|
|
|
|
def get_group_setting_value_for_audit_log_data(
|
|
|
|
setting_value: Union[int, AnonymousSettingGroupDict],
|
|
|
|
) -> Union[int, Dict[str, List[int]]]:
|
|
|
|
if isinstance(setting_value, int):
|
|
|
|
return setting_value
|
|
|
|
|
|
|
|
return asdict(setting_value)
|