mirror of https://github.com/zulip/zulip.git
a081428ad2
**Background** User groups are expected to comply with the DAG constraint for the many-to-many inter-group membership. The check for this constraint has to be performed recursively so that we can find all direct and indirect subgroups of the user group to be added. This kind of check is vulnerable to phantom reads which is possible at the default read committed isolation level because we cannot guarantee that the check is still valid when we are adding the subgroups to the user group. **Solution** To avoid having another transaction concurrently update one of the to-be-subgroup after the recursive check is done, and before the subgroup is added, we use SELECT FOR UPDATE to lock the user group rows. The lock needs to be acquired before a group membership change is about to occur before any check has been conducted. Suppose that we are adding subgroup B to supergroup A, the locking protocol is specified as follows: 1. Acquire a lock for B and all its direct and indirect subgroups. 2. Acquire a lock for A. For the removal of user groups, we acquire a lock for the user group to be removed with all its direct and indirect subgroups. This is the special case A=B, which is still complaint with the protocol. **Error handling** We currently rely on Postgres' deadlock detection to abort transactions and show an error for the users. In the future, we might need some recovery mechanism or at least better error handling. **Notes** An important note is that we need to reuse the recursive CTE query that finds the direct and indirect subgroups when applying the lock on the rows. And the lock needs to be acquired the same way for the addition and removal of direct subgroups. User membership change (as opposed to user group membership) is not affected. Read-only queries aren't either. The locks only protect critical regions where the user group dependency graph might violate the DAG constraint, where users are not participating. **Testing** We implement a transaction test case targeting some typical scenarios when an internal server error is expected to happen (this means that the user group view makes the correct decision to abort the transaction when something goes wrong with locks). To achieve this, we add a development view intended only for unit tests. It has a global BARRIER that can be shared across threads, so that we can synchronize them to consistently reproduce certain potential race conditions prevented by the database locks. The transaction test case lanuches pairs of threads initiating possibly conflicting requests at the same time. The tests are set up such that exactly N of them are expected to succeed with a certain error message (while we don't know each one). **Security notes** get_recursive_subgroups_for_groups will no longer fetch user groups from other realms. As a result, trying to add/remove a subgroup from another realm results in a UserGroup not found error response. We also implement subgroup-specific checks in has_user_group_access to keep permission managing in a single place. Do note that the API currently don't have a way to violate that check because we are only checking the realm ID now. |
||
|---|---|---|
| .. | ||
| ci | ||
| documentation_crawler | ||
| droplets | ||
| i18n | ||
| lib | ||
| linter_lib | ||
| node_lib | ||
| oneclickapps | ||
| setup | ||
| test-install | ||
| tests | ||
| zulip-export | ||
| README.md | ||
| __init__.py | ||
| build-docs | ||
| build-release-tarball | ||
| cache-zulip-git-version | ||
| check-capitalization | ||
| check-frontend-i18n | ||
| check-issue-labels | ||
| check-openapi | ||
| check-provision | ||
| check-schemas | ||
| check-templates | ||
| check-thirdparty | ||
| clean-branches | ||
| commit-message-lint | ||
| commit-msg | ||
| conf.ini-template | ||
| coveragerc | ||
| deploy-branch | ||
| diagnose | ||
| documentation.vnufilter | ||
| duplicate_commits.json | ||
| fetch-contributor-data | ||
| fetch-pull-request | ||
| fetch-rebase-pull-request | ||
| find-unused-css | ||
| generate-integration-docs-screenshot | ||
| lint | ||
| message-screenshot.js | ||
| pre-commit | ||
| provision | ||
| push-to-pull-request | ||
| rebuild-dev-database | ||
| rebuild-test-database | ||
| release | ||
| release-tarball-exclude.txt | ||
| renumber-migrations | ||
| reset-to-pull-request | ||
| review | ||
| run-codespell | ||
| run-dev | ||
| run-mypy | ||
| run-tsc | ||
| semgrep.yml | ||
| setup-git-repo | ||
| show-profile-results | ||
| stop-run-dev | ||
| tail-ses | ||
| test-all | ||
| test-api | ||
| test-backend | ||
| test-documentation | ||
| test-help-documentation | ||
| test-js-with-node | ||
| test-js-with-puppeteer | ||
| test-locked-requirements | ||
| test-migrations | ||
| test-queue-worker-reload | ||
| test-run-dev | ||
| test-tools | ||
| total-contributions | ||
| update-locked-requirements | ||
| update-prod-static | ||
| update-zuliprc-api-field | ||
| upload-release | ||
| webpack | ||
| zanitizer | ||
| zanitizer_config.pm.sample | ||
README.md
This directory contains scripts that are used in building, managing,
testing, and other forms of work in a Zulip development environment.
Note that tools that are also useful in production belong in
scripts/ or should be Django management commands.
For more details, see https://zulip.readthedocs.io/en/latest/overview/directory-structure.html.