Commit Graph

6946 Commits

Author SHA1 Message Date
Anders Kaseorg 55882fb343 python: Use modern set comprehension syntax.
Generated by pyupgrade.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-25 10:45:12 -07:00
Alex Vandiver eae4643cb4 message_edit: Ignore duplicates when re-muting new topic name.
This avoids an error when a user has already muted the new topic name.
We do this by ignoring duplicates, rather than catching the
IntegrityError, because this edit happens in a transaction, and that
would abort the transaction.
2022-03-24 21:27:11 -07:00
Alex Vandiver 781107308d muting: Add a flag to allow user duplicate mutes to silently succeed. 2022-03-24 21:27:11 -07:00
Anders Kaseorg 83c90c53df db: Fix types.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-24 19:16:52 -07:00
Tim Abbott f9f111f950 message_edit: Only move muted topic records when moving whole topics.
Our original implementation of moving muted topic records when a topic
is moved took a shortcut of treating all change_later usage as
something with intent to move the whole topic.

This works OK when moving the whole topic via this interface, but not
when moving a last off-topic message in the topic.

Address this by changing the rule to match the existing
moved_all_visible_messages variable.
2022-03-24 17:48:52 -07:00
Lauryn Menard 182c00248d help_docs: Update various docs related to permissions.
Adds tab for web-public streams in documentation for setting
who can create new streams, as well as some text about why
this is limited to certain roles.

Removes list of actions that can be restricted to full members
due to maintainability concerns for that type of list in the
documentation and replaces it with a short descriptive text
explaining that many settings in Zulip support this restriction.
2022-03-24 11:37:35 -07:00
Aman Agrawal b799ec32b0 upload: Allow rate limited access to spectators for uploaded files.
We allow spectators access to uploaded files in web public streams
but rate limit the daily requests to 1000 per file by default.
2022-03-24 10:50:00 -07:00
Tim Abbott d149af936d models: Rework Attachment.is_*_public to be a cache.
Previously, Attachment.is_realm_public and its cousin,
Attachment.is_web_public, were properties that began as False and
transitioned to True only when a message containing a link to the
attachment was sent to the appropriate class of stream, or such a link
was added as part of editing a message.

This pattern meant that neither field was updated in situations where
the access permissions for a message changed:

* Moving the message to a different stream.
* Changing the permissions for a stream containing links to the message.

This correctness issue has limited security impact, because uploaded
files are secured both by a random URL and by these access checks.

To fix this, we reformulate these fields as a cache, with code paths
that change the permissions affecting an attachment responsible for
setting these values to the `None` (uncached) state. We prefer setting
this `None` state over computing the correct permissions, because the
correct post-edit permissions are a function of all messages
containing the attachment, and we don't want to be responsible for
fetching all of those messages in the edit code paths.
2022-03-24 10:49:59 -07:00
Alex Vandiver 4f93b4b6e4 uploads: Skip the outgoing proxy if S3_KEY is unset.
When the credentials are provided by dint of being run on an EC2
instance with an assigned Role, we must be able to fetch the instance
metadata from IMDS -- which is precisely the type of internal-IP
request that Smokescreen denies.

While botocore supports a `proxies` argument to the `Config` object,
this is not actually respected when making the IMDS queries; only the
environment variables are read from.  See
https://github.com/boto/botocore/issues/2644

As such, implement S3_SKIP_PROXY by monkey-patching the
`botocore.utils.should_bypass_proxies` function, to allow requests to
IMDS to be made without Smokescreen impeding them.

Fixes #20715.
2022-03-24 10:21:35 -07:00
Alex Vandiver 1ac0035f8c markdown: Allow whitespace overlaps in topic linkifiers.
`prepare_linkifier_pattern`, as of db934be064, adds a match to the
end of the regex, of either the end of string, or a non-word character
-- this is in place of a negative look-ahead, which is no longer
possible in re2.  This causes the regex to consume trailing
whitespace, and thus not be able to match twice in succession with
`pattern.finditer` -- "#1234 #5678" fails to match because the space
is consumed by the first match of the regex.

Rather than use `pattern.finditer`, write own own version, which
rewinds over the non-word character consumed after the match, if any.
This allows the same "after" non-word character to also satisfy the
"before" of the next match.

Fixes #21502.
2022-03-22 15:40:03 -07:00
Alex Vandiver abed174b12 uploads: Add an endpoint which forces a download.
This is most useful for images hosted in S3, which are otherwise
always displayed in the browser.
2022-03-22 15:05:02 -07:00
Lauryn Menard b67288db67 help_docs: Extend options for getting links to Zulip content.
Extends the linking to Zulip documentation to cover:

- Getting URLs to messages via the message timestamp.
- Getting links to topics via the three-dots menu.
- Getting links to streams via right-click context menu.

Creates a new tabbed section for using the browser
address bar to copy URLs.
2022-03-22 12:38:09 -07:00
Aman Agrawal 7a25a80242 message_edit: Move muted status of the topic when moved between streams.
Add support for moving MutedTopic entries to another stream where
the user has access to shared history in both streams and
`propagate_mode != "change_one"`.

Also, we delete them the current user does not have access to the
target stream.
2022-03-21 12:42:39 -07:00
Tim Abbott aebd81c440 management: Remove unnecessary default=SUPPRESS logic. 2022-03-21 12:05:59 -07:00
Tim Abbott 2be2393d3e create_user: Extract get_create_user_params.
We set nocoverage for the new function. Ideally it'd eventually get an
automated test, but we don't want to block this helpful refactoring on
doing so.
2022-03-21 12:05:59 -07:00
Tim Abbott 57fa62ae4b initial_password: Add explicit development environment assertion.
The construction of INITIAL_PASSWORD_SALT is such that it should only
be set in development environments, but we should enforce this rule.
2022-03-21 12:05:59 -07:00
Tim Abbott 6b00c748fd onboarding: Deduplicate realm creation initial user logic.
We now call this function inside do_create_user(...,
realm_creation=True), which generally improves readability and
robustness of the codebase.

This fixes a bug where this onboarding content was not correctly done
when creating a realm via LDAP, and also will be important as we add
new code paths that might let you create a realm.
2022-03-21 12:05:59 -07:00
Tim Abbott a117b224a7 onboarding: Refactor setup_realm_internal_bots call.
This improves robustness of any code paths calling do_create_realm,
which previously needed to call this correctly to achieve the same
results as creating a user via the UI.

This also fixes a bug where this code was not called if a realm were
created using the LDAP code path.
2022-03-21 12:05:59 -07:00
Tim Abbott b4507df8fa create_user: Remove tos parameter from management command.
This parameter was introduced in
ea11ce4ae6, and no longer serves a
purpose. Zulip will already correctly record that the user has not
agreed to ToS, and either prompt them on first login or not depending
whether the server is configured to require ToS.
2022-03-21 12:05:59 -07:00
Tim Abbott 4f3894f9f1 management: Improve help text for create_user. 2022-03-21 12:05:59 -07:00
Tim Abbott 9761711351 management: Extract add_create_user_args.
This will avoid code duplication when adding a create_realm management command.
2022-03-21 12:05:59 -07:00
Mateusz Mandera af5d0d6f5e bots: Don't allow admins to change owner of bot with can_create_users.
Ordinary organization administrators shouldn't be allowed to change
ownership of a bot with the can_create_users permission.

This is a special permission that is granted manually by server
administrators to an organization (to a UserProfile of the org owners'
choice) after approval by a server administator. The code comments
provide more detail about why this is sensitive.
2022-03-20 17:18:21 -07:00
Tim Abbott f8146cfaa1 help: Fix label for Subscribed tab in stream settings. 2022-03-18 14:28:20 -07:00
Tim Abbott e45cebd636 message_edit: Fix unmute of topic when topic name is edited.
Previously, when a topic was edited (including being resolved), it
would become unmuted for any users who had muted it, which was
annoying.

While it's not possible to determine the user's intent completely,
this is clearly incorrect behavior in the `change_all` case, such as
resolving a topic.

The comments discuss some scenarios where we might want to enhance
this further, but this is the best we can do without large increases
in complexity.

Fixes #15210.

Co-authored-by: akshatdalton <akshat.dak@students.iiit.ac.in>
2022-03-18 12:32:46 -07:00
Suyash Vardhan Mathur 20a97bdb05 events: Add functionality to mark messages as unread.
Co-authored-by: Steve Howell <showell@zulip.com>
Co-authored-by: Tim Abbott <tabbott@zulip.com>

This commit adds the backend functionality to
mark messages as unread through update_message_flags
with `unread` flag and `remove` operation.

We also manage incoming events in the webapp.

Tweaked by tabbott to simplify the implementation and add an API
feature level update to the documentation.

This commit was originally drafted by showell, and showell
also finalized the changes.  Many thanks to Suyash here for
the main work here, which was to get all the tests and
documentation work moving forward.
2022-03-15 18:00:35 -07:00
Sahil Batra 6f93f07844 types: Create a new TypedDict RealmPlaygroundDict for playground objects.
This commit creates a new TypedDict RealmPlaygroundDict for realm
playground objects. Now the list of playgrounds in the events sent
to clients and the "added_playground" field of RealmAuditLog entry
use RealmPlaygroundDict instead of Dict.
2022-03-15 15:58:36 -07:00
Sahil Batra 4bbb5f18e4 actions: Use transaction.atomic for do_remove_realm_playground. 2022-03-15 15:58:36 -07:00
Sahil Batra 578b752766 actions: Use transaction.atomic for do_add_realm_playground. 2022-03-15 15:58:36 -07:00
Sahil Batra b86b9bdc02 realm: Create RealmAuditLog entry when removing realm playgrounds.
This commit also adds 'acting_user' parameter to
do_remove_realm_playground function.

Fixes a part of #21268.
2022-03-15 15:58:36 -07:00
Sahil Batra dea3389045 realm: Create RealmAuditLog entry when adding realm playgrounds.
This commit also adds 'acting_user' parameter to
do_add_realm_playground function.

Fixes a part of #21268.
2022-03-15 15:58:36 -07:00
Sahil Batra 9fd9c5f1d2 actions: Pass realm_playgrounds as argument to notify_realm_playgrounds.
This commit modifies the notify_realm_playgrounds function to accept
realm_playgrounds as argument from the caller instead of computing it
in the function to avoid duplicate queries since the realm playgrounds
list will be required in its caller functions as well in further commits.
2022-03-15 15:58:36 -07:00
Alex Vandiver 62ba8e455d CVE-2022-24751: Clear sessions outside of the transaction.
Clearing the sessions inside the transaction makes Zulip vulnerable to
a narrow window where the deleted session has not yet been committed,
but has been removed from the memcached cache.  During this window, a
request with the session-id which has just been deleted can
successfully re-fill the memcached cache, as the in-database delete is
not yet committed, and thus not yet visible.  After the delete
transaction commits, the cache will be left with a cached session,
which allows further site access until it expires (after
SESSION_COOKIE_AGE seconds), is ejected from the cache due to memory
pressure, or the server is upgraded.

Move the session deletion outside of the transaction.

Because the testsuite runs inside of a transaction, it is impossible
to test this is CI; the testsuite uses the non-caching
`django.contrib.sessions.backends.db` backend, regardless.  The test
added in this commit thus does not fail before this commit; it is
merely a base expression that the session should be deleted somehow,
and does not exercise the assert added in the previous commit.
2022-03-15 13:52:15 -07:00
Alex Vandiver 7650b5a972 session: Enforce that changes cannot happen in a transaction. 2022-03-15 13:52:15 -07:00
Anders Kaseorg 29ecf415fc validator: Add WildValue class for enforcing JSON type checking.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-15 13:02:02 -07:00
Anders Kaseorg 04d772b582 request: Support converter or json_validator with argument_type="body".
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-15 13:02:02 -07:00
Anders Kaseorg 970f22380a request: Replace default_assigned flag with continue.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-15 13:02:02 -07:00
Anders Kaseorg 5f92078d07 request: Add a var_name parameter to converter.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-15 13:02:02 -07:00
Alex Vandiver 95892a5ed3 emoji: Support animated PNGs. 2022-03-15 12:47:21 -07:00
Lauryn Menard 6cc818b672 help_docs: Document user management via user profile modal.
Adds documentation for admins to manage users via the user profile
modal for these actions:
- Deactivating a user
- Changing a user's role
- Changing a user's name

Creates two new tab sections because we still want to document
the ability to do these actions through the users section in
the organizational settings modal.

Also cleans up some text in the help center article for changing
a user's role.

Fixes #21318.
Fixes #21415.
2022-03-15 12:35:03 -07:00
Sahil Batra f0606b34ad user_groups: Add cron job for adding users to full members system group.
This commit adds a cron job which runs every hour to add the users to
full members system group if user is promoted to a full member.

This should ensure that full member status is available no more than
an hour after configuration suggests it should be.
2022-03-14 18:53:47 -07:00
Sahil Batra 565d0928dd import: Create system groups and add users to it while importing.
There can be cases when system groups data is not present while
importing, like when importing from other products, so this
commit adds code to create system user groups and add users to
it according to their role.
2022-03-14 18:53:47 -07:00
Sahil Batra bbf4c25553 user_groups: Update user groups on changing waiting_period_threshold.
The members of "Full members" user group are updated with change in
waiting_period_threshold setting.
2022-03-14 18:53:47 -07:00
Sahil Batra 9345c344d7 user_groups: Add members to the System groups created.
This commit adds users to the appropriate system user group
based on their role. We also change the user groups when
changing role of the user.

We also add migration to add existing users to the appropriate
user groups.

This commit adds update_users_in_full_members_system_group which
is currently used to update the full members group on changing
role of a user. This function will be modified in next commit such
that it can be used to update full members group on changing
waiting_period_threshold setting of realm.
2022-03-14 18:53:46 -07:00
Sahil Batra c33ba4ed6e realm: Create role-based system user groups on creating realm.
We create system user groups for following roles - owners,
admins, moderators, members and guests. Full members user
group will be handled separately.
2022-03-14 18:52:47 -07:00
Sahil Batra f31bb6754b user_groups: Pass list of user ids to remove_members_from_user_group.
We pass list of user ids instead of user profile objects to
remove_members_from_user_group. We still need to call user_id_to_users
in the views function instead of directly passing the ids to
remove_members_from_user_group to make sure we check whether all
ids are valid or not.
2022-03-14 18:52:47 -07:00
Sahil Batra 303fca2ce7 user_groups: Pass list of user ids to bulk_add_members_to_user_group.
We pass list of user ids instead of user profile objects to
bulk_add_members_to_user_group. We still need to call user_id_to_users
in the views function instead of directly passing the ids to
bulk_add_members_to_user_group to make sure we check whether all
ids are valid or not.
2022-03-14 18:52:47 -07:00
Sahil Batra b48b98f1d9 user_groups: Use atomic transaction for adding/removing members.
We execute code for bulk adding and removing members from a user
group inside a transaction.
2022-03-14 18:52:47 -07:00
Steve Howell f9ddd80122 subscriptions: Improve types for subscription info.
The newly extracted module here has significant changes
to all the functions, but they achieve the same thing
as before.
2022-03-14 18:01:39 -07:00
Steve Howell 8f99894302 streams: Extract stream_color library.
This is a pure code move.
2022-03-14 18:01:36 -07:00
Steve Howell 200a89cef0 streams: Extract stream_traffic library.
This is a pure code move.
2022-03-14 18:01:36 -07:00