Commit Graph

523 Commits

Author SHA1 Message Date
tnmkr 5128d8f9af streams: Add creator field.
Adds nullable creator field, containing a reference to the user who
created the stream. When creating a stream, acting user is set as
the creator of the stream. Since API calls to create streams always
have an acting user, this field should always be set when streams
are created using the API.

Because streams can be created with no acting user, this field is
nullable. We try to backfill existing streams using RealmAuditLog table,
but not all streams are guaranteed to have a recorded create log. Thus
this new field is left null when it cannot be backfilled. We also set
this field to null when the creator user is deleted.
2024-04-20 19:03:43 -07:00
Ujjawal Modi 706c380971 audit_log: Update audit log entries created while creating a group.
Earlier a extra audit log entry of type
USER_GROUP_GROUP_BASED_SETTING_CHANGED was made when a new user
group is created. This commit updates the code to not create
that audit log entry.

There is no need to create these entry as we would still
have the required data from the "OLD_VALUE" field in the
audit log entry created when changing the setting and this
also makes it consistent with the entries created for
other operations like stream creation.
2024-04-19 10:18:45 -07:00
Anders Kaseorg 95a1481f99 message_flags: Remove inappropriate use of zerver.lib.timeout.
zerver.lib.timeout abuses asynchronous exceptions, so it’s only safe
to use on CPU computations with no side effects.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-18 11:50:38 -07:00
Anders Kaseorg f31579a220 python: Avoid relying on Collection supertype of QuerySet.
QuerySet doesn’t implement __contains__, so it can’t be a subtype of
Container or Collection (https://code.djangoproject.com/ticket/35154).
This incorrect subtyping annotation was removed in
https://github.com/typeddjango/django-stubs/pull/1925, so we need to
stop relying on it before upgrading to django-stubs 5.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-16 23:02:16 -07:00
roanster007 68b4298d8e settings: Add option to disable seeing typing notifications.
This commit adds an option to the advanced section of
Preferences settings, that would allow users to choose
whether to receive typing notifications from other
users.

Fixes #29642
2024-04-16 11:38:12 -07:00
Shubham Padia 1abd356a91 custom_profile_fields: Make required field optional during update. 2024-04-15 16:52:11 -07:00
Shubham Padia 8c30b61201 custom_profile_fields: Display_in_profile_summary optional for update. 2024-04-15 16:35:42 -07:00
Shubham Padia 137776b092 custom_profile_fields: Make field_data optional during update. 2024-04-15 16:35:42 -07:00
Shubham Padia c6c392bcc7 custom_profile_fields: Make name & hint optional during update.
Fixes #23022.
Added a new validation function that validates updates to custom
profile fields.
2024-04-15 16:35:39 -07:00
Mateusz Mandera 8038e2322c realm: Change implementation approach for upload_quota_gb.
Most importantly, fixes a bug where a realm with a custom
.upload_quota_gb value (set by changing it in the database via e.g.
manage.py shell) would end up having it lowered while upgrading their
plan via the do_change_realm_plan_type function, which used to just set
it to the value implied by the new plan without caring about whether
that isn't lower than the original limit.

The new approach is cleaner since we don't do db queries by
upload_quota_gb so it's nicer to just generate these dynamically, making
changes to our limit-per-plan rules much easier - skipping the need for
migrations.
2024-04-15 15:08:56 -07:00
Alex Vandiver a49111c89a invites: Use query.count() not len(query). 2024-04-12 17:37:11 -07:00
Alex Vandiver 436dab0e01 messages: Remove use of @overload in access_message.
f92d43c690 added uses of `@overload` to probide multiple type
signatures for `access_message`, based on the `get_user_message`
parameter.  Unfortunately, mypy does not check the function body
against overload signatures, so it allows type errors to go
undetected.

Replace the overloads with two functions, for one of which also
returns the usermessage.  The third form, of only returning if the
usermessage exists, is not in a high-enough performance endpoint that
a third form is worth maintaining; it uses the usermessage form.
2024-04-12 11:41:27 -07:00
Mateusz Mandera c9ca4e68e5 scim: Add config option to disable initial streams for guests.
When an organization (without open ability for anyone to join) invites a
guest user, the invitation prompts allows them to choose whether the
guest should be added to default streams or not. This is useful, because
since we don't have per-role default streams configs, they may want
default streams to be for full Members.

SCIM provisioning doesn't have this control, since a newly provisioned
user gets created via a direct do_create_user call, thus adding them to
the organization's default streams, with no workaround possible aside of
just getting rid of default streams in the organization.

To make provisioning guests in such an organization usable, we add a
simple config option to create them with no streams. It's configured by
adding
```
"create_guests_without_streams": True
```

to the config dict in settings.SCIM_CONFIG.
2024-04-11 12:28:26 -07:00
Sahil Singh 6c2535fe46 bots: Avoid multiple active bots with the same name.
Creating a bot with a name that is already in use
will raise an error. However, by deactivating
the existing bot, creating a new bot with the
same name, and then reactivating the original bot,
it is possible to have multiple bots with the same name.

To fix this, we check if the bot name is already
in use in the active bots list. If it is,
an error will be raised, prompting either the
name of the existing bot to be changed or
the bot to be deactivated.

Co-authored-by: Sujal Shah <sujalshah28092004@gmail.com>
2024-04-10 13:14:25 -07:00
Prakhar Pratyush 2abc25ac4c create_user: Send a group DM to admins for low licenses warning.
Earlier, low licenses warning message was sent to the
"New user announcements" stream.

We plan to disable the stream by default as a part of improving
onboarding experience.

Now, we send a group DM to admins for low licenses warning
to make it independent of the setting. These warning messages
are important and shouldn't be missed.
2024-04-03 12:28:05 -07:00
Prakhar Pratyush db214c8377 message: Add recipient_users parameter to internal_send_huddle_message.
This is a prep commit to add a 'recipient_users' parameter to
the 'internal_send_huddle_message' function.

'emails' is no longer a required parameter. We can use either
of the 'emails' or 'recipient_users' parameter. 'emails' is
eventually used to fetch 'recipient_users', so if the
'recipient_users' is already available we should use that to
skip database query.
2024-04-03 12:28:05 -07:00
roanster007 c7a08f3b77 settings: Add permission to enforce unique names in realm.
Previously, users were allowed to signup or change their names to
those which already existed in the realm.

This commit adds an Organization Permission, that shall enforce
users to use unique names while signing up or changing their
names. If a same or normalized full name is found in realm,
then a validation error is thrown.

Fixes #7830.
2024-04-02 14:55:59 -07:00
Aman Agrawal d21f5c9b75 registration: Ask user how they found Zulip. 2024-04-01 12:44:12 -07:00
Alex Vandiver 7988aad159 user_message: Use INSERT ... ON CONFLICT for historical UM creation.
Rather than use a bulk insert via Django, use the faster
`bulk_insert_all_ums` that we already have.  This also adds a `ON
CONFLICT` clause, to make the insert resilient to race conditions.

There are currently two callsites, with different desired `ON
CONFLICT` behaviours:
 - For `notify_reaction_update`, if the `UserMessage` had already been
   created, we would have done nothing to change it.
 - For `do_update_message_flags`, we would have ensured a specific bit
   was (un)set.

Extend `create_historical_user_messages` and `bulk_insert_all_ums` to
support `ON CONFLICT (...) UPDATE SET flags = ...`.
2024-03-26 10:52:37 -07:00
Alex Vandiver 52e3c8e1b2 user_message: Move create_historical_user_messages to zerver.lib.user_message.
It is not even used in zerver.actions.create_user.
2024-03-26 10:52:37 -07:00
Alex Vandiver f92d43c690 messages: Use overloads to only return a user_message if needed. 2024-03-22 09:30:17 -07:00
John Lu a5cf0ec526
refactor: Replace HUDDLE with DIRECT_MESSAGE_GROUP.
Replaced HUDDLE attribute with DIRECT_MESSAGE_GROUP using VS Code search,
part of a general renaming of the object class.

Fixes part of #28640.

Co-authored-by: JohnLu2004 <JohnLu10212004@gmail.com>
2024-03-21 16:39:33 -07:00
Vector73 f758ca596b custom_profile_fields: Add "required" parameter to the profile fields.
Fixes #28512.
2024-03-21 10:48:54 -07:00
Anders Kaseorg a1a341f0ae ruff: Fix UP032 Use f-string instead of `format` call.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-21 08:59:17 -07:00
Prakhar Pratyush 118a7e8d9d zulip_updates: Send zulip updates based on zulip_update_*_level.
This commit adds a management command that will run regularly
as a cron job to send zulip updates to realms based on their
current and latest zulip_update_announcements_level.

For realms with:
* level = None: Send a group DM to admins notifying them about
this new feature & suggestion to set the stream accordingly.

* level = 0:
  * If stream is still not configured, wait for a week
    before setting their level to latest level. They will
    miss updates until their configure the stream.
  * If stream is configured, send updates.

* level > 0: Send one message/update per level & increase
  the level by 1 till the latest level.

Fixes #28604.
2024-03-20 11:48:06 -07:00
Prakhar Pratyush 336d46001c message_send: Add 'internal_prep_huddle_message' function.
This is a prep commit to extract out the logic to
create message from 'internal_send_huddle_message'
into a separate function 'internal_prep_huddle_message'.

We will use this new function to get the huddle message
without sending it immediately.
2024-03-20 11:48:06 -07:00
Prakhar Pratyush 23fc04577b message_send: Don't create savepoint in do_send_messages.
In general, we never want to use savepoints.

This prep commit adds savepoint=False in do_send_messages
as we don't want to just rollback to this savepoint and
proceed if we encounter any error while sending zulip updates
via cron.
2024-03-20 11:48:06 -07:00
Alex Vandiver 0f0631813f message_edit: Do set differences in QuerySets. 2024-03-19 18:22:24 -04:00
Alex Vandiver c71b7afe9f message_edit: Handle user, not subscription, collections.
Nothing about the subscription is relevant -- we instead use
collections of User objects for readability.
2024-03-19 18:22:24 -04:00
Alex Vandiver f15e006873 message_edit: Perform cross join in SQL, not Python.
The database can do this faster than we can, and it reduces the amount
of information transmitted over the wire.
2024-03-19 18:22:24 -04:00
Alex Vandiver a8795c5d0b message_edit: Pull user_ids_gaining_usermessages higher, rename. 2024-03-19 18:22:24 -04:00
Alex Vandiver 7ce6095003 message_edit: Handle previous subscribers and private-history streams. 2024-03-19 18:22:24 -04:00
Alex Vandiver 7b1feac06a CVE-2024-27286: Remove UserMessage rows for non-active Subscriptions.
A user who was no longer subscribed to a private stream kept their
UserMessage row for a message sent while they were in it; this is
expected.  However, they _also_ kept that row even if the message was
moved to a different private stream that they were also not subscribed
to.  This violates the invariant that users without subscriptions
never have UserMessage rows.
2024-03-19 18:22:24 -04:00
Alex Vandiver e964536139 CVE-2024-27286: Run usermessage modifications even for change_one.
This `if new_stream is not None` block was improperly indented,
causing it to only run if the propagation mode was not `change_one`.
Since the block controlled creation and deletion of UserMessage rows,
this led to messages being improperly still visible to members of the
old stream if they were being moved from public to private streams.
Clients also failed to receive `delete_message` events, so the
messages remained visible in their feeds until they reloaded the
application.
2024-03-19 18:22:24 -04:00
Mateusz Mandera 1ede8da46c events: Change format of authentication_methods realm_update_dict event.
Makes the event in line with state_data.realm_authentication_methods.
2024-03-05 11:48:58 -08:00
Mateusz Mandera da9e4e6e54 backends: Implementation of restricting certain backends by plan.
Only affects zulipchat, by being based on the BILLING_ENABLED setting.

The restricted backends in this commit are
- AzureAD - restricted to Standard plan
- SAML - restricted to Plus plan, although it was already practically
  restricted due to requiring server-side configuration to be done by us

This restriction is placed upon **enabling** a backend - so
organizations that already have a backend enabled, will continue to be
able to use it. This allows us to make exceptions and enable a backend
for an org manually via the shell, and to grandfather organizations into
keeping the backend they have been relying on.
2024-03-05 11:48:58 -08:00
Alex Vandiver 352dbf9387 caches: Only take the realm_id, not the Realm, as a cache key function.
This saves a hit to the database to fetch the Realm of a UserProfile
that we are trying to flush.
2024-03-04 16:35:57 -08:00
Anders Kaseorg 570f3dd447 python: Reformat with Ruff formatter.
https://docs.astral.sh/ruff/formatter/

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-02-29 17:07:16 -08:00
Prakhar Pratyush fe1a20ebb3 settings: Add realm-level setting 'zulip_update_announcements_stream'.
This commit adds a realm-level setting named
'zulip_update_announcements_stream' that configures the
stream to which zulip updates should be posted.

Fixes part of #28604.
2024-02-25 09:33:00 -08:00
Prakhar Pratyush ee612dafac settings: Rename signup_notifications_stream realm setting.
This commit renames the realm-level setting
'signup_notifications_stream' to 'signup_announcements_stream'.

The new name reflects better what the setting does.
2024-02-21 09:04:23 -08:00
Prakhar Pratyush ab453fbe20 settings: Rename notifications_stream to new_stream_announce..._stream.
This commit renames the realm-level setting 'notifications_stream'
to 'new_stream_announcements_stream'.

The new name reflects better what the setting does.
2024-02-21 09:04:23 -08:00
sayyedarib 5c993f0479 direct_message: Allow DMs to bots and self regardless of policy.
Previously, in DM disabled org messaging to bot was not working when
starting new conversation and adding bot as recipient because of not
updating on recipient change. And secondly, self messaging was not
allowed.
This commit ensures that the DM to bot and self are allowed irrespective
of dm restrictions.

tests: Verify DMs adhere to DM restriction policy.

Fixes #28412

Signed-off-by: sayyedarib <sayyedaribhussain4321@gmail.com>
2024-02-15 16:13:36 -08:00
Alex Vandiver b3e8878252 message_cache: Rename update_to_dict_cache to update_message_cache.
This better describes what it does.
2024-02-14 12:27:03 -08:00
Alex Vandiver 737a751f5c message: Split MessageDict and friends into its own file. 2024-02-14 12:27:03 -08:00
Alex Vandiver 0f9b7f112b message: Move render_markdown into zerver.lib.markdown. 2024-02-14 12:27:03 -08:00
Alex Vandiver 22837fc1b4 message_edit: Carry the QuerySet through as much as possible.
Rather than pass around a list of message objects in-memory, we
instead keep the same constructed QuerySet which includes the later
propagated messages (if any), and use that same query to pick out
affected Attachment objects, rather than limiting to the set of ids.
This is not necessarily a win -- the list of message-ids *may* be very
long, and thus the query may be more concise, easier to send to
PostgreSQL, and faster for PostgreSQL to parse.  However, the list of
ids is almost certainly better-indexed.

After processing the move, the QuerySet must be re-defined as a search
of ids (and possibly a very long list of such), since there is no
other way which is guaranteed to correctly single out the moved
messages.  At this point, it is mostly equivalent to the list of
Message objects, and certainly takes no less memory.
2024-02-14 12:27:03 -08:00
Alex Vandiver 822131fef4 message: Add a bulk_access_stream_messages_query method.
This applies access restrictions in SQL, so that individual messages
do not need to be walked one-by-one.  It only functions for stream
messages.

Use of this method significantly speeds up checks if we moved "all
visible messages" in a topic, since we no longer need to walk every
remaining message in the old topic to determine that at least one was
visible to the user.  Similarly, it significantly speeds up merging
into existing topics, since it no longer must walk every message in
the new topic to determine if the user could see at least one.

Finally, it unlocks the ability to bulk-update only messages the user
has access to, in a single query (see subsequent commit).
2024-02-14 12:27:03 -08:00
Alex Vandiver e04891779f message_edit: len(changed_messages) is always > 0.
It is only ever added to, and starts as a one-element list.
2024-02-14 12:27:03 -08:00
Tim Abbott 79ddfa1e6f message_edit: Ensure gaining_usermessage_user_ids is set.
This makes this code path a bit easier to follow.
2024-02-14 12:27:03 -08:00
Tim Abbott 4a049299ee message_edit: Clarify unmodified_user_messages logic. 2024-02-14 12:27:03 -08:00