When importing a Stream, UserProfiles don't yet exist. so trying to
import stream.creator fails with something like:
```
psycopg2.errors.ForeignKeyViolation: insert or update on table
"zerver_stream" violates foreign key constraint
"zerver_stream_creator_id_65aeba7e_fk_zerver_userprofile_id"
DETAIL: Key (creator_id)=(5) is not present in table "zerver_userprofile".
```
When the export is NOT generated by another zulip server,
while importing:
* Set the 'zulip_update_announcements_level' to the latest level
as we don't want to send all the older update messages to them.
* Send a group DM to admins, suggesting them to configure the
stream in order to avoid missing future update messages.
Fixes#29041.
Gitter broke their older API as part of being integrated
into Matrix.
Their announcement blog says:
"Anything left using the Gitter APIs will need to be
updated to use the Matrix API"
This commit drops the legacy Gitter import tool and
we plan to build a new one for Matrix in future.
This commit removes name, description, is_system_group and
can_mention_group fields from UserGroup model and rename
them in NamedUserGroup model.
Fixes#29554.
Adds nullable creator field, containing a reference to the user who
created the stream. When creating a stream, acting user is set as
the creator of the stream. Since API calls to create streams always
have an acting user, this field should always be set when streams
are created using the API.
Because streams can be created with no acting user, this field is
nullable. We try to backfill existing streams using RealmAuditLog table,
but not all streams are guaranteed to have a recorded create log. Thus
this new field is left null when it cannot be backfilled. We also set
this field to null when the creator user is deleted.
The bots do not exist in the user table to look up their active
status, and attempting to import them into the analytics table will
result in duplicate rows.
Replaced HUDDLE attribute with DIRECT_MESSAGE_GROUP using VS Code search,
part of a general renaming of the object class.
Fixes part of #28640.
Co-authored-by: JohnLu2004 <JohnLu10212004@gmail.com>
Only affects zulipchat, by being based on the BILLING_ENABLED setting.
The restricted backends in this commit are
- AzureAD - restricted to Standard plan
- SAML - restricted to Plus plan, although it was already practically
restricted due to requiring server-side configuration to be done by us
This restriction is placed upon **enabling** a backend - so
organizations that already have a backend enabled, will continue to be
able to use it. This allows us to make exceptions and enable a backend
for an org manually via the shell, and to grandfather organizations into
keeping the backend they have been relying on.
This commit adds a realm-level setting named
'zulip_update_announcements_stream' that configures the
stream to which zulip updates should be posted.
Fixes part of #28604.
This commit renames the realm-level setting
'signup_notifications_stream' to 'signup_announcements_stream'.
The new name reflects better what the setting does.
This commit renames the realm-level setting 'notifications_stream'
to 'new_stream_announcements_stream'.
The new name reflects better what the setting does.
This commit creates a RealmAuditlog entry with a new event_type
'RealmAuditLog.REALM_IMPORTED' after the realm is reactivated.
It contains user count data (using realm_user_count_by_role)
stored in extra_data.
This helps to have an accurate user count data for the billing
system if someone tries to signup just after doing an import.
This is a rename of the previous
enqueue_register_realm_with_push_bouncer_if_needed but is clearer
about the fact that this will also upload audit logs if available.
This commit moves constants for system group names to a new
"SystemGroups" class so that we can use these group names
in multiple classes in models.py without worrying about the
order of defining them.
This commit renames permissions_configuration variable to
permission_configuration since the object contains config for
a single permission setting and thus permission_configuration
seems like a better name.
This commit does the backend changes required for adding a realm
setting based on groups permission model and does the API changes
required for the new setting `Who can create multiuse invite link`.
`remove_denormalized_recipient_column_from_data` removes the
`recipient` data from `zerver_userprofile`, but did not remove it from
`zerver_userprofile_mirrordummy`, which was later appended to the list
of `zerver_userprofile` objects. This led to failure when inserting,
as the mirrordummy objects still tried to reference their previous
`recipient_id`s.
Move the merging of the two sets earlier, before we call
`remove_denormalized_recipient_column_from_data`.
This tracks user group membership changes when the realm is first set
up, either through an import or not. This happens when we add users to
the system user groups by their roles.
For an imported realm, we do extra handling when the data doesn't include
user groups. This gets audited as well.
Django seems to have an aggressive check on the type of a field when
setting it through an relation, requiring the argument to be a UserGroup in
our case.
Reference:
02966a30dd/django/db/models/base.py (L537-L546)
Make the import of `Realm`, `Stream` and `UserGroup` objects be
done in single transaction, to make the import process in general
more atomic.
This also removes the need to temporarily unset the Stream references
on the Realm object. Since Django creates foreign key constraints
with `DEFERRABLE INITIALLY DEFERRED`, an insertion of a Realm row can
reference not-yet-existing Stream rows as long as the row is created
before the transaction commits.
Discussion - https://chat.zulip.org/#narrow/stream/101-design/topic/New.20permissions.20model/near/1585274.
We now set tos_version to "-1" for imported users and the ones
created using API or using other methods like LDAP, SCIM and
management commands. This value will help us to allow users to
change email address visibility setting during first login.
The comment was outdated, currently we import UserProfiles before
realm_tables - because some models in realm_tables have a dependency on
UserProfile.
Also makes sense to elaborate a bit more in the comment that it's just
an outline of the ordering, not an exhaustive list.
Fixes#25414.
We add Attachment.scheduled_messages relation to track ScheduledMessages
which reference the attachment.
The import bits can be done after merging this, by updating #25345.
This implements the core of the rewrite described in:
For the backend data model for UserPresence to one that supports much
more efficient queries and is more correct around handling of multiple
clients. The main loss of functionality is that we no longer track
which Client sent presence data (so we will no longer be able to say
using UserPresence "the user was last online on their desktop 15
minutes ago, but was online with their phone 3 minutes ago"). If we
consider that information important for the occasional investigation
query, we have can construct that answer data via UserActivity
already. It's not worth making Presence much more expensive/complex
to support it.
For slim_presence clients, this sends the same data format we sent
before, albeit with less complexity involved in constructing it. Note
that we at present will always send both last_active_time and
last_connected_time; we may revisit that in the future.
This commit doesn't include the finalizing migration, which drops the
UserPresenceOld table.
The way to deploy is to start the backfill migration with the server
down and then start the server *without* the user_presence queue worker,
to let the migration finish without having new data interfering with it.
Once the migration is done, the queue worker can be started, leading to
the presence data catching up to the current state as the queue worker
goes over the queued up events and updating the UserPresence table.
Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
Mateusz Mandera
Alex Vandiver
Anders Kaseorg
bedo
Prakhar Pratyush
Prakhar Pratyush
Sahil Batra
tnmkr
Tim Abbott
John Lu
Ujjawal Modi
Zixuan James Li