Migrate all `ids` of anything which does not have a foreign key from
the Message or UserMessage table (and would thus require walking
those) to be `bigint`. This is done by removing explicit
`BigAutoField`s, trading them for explicit `AutoField`s on the tables
to not be migrated, while updating `DEFAULT_AUTO_FIELD` to the new
default.
In general, the tables adjusted in this commit are small tables -- at
least compared to Messages and UserMessages.
Many-to-many tables without their own model class are adjusted by a
custom Operation, since they do not automatically pick up migrations
when `DEFAULT_AUTO_FIELD` changes[^1].
Note that this does multiple scans over tables to update foreign
keys[^2]. Large installs may wish to hand-optimize this using the
output of `./manage.py sqlmigrate` to join multiple `ALTER TABLE`
statements into one, to speed up the migration. This is unfortunately
not possible to do generically, as constraint names may differ between
installations.
This leaves the following primary keys as non-`bigint`:
- `auth_group.id`
- `auth_group_permissions.id`
- `auth_permission.id`
- `django_content_type.id`
- `django_migrations.id`
- `otp_static_staticdevice.id`
- `otp_static_statictoken.id`
- `otp_totp_totpdevice.id`
- `two_factor_phonedevice.id`
- `zerver_archivedmessage.id`
- `zerver_client.id`
- `zerver_message.id`
- `zerver_realm.id`
- `zerver_recipient.id`
- `zerver_userprofile.id`
[^1]: https://code.djangoproject.com/ticket/32674
[^2]: https://code.djangoproject.com/ticket/24203
This helps prevent wraparound on exceedingly large and old installs,
particularly Zulip Cloud. These are relatively simple migrations
since they are not referenced by any other tables; however, they are
quite large, and are actively used from Django by running servers,
making this not a migration which is possible to run without stopping
the server.
Use the escape hatch in the previous commit to temporarily pause
analytics writes while the migration happens. This should make the
migration transparent to users, at the small cost of an artificial dip
in statistics (specifically, to push notification counts, and unread
message counts) while the migration runs.
This commit updates code, majorly in tests, to use
setting values from enums instead of directly using
the constants defined in Realm.
We still have those constants defined Realm as they
are used in a couple of places where the same code
is used for different settings. These will be
handled later.
This is a follow-up to #29425. It assigns creator to three streams
that are used in tests and another two that aren't. For tests,
we assign a guest, an admin and an owner as creator.
We no longer create the 'core team' private channel when
a realm is created.
Earlier, "New user announcements" channel was set to the
"core team" channel. Now it is disabled by default.
populate_db still creates the 'core team' channel to
represent a private channel.
In #23380, we are changing all occurrences of uri with url in order to
follow the latest URL standard. Previous PRs #25038 and #25045 has
replaced the occurences of uri that has no direct relation with realm.
This commit changes just the model property, which has no API
compatibility concerns.
Since we're just creating an initial database state, there's no reason
we need to submit increment analytics updates to the bouncer service;
such updates will happen during queue worker startup, if needed.
migrated views:
- `zilencer.views.register_remote_server`
- `zilencer.views.register_remote_push_device`
- `zilencer.views.unregister_remote_push_device`
- `zilencer.views.unregister_all_remote_push_devices`
- `zilencer.views.remote_server_notify_push`
to make sure the previous checks for `remote_server_notify_push` matches
to old one, The `RemoteServerNotificationPayload` is defined.
The IntegrityError shows up in the database logs, which looks
unnecessarily concerning. Use `ON CONFLICT IGNORE` to mark this as
expected, especially since the return value is never used.
As explained in the comment, when we're moving the server plan to the
remote realm's Customer object, the realm Customer may not have
stripe_customer_id set and therefore that value needs to get moved from
the server Customer.
Replaced HUDDLE attribute with DIRECT_MESSAGE_GROUP using VS Code search,
part of a general renaming of the object class.
Fixes part of #28640.
Co-authored-by: JohnLu2004 <JohnLu10212004@gmail.com>
When a server doesn't submit a remote realm info which was
previously submitted, we mark it as locally deleted.
If such a realm has paid plan attached to it, we should investigate.
This commit adds logic to send an email to sales@zulip.com for
investigation.
This commit adds a realm-level setting named
'zulip_update_announcements_stream' that configures the
stream to which zulip updates should be posted.
Fixes part of #28604.
When profiling the database queries for the remote support view,
getting the user counts for remote realms was identifed as an
expensive query. Adds an index on RemoteRealmAuditLog to improve
this relatively common query for remote billing information.
We no longer want to migrate Legacy plans from server to realms, since
Legacy plans are not really a thing in the original sense anymore, since
February 15th.
Now they're just a tool to give temporary extensions of access to the
push notification service for users, when needed. And as such, it makes
no sense to migrate like that.
The remaining code in this function is for migrating (any) plan from the
server object to the realm object, if the server has just a single
realm.
This commit renames the realm-level setting 'notifications_stream'
to 'new_stream_announcements_stream'.
The new name reflects better what the setting does.
The logic in the case where there's only one realm and the function
tries to migrate the server's plan to it, had two main unhandled edge
cases that would throw exceptions:
1.
```
remote_realm = RemoteRealm.objects.get(
uuid=realm_uuids[0], plan_type=RemoteRealm.PLAN_TYPE_SELF_MANAGED
)
```
This could throw an exception if the RemoteRealm exists, but has an
active e.g. Legacy plan. Then there'd be no object matching the
plan_type in the query, raising RemoteRealm.DoesNotExist.
2. If the RemoteRealm had e.g. a Legacy plan in the past, that's now
expired, then it'd have a Customer object. Meaning that the attempt
to move the server's customer to the realm:
`server_plan.customer = remote_realm_customer`
would trigger an IntegrityError since a RemoteRealm can't have two
Customer objects.
In simple cases the situation in (2) can still be easily migrated, by
moving the plan from the server's customer to the realm's customer.
Just like deactivated realms should be excluded, so should locally
deleted realms.
In particular, failure to exclude locally deleted realms breaks
handle_customer_migration_from_server_to_realms.
When profiling the database query in `remote_activity.py`,
push_forwarded_count was identified as an expensive part of
the overall work. Adds an index on RemoteInstallationCount
so this is more efficient.
Mateusz Mandera
Alex Vandiver
Kislay Verma
Sahil Batra
Anders Kaseorg
tnmkr
Prakhar Pratyush
Vector73
Kenneth Rodrigues
Tim Abbott
Mahhheshh
Aman Agrawal
John Lu
Lauryn Menard