In a non interleaved view when composing a message to another
conversation we fade messages which the user is not replying to,
to reduce the chance they send a message to a recipient they didn't
intend to. Also, it reduces the visual/cognitive processing required
to figure out where their message is going to go.
But, it's not necessarily clear to users that what the
fading means, so this commit adds a one-time compose banner
to explain what's going on the first time this comes up.
Fixes part of #29076.
A new table is created to track which path_id attachments are images,
and for those their metadata, and which thumbnails have been created.
Using path_id as the effective primary key lets us ignore if the
attachment is archived or not, saving some foreign key messes.
A new worker is added to observe events when rows are added to this
table, and to generate and store thumbnails for those images in
differing sizes and formats.
Imports (including conversions from Slack, Hipchat, etc) do not write
the `.original` of the file (nor the still versions, for animations),
only the thumbnailed (and possibly animated) version. Looking for
`.original` would thus fail, and overwrite the (working) emoji with
the "?" and disable it.
In the event that we do not find the `.original`, fall back to the
bytes from the non-`.original`, to handle these import cases.
In a2ef1d7e93, we made changes so
that when you send a message, your view jumps to the conversation
where you sent it.
For some users it was an improvement, few reported that it
disrupts their workflows.
This prep commit adds a setting which will be used to allow users
to decide whether to automatically go to conversation where they
sent a message.
This commit renames the "Huddle" Django model class to
"DirectMessageGroup", while maintaining the same table --
"zerver_huddle".
Fixes part of #28640.
Hash the salt, user-id, and now avatar version into the filename.
This allows the URL contents to be immutable, and thus to be marked as
immutable and cacheable. Since avatars are served unauthenticated,
hashing with a server-side salt makes the current and past avatars not
enumerable.
This requires plumbing the current (or future) avatar version through
various parts of the upload process.
Since this already requires a full migration of current avatars, also
take the opportunity to fix the missing `.png` on S3 uploads (#12852).
We switch from SHA-1 to SHA-256, but truncate it such that avatar URL
data does not substantially increase in size.
Fixes: #12852.
Previously, when a referrer's invitation to Zulip was accepted,
they got a notification from notification-bot indicating
their invitation has been accepted.
This commit adds an option for referrer to decide
whether he wants to receive the direct notification
from the notification-bot.
Fixes: #20398
This prep commit adds a new OnboardingUserMessage model
that will be used to mark the new onboarding messages
for new users as unread and the first message of each
onboarding topic as starred.
This table won't include the old onboarding messages.
Because this might impact how past RealmAuditLog-reading migrations
are processed, I think it's better to this one as an actual Django
migration, even though it has no actual SQL to run.
This commit removes create_private_stream_policy setting as
we now use new group based setting.
The "/register" response includes realm_create_private_stream_policy
field to return a value representing superset of users who have the
permission to create private channels, as older clients still expect
this field.
This commit removes create_public_stream_policy setting
since public channel creation permissions are now handled
by group-based setting.
We still pass "realm_create_public_stream_policy" in
"/register" response though for older clients with its
value being set depending on the value of group based
setting. If we cannot set its value to an appropriate
enum corresponding to the group setting, then we set
it to "Members only" considering that server will not
allow the users without permissions to create public
channels but the client can make sure that UI is
available to the users who have permission.
Migrate all `ids` of anything which does not have a foreign key from
the Message or UserMessage table (and would thus require walking
those) to be `bigint`. This is done by removing explicit
`BigAutoField`s, trading them for explicit `AutoField`s on the tables
to not be migrated, while updating `DEFAULT_AUTO_FIELD` to the new
default.
In general, the tables adjusted in this commit are small tables -- at
least compared to Messages and UserMessages.
Many-to-many tables without their own model class are adjusted by a
custom Operation, since they do not automatically pick up migrations
when `DEFAULT_AUTO_FIELD` changes[^1].
Note that this does multiple scans over tables to update foreign
keys[^2]. Large installs may wish to hand-optimize this using the
output of `./manage.py sqlmigrate` to join multiple `ALTER TABLE`
statements into one, to speed up the migration. This is unfortunately
not possible to do generically, as constraint names may differ between
installations.
This leaves the following primary keys as non-`bigint`:
- `auth_group.id`
- `auth_group_permissions.id`
- `auth_permission.id`
- `django_content_type.id`
- `django_migrations.id`
- `otp_static_staticdevice.id`
- `otp_static_statictoken.id`
- `otp_totp_totpdevice.id`
- `two_factor_phonedevice.id`
- `zerver_archivedmessage.id`
- `zerver_client.id`
- `zerver_message.id`
- `zerver_realm.id`
- `zerver_recipient.id`
- `zerver_userprofile.id`
[^1]: https://code.djangoproject.com/ticket/32674
[^2]: https://code.djangoproject.com/ticket/24203
These models have no external references, and are among the larger
tables. Migrate them to bigints, starting with `useractivityinterval`
which is less likely to be being touched.
This table, while it does not contain a large number of rows, consumes
a primary key for every message send and update. Since it is not
referenced by any other table, the migration is simple; and since it
does not contain many rows at any time, it should be fast.
This index is used by `active_users_audit:is_bot:day`, and provides
roughly a 2x speedup. The existing
`zerver_realmauditlog_realm__event_type__event_time` is used if there
is a realm limit, but the standard statistics fill runs for all realms
at once, and thus cannot use it.
Migration plan:
1. Add NULLable .last_update_id column to UserPresence with default 0
for new objects.
2. Backfill the value to 0 for old UserPresences, can be done in the
background while server is running.
3. Make the column non-NULL.
4. Add new model PresenceSequence and create its rows for old realms.
Removes the 'onboarding_steps' field on 'UserProfile'
model which is no longer used.
It was introduced back in 2013 (b5e22bf), and is no
longer used.
Fixes part of #30043.
This commit adds include_realm_default_subscriptions parameter
to the invite endpoints and the corresponding field in
PreregistrationUser and MultiuseInvite objects. This field will
be used to subscribe the new users to the default streams at the
time of account creation and not to the streams that were default
when sending the invite.
We update the existing migration, because any deployment which had
invitations with non-default streams in this state cannot have
proceeded past this migration yet.
In #23380, we are changing all occurrences of uri with url in order to
follow the latest URL standard. Previous PRs #25038 and #25045 has
replaced the occurences of uri that has no direct relation with realm.
This commit changes just the model property, which has no API
compatibility concerns.
Migration 0517 migration was already run as 0497 on `main`, but was
accidentally omitted on 8.x until this point.
Merge the 0517 migration into the migration history. It is included
as a no-op in `main` because it has already run as 0497.
These models should be one-to-one. Various bugs in the past have
leaked violations in both directions; we clean them up such that every
PreregistrationUser which is not a multi-use invite has exactly one
Confirmation object associated with it.
Fixes: #22025
This commit removes name, description, is_system_group and
can_mention_group fields from UserGroup model and rename
them in NamedUserGroup model.
Fixes#29554.
Adds nullable creator field, containing a reference to the user who
created the stream. When creating a stream, acting user is set as
the creator of the stream. Since API calls to create streams always
have an acting user, this field should always be set when streams
are created using the API.
Because streams can be created with no acting user, this field is
nullable. We try to backfill existing streams using RealmAuditLog table,
but not all streams are guaranteed to have a recorded create log. Thus
this new field is left null when it cannot be backfilled. We also set
this field to null when the creator user is deleted.
Adds an index on RealmAuditLog for the realm, event_type, and
event_time in order to improve database queries on these audit logs.
tabbott verified using EXPLAIN ANALYZE that this also considerably
speeds up queries that order by ID rather than event_time, but
event_time is how these should be ordered given the possibility of
backfills.
This commit adds an option to the advanced section of
Preferences settings, that would allow users to choose
whether to receive typing notifications from other
users.
Fixes#29642
Most importantly, fixes a bug where a realm with a custom
.upload_quota_gb value (set by changing it in the database via e.g.
manage.py shell) would end up having it lowered while upgrading their
plan via the do_change_realm_plan_type function, which used to just set
it to the value implied by the new plan without caring about whether
that isn't lower than the original limit.
The new approach is cleaner since we don't do db queries by
upload_quota_gb so it's nicer to just generate these dynamically, making
changes to our limit-per-plan rules much easier - skipping the need for
migrations.
The `zerver/0501_delete_dangling_usermessages` was backported to the
`8.x` branch (and the 8.3 release) in 3db1733310. However, because
`main` contained migrations which `8.x` did not, it was backported
with a different `dependencies`:
```
dependencies = [
("zerver", "0496_alter_scheduledmessage_read_by_sender"),
]
```
...as opposed to in `main`:
```
dependencies = [
("zerver", "0500_realm_zulip_update_announcements_stream"),
]
```
This causes upgrades from 8.3 to `main` to fail:
```
django.db.migrations.exceptions.InconsistentMigrationHistory:
Migration zerver.0501_delete_dangling_usermessages is applied before
its dependency zerver.0500_realm_zulip_update_announcements_stream on
database 'default'.
```
Adjust the dependencies in `main` to match those in `8.x` where many
deploys will first have encountered the migration.
Previously, users were allowed to signup or change their names to
those which already existed in the realm.
This commit adds an Organization Permission, that shall enforce
users to use unique names while signing up or changing their
names. If a same or normalized full name is found in realm,
then a validation error is thrown.
Fixes#7830.
Previously, #26419 addressed the majority of these calls, but did not
prevent more from creeping in. Remove the one remaining
callsite (after the cleanup from the previous commits), and ban any
future use of the pattern.
Replaced HUDDLE attribute with DIRECT_MESSAGE_GROUP using VS Code search,
part of a general renaming of the object class.
Fixes part of #28640.
Co-authored-by: JohnLu2004 <JohnLu10212004@gmail.com>
This commit adds a management command that will run regularly
as a cron job to send zulip updates to realms based on their
current and latest zulip_update_announcements_level.
For realms with:
* level = None: Send a group DM to admins notifying them about
this new feature & suggestion to set the stream accordingly.
* level = 0:
* If stream is still not configured, wait for a week
before setting their level to latest level. They will
miss updates until their configure the stream.
* If stream is configured, send updates.
* level > 0: Send one message/update per level & increase
the level by 1 till the latest level.
Fixes#28604.
We don't want to show one-time modals introducing 'Inbox' and
'Recent conversations' views to existing users.
When a user views a modal, we mark it as read by storing a row
in the 'OnboardingStep' model. Once marked as read, the user will
no longer see the modal.
This commit adds a migration to create
two rows per user (mark them as read) in OnboardingStep model.
This commit updates default for delete_own_message_policy
setting to "Everyone" as it is helpful to allow everyone
to delete their own messages in a new organization where
users might be using Zulip for the first time.
This commit updates default for move_messages_between_streams_policy
setting to "Members and above" as it is helpful to allow members
to move messages between streams in new organizations where users
might be using Zulip for first time.
This commit adds a realm-level setting named
'zulip_update_announcements_stream' that configures the
stream to which zulip updates should be posted.
Fixes part of #28604.
This commit renames the realm-level setting
'signup_notifications_stream' to 'signup_announcements_stream'.
The new name reflects better what the setting does.
This commit renames the realm-level setting 'notifications_stream'
to 'new_stream_announcements_stream'.
The new name reflects better what the setting does.
5c96f94206 mistakenly appended, rather than prepended, the edit to
the history. This caused AssertionErrors when attempting to view the
history of moved messages, which check that the `last_edit_time`
matches the timestamp of the first edit in the list.
Fix the ordering, and update the `edit_history` for messages that were
affected. We limit to only messages edited since the commit was
merged, since that helps bound the affected messages somewhat.
This prep commit moves the 'rename_indexes_constraints'
function to 'lib/migrate' as we're going to re-use it for
the 'UserHotspot' to 'OnboardingStep' table rename operation.
In general, this function would be helpful in migrations
involving table rename operations, subject to the caution
mentioned in the function via comments.
Alex Vandiver
Prakhar Pratyush
Anders Kaseorg
Vector73
Aman Agrawal
Prakhar Pratyush
roanster007
Tim Abbott
Shashank Singh
Sahil Batra
Mateusz Mandera
tnmkr
Lauryn Menard
Karl Stolley
John Lu
ColeBurch