Commit Graph

577 Commits

Author SHA1 Message Date
Anders Kaseorg a2825e5984 python: Use Python 3.8 typing.{Protocol,TypedDict}.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-27 12:57:49 -07:00
Anders Kaseorg 1c228de049 models: Excise NULL from pointlessly nullable fields.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-27 12:40:14 -07:00
Mateusz Mandera ee0021de08 models: Change Stream.history_public_to_subscribers value to True.
The default of a Stream is to be public - having
history_public_to_subscribers default to False is inconsistent with
that. The defaults on the model should generally be consistent.
2022-04-27 12:08:01 -07:00
Mateusz Mandera 04fdf3e4d9 import_utils: Fix history_public_to_subscribers being set incorrectly.
history_public_to_subscribers wasn't explicitly set when creating
streams via build_stream, thus relying on the model's default of False.
This lead to public streams being created with that value set to False,
which doesn't make sense.

We can solve this by inferring the correct value based on invite_only in
the build_stream funtion itself - rather than needing to add a flag
argument to it.

This commit also includes a migration to fix public stream with the
wrong history_public_to_subscribers value.

Fixes #21784.
2022-04-27 12:08:01 -07:00
Raghav Luthra 22a5d008c1 user_settings: Add a setting to display names of users who reacted.
Added a setting to the bottom of Settings > Display settings > Theme section
to display the reacting users on a message when numnber of reactions are
small.

This is a preparatory commit for #20980.
2022-04-19 17:30:16 -07:00
Zixuan James Li 8e98bbae85 invite: Bookkeep UserProfile with PreregistrationUser.
Fixes #21267

Signed-off-by: Zixuan James Li <359101898@qq.com>
2022-04-14 12:47:51 -07:00
Alex Vandiver 35e27aef4a migrations: Remove the possibly-duplicated emoji re-uploading.
In 85e531e377, we duplicated this block
of migration code to fix a bug, but moving it (aka deleting the
original copy) is a cleaner solution.
2022-04-01 17:51:00 -07:00
Tim Abbott 85e531e377 migrations: Repeat part of migration 0376.
The blockquote explains the motivation for this change in detail.

Fixes #21608.
2022-04-01 15:20:43 -07:00
Anders Kaseorg 55882fb343 python: Use modern set comprehension syntax.
Generated by pyupgrade.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-25 10:45:12 -07:00
Tim Abbott e3600900c0 migrations: Adjust related_name settings for ArchivedAttachment.
This is necessary for the migration 0386_fix_attachment_caches to run,
and likely makes more convenient any future parallel code interacting
with both Attachment and ArchivedAttachment.
2022-03-24 19:15:27 -07:00
Tim Abbott abea1f4598 migrations: Add migration to fix Attachment cache.
This migration needs to be run after the previous commit is deployed
to a given Zulip installation, to fix any stale values of
is_realm_public and is_web_public.
2022-03-24 10:50:00 -07:00
Tim Abbott d149af936d models: Rework Attachment.is_*_public to be a cache.
Previously, Attachment.is_realm_public and its cousin,
Attachment.is_web_public, were properties that began as False and
transitioned to True only when a message containing a link to the
attachment was sent to the appropriate class of stream, or such a link
was added as part of editing a message.

This pattern meant that neither field was updated in situations where
the access permissions for a message changed:

* Moving the message to a different stream.
* Changing the permissions for a stream containing links to the message.

This correctness issue has limited security impact, because uploaded
files are secured both by a random URL and by these access checks.

To fix this, we reformulate these fields as a cache, with code paths
that change the permissions affecting an attachment responsible for
setting these values to the `None` (uncached) state. We prefer setting
this `None` state over computing the correct permissions, because the
correct post-edit permissions are a function of all messages
containing the attachment, and we don't want to be responsible for
fetching all of those messages in the edit code paths.
2022-03-24 10:49:59 -07:00
Tim Abbott ceb9dd5854 migrations: Fix confusing output for migration 0383.
* Don't print the empty list for the vast majority of realms where
  this is a noop.

* Make output a little more clear that this isn't revoking all
  Confirmations, just those associated with deactivated users.
2022-03-21 23:25:06 -07:00
byshen-dev 26d97ce7e3 migrations: Change realm field to be not null in Attachment.
he possibility for it being null was likely an oversight -- it should
have been removed after the early migrations to backfill the field
when it was added.

We've confirmed there are no existing violations of this invariant in
Zulip Cloud.
2022-03-18 12:01:15 -07:00
Mateusz Mandera 995cbc69b4 migrations: Add migration to revoke invites from old deactivated users.
This is a natural follow-up to
93e8740218 - invitations sent by users
deactivated before the commit still need to be revoked, via a
migration.

The logic for finding the Confirmations to deactivated is based on
get_valid_invite_confirmations_generated_by_user in actions.py.
2022-03-17 15:16:05 -07:00
Sahil Batra 9345c344d7 user_groups: Add members to the System groups created.
This commit adds users to the appropriate system user group
based on their role. We also change the user groups when
changing role of the user.

We also add migration to add existing users to the appropriate
user groups.

This commit adds update_users_in_full_members_system_group which
is currently used to update the full members group on changing
role of a user. This function will be modified in next commit such
that it can be used to update full members group on changing
waiting_period_threshold setting of realm.
2022-03-14 18:53:46 -07:00
Mateusz Mandera c3209d379c models: Make UserProfile.uuid not nullable.
This is in a separate commit to make deployment easier. It ensures that
this is only marked non-null after the backfill migration (backfilling
.uuid for all old UserProfiles) runs - which was added in the previous
commit.
2022-03-14 17:47:30 -07:00
Mateusz Mandera 3e2ad84bbe models: Add UserProfile.uuid field and backfill migrations. 2022-03-14 17:47:30 -07:00
byshen-dev 73bc5480f3 models: Add unique constraint on RealmUserDefault.realm.
This model is by designed intended to exist on a 1:1 relationship with
Realms, and we attempt to ensure that with application code, but we
should have a unique constraint too, since a database with duplicate
such entries would be corrupted.

We do this via the standard Django OneToOneField.
2022-03-07 21:43:07 -08:00
Tim Abbott 7e298365dc edit_history: Migrate database to modernized edit history format.
This migration can only be safely run after the previous commit has
been deployed.
2022-03-04 10:25:48 -08:00
Anders Kaseorg 1682a6a50d migrations: Lazily import zerver.lib.streams from 0206.
A migration should not import zerver.lib.streams at all, but this
solves the immediate problem with check-database-compatibility.py.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-24 14:31:24 -08:00
Anders Kaseorg 80def8d2c2 models: Manage indexes from migration 0001 with Django.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-23 11:59:45 -08:00
Anders Kaseorg 5703b581fd models: Manage index from migration 0082 with Django.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-23 11:59:45 -08:00
Anders Kaseorg 0e7ee68bcd models: Manage index from migration 0083 with Django.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-23 11:59:45 -08:00
Anders Kaseorg 001a310d75 models: Manage index from migration 0095 with Django.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-23 11:59:45 -08:00
Anders Kaseorg ddd217efdf models: Manage index from migration 0098 with Django.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-23 11:59:45 -08:00
Anders Kaseorg 09bd08ba0d models: Manage index from migration 0099 with Django.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-23 11:59:45 -08:00
Anders Kaseorg 903e219654 models: Manage index from migration 0112 with Django.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-23 11:59:45 -08:00
Anders Kaseorg 71f2b43336 models: Manage index from migration 0177 with Django.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-23 11:59:45 -08:00
Anders Kaseorg 23c8a98157 models: Manage index from migration 0180 with Django.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-23 11:59:45 -08:00
Anders Kaseorg cf0eb46afc models: Manage index from migration 0244 with Django.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-23 11:59:45 -08:00
Anders Kaseorg 0799acf3b6 models: Manage indexes from migration 0279 with Django.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-23 11:59:45 -08:00
Anders Kaseorg 1fa2761790 upgrade-zulip-stage-2: Remove create_large_indexes optimization.
This was only used for upgrading from Zulip < 1.9.0, which is no
longer possible because Zulip < 2.1.0 had no common supported
platforms with current main.

If we ever want this optimization for a future migration, it would be
better implemented using Django merge migrations.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-23 11:59:45 -08:00
Anders Kaseorg 4d858b874e python: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-18 18:03:13 -08:00
Mateusz Mandera 26f9f3cc4b migrations: Don't push events to deferred_work queue if TEST_SUITE.
The deferred_work events can't be processed in the test suite
environment, since it tries to make requests to the host <realm.uri>
which is "zulip.testserver" and obviously not going to work. Since the
test suite base data set has no animated emoji, there's nothing to do,
and we can skip this step.

This code only runs when applying the migration to an already
provisioned test db - because during an initial db set up there are no
realms yet, so no events get pushed by the migration.
2022-02-14 16:05:25 -08:00
Mateusz Mandera 30ac291eba emoji: Add migration to reupload all RealmEmoji and ensure .author.
Fixes #19732.
2022-02-10 17:45:31 -08:00
Anders Kaseorg b0ce4f1bce docs: Fix many spelling mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-07 18:51:06 -08:00
Anders Kaseorg cd9b9675cd migrations: Fix typo in 0099 reverse_sql.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-07 18:47:33 -08:00
Mateusz Mandera 8affb8e4a3 migrations: Remove disallowed characters from stream names.
character_is_printable logic is taken from similar work by @madrix01
2022-02-01 15:14:57 -08:00
Mateusz Mandera 40aa156b28 migrations: Fix RealmAuditLog creation in migration 0374. 2022-01-25 17:20:13 -08:00
Anders Kaseorg 3437e38612 string_validation: Use set comprehension syntax.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:17:02 -08:00
Alex Vandiver 3e90a5cdc0 migrations: Remove an unnecessary list. 2022-01-11 15:59:50 -08:00
Alex Vandiver ce09c8b65f string_validation: Make `unicode_non_chars` a set, for efficiency. 2022-01-11 15:59:38 -08:00
Alex Vandiver a22a6e941f migrations: Fix inversion of character_is_printable. 2022-01-11 15:42:25 -08:00
Alex Vandiver df50280c54 string_validation: Loosen to allow some `Cn` unicode characters.
Under the unicodedata distributed with Python 3.6, some Emoji are
classified as `Cn`, and not `So`:

```
$ unicode 1f929 --long
U+1F929 GRINNING FACE WITH STAR EYES
UTF-8: f0 9f a4 a9 UTF-16BE: d83edd29 Decimal: &#129321; Octal: \0374451
🤩
Category: So (Symbol, Other); East Asian width: W (wide)
Unicode block: 1F900..1F9FF; Supplemental Symbols and Pictographs
Bidi: ON (Other Neutrals)

$ python3.6 -c 'import unicodedata; print(unicodedata.category("\U0001f929"))'
Cn

$ python3.7 -c 'import unicodedata; print(unicodedata.category("\U0001f929"))'
So
```

Drop `Cn` from the list of excluded Unicode character classes, and
replace it with an explicit list of the 66 non-characters, which are
invariant.

Co-authored-by: Shlok Patel <shlokcpatel2001@gmail.com>
2022-01-11 15:17:53 -08:00
Mateusz Mandera 444bb6d0e9 do_delete_user: Create RealmAuditLog entries. 2022-01-04 15:42:03 -08:00
Mateusz Mandera 5939329485 do_delete_user: Add migration to fix bugged UserProfiles.
do_delete_users had two bugs:
1. Creating the replacement dummy users
with active=True
2. Creating the replacement dummy users with email domain set to
realm.uri, which may not be a valid email domain.
Prior commits fixed the bugs, and this migration fixes the pre-existing
objects.
2022-01-04 15:42:03 -08:00
odunybrad 90aa45a316 emoji: Add database-level uniqueness constraint for RealmEmoji.
While races here are unlikely, it is most correct to enforce this
invariant at the database layer, and having a database-level
constraint makes the models file a bit more readable.
2021-12-09 17:48:53 -08:00
Mateusz Mandera 93e18fe289 migrations: Remove disallowed characters from topics.
Following b3c58f454f, we want to clean up
old topics that may contain the disallowed characters. The Message table
is large, so we go in batches, making sure we limit topic fetches and
UPDATE query to no more than BATCH_SIZE Message rows per query.
2021-12-09 09:51:06 -08:00
Aman Agrawal 2a922409aa realm: Add option to enable/disable spectator login.
We restrict access of messages from web public streams if
anonymous login is disabled via `enable_spectator_access`.

Display of `Anonymous login` button is now controlled by
the value of `enable_spectator_access`.

Admins can toggle `enable_spectator_access` via org settings in UI.
2021-11-18 12:55:12 -08:00