zulip/zerver/actions
Zixuan Li e39e04c3ce
migration: Add `extra_data_json` for audit log models.
Note that we use the DjangoJSONEncoder so that we have builtin support
for parsing Decimal and datetime.

During this intermediate state, the migration that creates
extra_data_json field has been run. We prepare for running the backfilling
migration that populates extra_data_json from extra_data.

This change implements double-write, which is important to keep the
state of extra data consistent. For most extra_data usage, this is
handled by the overriden `save` method on `AbstractRealmAuditLog`, where
we either generates extra_data_json using orjson.loads or
ast.literal_eval.

While backfilling ensures that old realm audit log entries have
extra_data_json populated, double-write ensures that any new entries
generated will also have extra_data_json set. So that we can then safely
rename extra_data_json to extra_data while ensuring the non-nullable
invariant.

For completeness, we additionally set RealmAuditLog.NEW_VALUE for
the USER_FULL_NAME_CHANGED event. This cannot be handled with the
overridden `save`.

This addresses: https://github.com/zulip/zulip/pull/23116#discussion_r1040277795

Note that extra_data_json at this point is not used yet. So the test
cases do not need to switch to testing extra_data_json. This is later
done after we rename extra_data_json to extra_data.

Double-write for the remote server audit logs is special, because we only
get the dumped bytes from an external source. Luckily, none of the
payload carries extra_data that is not generated using orjson.dumps for
audit logs of event types in SYNC_BILLING_EVENTS. This can be verified
by looking at:

`git grep -A 6 -E "event_type=.*(USER_CREATED|USER_ACTIVATED|USER_DEACTIVATED|USER_REACTIVATED|USER_ROLE_CHANGED|REALM_DEACTIVATED|REALM_REACTIVATED)"`

Therefore, we just need to populate extra_data_json doing an
orjson.loads call after a None-check.

Co-authored-by: Zixuan James Li <p359101898@gmail.com>
2023-06-07 12:14:43 -07:00
..
__init__.py
alert_words.py
bots.py
create_realm.py create_realm: Remove unused argument from do_create_realm. 2023-05-11 12:13:50 -07:00
create_user.py invites: Do not add user to default streams if streams list is empty. 2023-05-09 16:59:06 -07:00
custom_profile_fields.py
default_streams.py
hotspots.py
invites.py emails: Add corporate_enabled to context for emails. 2023-05-01 10:32:43 -07:00
message_delete.py
message_edit.py message_edit: Add period to error message raised due to time limit. 2023-05-29 14:49:32 -07:00
message_flags.py Remove statsd support. 2023-04-25 19:58:16 -07:00
message_send.py message_send: Don't mark scheduled messages to self as read. 2023-05-12 17:55:46 -07:00
muted_users.py
presence.py presence: Support null values in UserPresence. 2023-04-26 14:26:47 -07:00
reactions.py
realm_domains.py
realm_emoji.py
realm_export.py
realm_icon.py
realm_linkifiers.py linkifier: Support URL templates for linkifiers. 2023-04-19 12:20:49 -07:00
realm_logo.py
realm_playgrounds.py
realm_settings.py actions: Handle cross-realm messages in do_scrub_realm. 2023-05-28 15:24:31 -07:00
scheduled_messages.py scheduled-messages: Split out edit scheduled message endpoint. 2023-05-26 18:05:55 -07:00
streams.py streams: Prevent already-deactivated streams from being deactivated. 2023-05-12 13:26:43 -07:00
submessage.py
typing.py
uploads.py scheduled_message: Handle attachments properly. 2023-05-08 09:56:02 -07:00
user_activity.py Remove statsd support. 2023-04-25 19:58:16 -07:00
user_groups.py
user_settings.py migration: Add `extra_data_json` for audit log models. 2023-06-07 12:14:43 -07:00
user_status.py
user_topics.py
users.py
video_calls.py