zulip/zerver/actions
Zixuan Li e39e04c3ce
migration: Add `extra_data_json` for audit log models.
Note that we use the DjangoJSONEncoder so that we have builtin support
for parsing Decimal and datetime.

During this intermediate state, the migration that creates
extra_data_json field has been run. We prepare for running the backfilling
migration that populates extra_data_json from extra_data.

This change implements double-write, which is important to keep the
state of extra data consistent. For most extra_data usage, this is
handled by the overriden `save` method on `AbstractRealmAuditLog`, where
we either generates extra_data_json using orjson.loads or
ast.literal_eval.

While backfilling ensures that old realm audit log entries have
extra_data_json populated, double-write ensures that any new entries
generated will also have extra_data_json set. So that we can then safely
rename extra_data_json to extra_data while ensuring the non-nullable
invariant.

For completeness, we additionally set RealmAuditLog.NEW_VALUE for
the USER_FULL_NAME_CHANGED event. This cannot be handled with the
overridden `save`.

This addresses: https://github.com/zulip/zulip/pull/23116#discussion_r1040277795

Note that extra_data_json at this point is not used yet. So the test
cases do not need to switch to testing extra_data_json. This is later
done after we rename extra_data_json to extra_data.

Double-write for the remote server audit logs is special, because we only
get the dumped bytes from an external source. Luckily, none of the
payload carries extra_data that is not generated using orjson.dumps for
audit logs of event types in SYNC_BILLING_EVENTS. This can be verified
by looking at:

`git grep -A 6 -E "event_type=.*(USER_CREATED|USER_ACTIVATED|USER_DEACTIVATED|USER_REACTIVATED|USER_ROLE_CHANGED|REALM_DEACTIVATED|REALM_REACTIVATED)"`

Therefore, we just need to populate extra_data_json doing an
orjson.loads call after a None-check.

Co-authored-by: Zixuan James Li <p359101898@gmail.com>
2023-06-07 12:14:43 -07:00
..
__init__.py
alert_words.py actions: Split out zerver.actions.alert_words. 2022-04-14 17:14:31 -07:00
bots.py bots: Allow changing bot owner without an acting_user. 2022-08-12 17:08:04 -07:00
create_realm.py create_realm: Remove unused argument from do_create_realm. 2023-05-11 12:13:50 -07:00
create_user.py invites: Do not add user to default streams if streams list is empty. 2023-05-09 16:59:06 -07:00
custom_profile_fields.py markdown: Pass realm down into render_stream_description. 2022-11-01 08:20:47 -07:00
default_streams.py typing: Broaden type annotations for QuerySet compatibility. 2022-07-07 11:27:42 -07:00
hotspots.py actions: Split out zerver.actions.hotspots. 2022-04-14 17:14:31 -07:00
invites.py emails: Add corporate_enabled to context for emails. 2023-05-01 10:32:43 -07:00
message_delete.py ruff: Fix PLC0414 Import alias does not rename original package. 2022-12-04 22:11:24 -08:00
message_edit.py message_edit: Add period to error message raised due to time limit. 2023-05-29 14:49:32 -07:00
message_flags.py Remove statsd support. 2023-04-25 19:58:16 -07:00
message_send.py message_send: Don't mark scheduled messages to self as read. 2023-05-12 17:55:46 -07:00
muted_users.py muted users: Make file naming consistent. 2023-02-10 15:39:57 -08:00
presence.py presence: Support null values in UserPresence. 2023-04-26 14:26:47 -07:00
reactions.py actions: Split out zerver.actions.reactions. 2022-04-14 17:14:35 -07:00
realm_domains.py black: Reformat with Black 23. 2023-02-02 10:40:13 -08:00
realm_emoji.py requirements: Upgrade Django to 4.1. 2022-10-06 15:59:07 -07:00
realm_export.py realm_export: Add transaction.atomic to export_realm. 2023-01-26 10:49:19 -08:00
realm_icon.py realm_audit_log: Explicitly stringify dict before insertion. 2022-07-26 09:48:33 -07:00
realm_linkifiers.py linkifier: Support URL templates for linkifiers. 2023-04-19 12:20:49 -07:00
realm_logo.py actions: Split out zerver.actions.realm_logo. 2022-04-14 17:14:31 -07:00
realm_playgrounds.py actions: Split out zerver.actions.realm_playgrounds. 2022-04-14 17:14:30 -07:00
realm_settings.py actions: Handle cross-realm messages in do_scrub_realm. 2023-05-28 15:24:31 -07:00
scheduled_messages.py scheduled-messages: Split out edit scheduled message endpoint. 2023-05-26 18:05:55 -07:00
streams.py streams: Prevent already-deactivated streams from being deactivated. 2023-05-12 13:26:43 -07:00
submessage.py actions: Split out zerver.actions.submessage. 2022-04-14 17:14:30 -07:00
typing.py black: Reformat with Black 23. 2023-02-02 10:40:13 -08:00
uploads.py scheduled_message: Handle attachments properly. 2023-05-08 09:56:02 -07:00
user_activity.py Remove statsd support. 2023-04-25 19:58:16 -07:00
user_groups.py user_groups: rename create_user_group to create_user_group_in_database. 2023-03-27 09:05:00 -07:00
user_settings.py migration: Add `extra_data_json` for audit log models. 2023-06-07 12:14:43 -07:00
user_status.py user-status: Stop updating the UserStatus model for `away` updates. 2022-09-23 12:27:54 -07:00
user_topics.py user_topics: Update 'set visibility policy' codepath for bulk db operations. 2023-03-29 09:50:06 -07:00
users.py actions: Implement do_delete_user_preserving_messages function. 2023-03-29 12:51:21 -07:00
video_calls.py