This raises an error when we forget to make migration non-atomic when
doing it in batches. BATCH_SIZE seems to be a good indicator for when
we do batching with few early exceptions.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
If the `postgresql.version` in `/etc/zulip/zulip.conf` is out of date
or wrong, upgrading to the actual current version would drop your
production database without prompting. While we do document taking a
Zulip backup (which includes a database backup) before running
`upgrade-postgresql`[^1], not everyone does so, with possibly
catastrophic consequences.
Do a true end-to-end check of the version in `/etc/zulip/zulip.conf`
by asking Django to query the database for its version, checking that
against the configured value, and aborting if there is any
disagreement.
[^1]: https://zulip.readthedocs.io/en/latest/production/upgrade.html#upgrading-postgresql
Note that we use the DjangoJSONEncoder so that we have builtin support
for parsing Decimal and datetime.
During this intermediate state, the migration that creates
extra_data_json field has been run. We prepare for running the backfilling
migration that populates extra_data_json from extra_data.
This change implements double-write, which is important to keep the
state of extra data consistent. For most extra_data usage, this is
handled by the overriden `save` method on `AbstractRealmAuditLog`, where
we either generates extra_data_json using orjson.loads or
ast.literal_eval.
While backfilling ensures that old realm audit log entries have
extra_data_json populated, double-write ensures that any new entries
generated will also have extra_data_json set. So that we can then safely
rename extra_data_json to extra_data while ensuring the non-nullable
invariant.
For completeness, we additionally set RealmAuditLog.NEW_VALUE for
the USER_FULL_NAME_CHANGED event. This cannot be handled with the
overridden `save`.
This addresses: https://github.com/zulip/zulip/pull/23116#discussion_r1040277795
Note that extra_data_json at this point is not used yet. So the test
cases do not need to switch to testing extra_data_json. This is later
done after we rename extra_data_json to extra_data.
Double-write for the remote server audit logs is special, because we only
get the dumped bytes from an external source. Luckily, none of the
payload carries extra_data that is not generated using orjson.dumps for
audit logs of event types in SYNC_BILLING_EVENTS. This can be verified
by looking at:
`git grep -A 6 -E "event_type=.*(USER_CREATED|USER_ACTIVATED|USER_DEACTIVATED|USER_REACTIVATED|USER_ROLE_CHANGED|REALM_DEACTIVATED|REALM_REACTIVATED)"`
Therefore, we just need to populate extra_data_json doing an
orjson.loads call after a None-check.
Co-authored-by: Zixuan James Li <p359101898@gmail.com>
This in-progress feature was started in 2018 and hasn't
been worked on much since. It's already in a broken state,
which makes it hard to iterate on the existing search bar
since it's hard to know how those changes will affect search
pills.
We do still want to add search pills eventually, and when
we work on that, we can refer to this diff to readd the
changes back.
This commit revamps the invite type section of the "Invite user" modal
to provide a more intuitive user experience. The section now includes a
radio button option with two choices: "Send an email" and "Generate an
invite link." The email input box is hidden when the "Generate an invite
link" option is selected.
Fixes#24692.
An implicit coercion from an untyped dict to the TypedDict was hiding
a type error: CapturedQuery.sql was really str, not bytes. We should
always prefer dataclass over TypedDict to prevent such errors.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
The theory here is that we should prefer to give component callbacks a
simple interface relative to what happened in the component, rather
than a simple implementation relative to what the parent context will
do with that information. Components should be designed to support
being embedded in many possible contexts, including unit tests and
Storybook pages. We’re nowhere near that future but we might as well
take steps in that direction.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Undo a debugging commit
db5f39c506 (#21476) that was made to
help diagnose the issue eventually fixed in commit
6242602276 (#22728). Also remove an
unnecessary mutable variable.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
This adds support to accepting extra_data being dict from remote
servers' RealmAuditLog entries. So that it is forward-compatible with
servers that have migrated to use JSONField for RealmAuditLog just in
case. This prepares us for migrating zilencer's audit log models to use
JSONField for extra_data.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This prepares for the audit log migration which requires us to populate
a JSONField from the extra_data field. "data" is not representative of
the actual extra_data field for RealmAuditLog entries of event types
in SYNC_BILLING_EVENTS.
We intentionally leave the test cases unchanged without bothering to
verify if the extra_data arrives as-is to keep this change minimal.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
It turns out that for some some deployments, there exists a second,
duplicate, foreign key constraint for user_profile_id. The logic below
would try to rename both to the same name, which would fail on the
second:
```
psycopg2.errors.DuplicateObject: constraint "zerver_userpresenceo_user_profile_id_d75366d6_fk_zerver_us" for relation "zerver_userpresence" already exists
```
Eliminate the duplicate constraint, rather than attempting to rename
it. Also add a block, in case of future reuse of this pattern, which
caveats that this approach will not work in the presence of
explicitly-named indexes. UserPresence happens to not have any, so
this technique is safe in this instance.
Co-authored-by: Alex Vandiver <alexmv@zulip.com>
Before this commit our docs mentioned `string[]` data type for
`submessages` field on the `message` object. This commit changes the
type to `object[]` and correctly mentions all fields of the `submessage`
object.
Previously, if the drafts modal was open in two different tabs and
if the user deletes one or more drafts in one tab, then the count of
drafts wouldn't get updated on the other tab.
The right long-term fix likely involves moving drafts from local storage to
the cloud. But we can greatly reduce the impact of the bug by updating the count
whenever the overlay is opened or closed.
It was throwing error while schudiling a message having wildcard mention,
because the function `open_send_later_menu` was using param instance to track down
interval, but the parametere instance was not passed from when it was
called from warning banner action. This commit removes the instance
param as it is of no use, and uses a variable to track interval.
This code clearly meant to return host and returning realm.host is a
mistake. realm.host is not accessible in a migration due to being a
@property-decorated method. The code constructs the host var value just
above this line.
Prakhar Pratyush
evykassirer
Zixuan James Li
Alex Vandiver
Tim Abbott
Anders Kaseorg
Zixuan Li
Alexander Colloredo-Mansfeld
acolloredo
Akarsh Jain
Mateusz Mandera
Lalit
Satyam Bansal
dhruv302003
Brijmohan Siyag