This commit updates all third-party importer tools (Slack, Mattermost,
and Rocket Chat) in the `zerver/data_import` directory to also output a
migration_status.json file in their output tarball.
This is required because all importable tarball will be checked for
migration compatibility during import.
Fixes#28443.
When transferring a realm to a server that has a different set of
applied migrations (different Zulip versions), there is a chance that
the imported data formats appear to be compatible but data invariants
could still be violated.
This commit adds an assertion during the import process to verify
that both the exported realm and the importing server have matching
Zulip versions and have a compatible set of migrations.
This commit updates the export process to write the migration status of
the realm as a JSON file to be included in the export tarball.
This is a preparatory step for adding an assertion to ensure that the
importing and exporting realms have a compatible set of applied
migrations.
Newrelic has introduced a new update. This commit
updates the exisiting integration by adding support
for new types of payloads and cleaning up legacy
codes and fixtures.
Fixes#29729.
Co-authored-by: Pieter CK <pieterceka123@gmail.com>
Co-authored-by: Lauryn Menard <lauryn@zulip.com>
For resizing the icon.png files, we use resize_avatar, not resize_logo.
This is pretty confusing - sure, for icons we use the same function as
for avatars, but we should have a proper name for the function called in
the icon context. So this commit also adds resize_realm_icon, and
changes the calls to resize_avatar in icon contexts to
resize_realm_icon.
This commit refactors `render_attachment` and `render_block`
out of slack_incoming.py to promote reusability.
The primary motivation for this refactor is to add support for
converting integration bots messages in Slack exports, which could use
the same functions.
Part of #31311.
Previously, the GoCD integration relied on GoCD's integration with bug
tracking and monitoring tools through the users' Config XML file [1].
However, this feature no longer works as expected, as it cannot send
HTTPS payloads to an external endpoint.
This commit updates our GoCD integration to use Sentry's WebHook
notifier GoCD plugin [2] to send webhook payloads from users GoCD server
to Zulip. We are using an older version of the plugin—v0.0.6—because the
newer version—v0.0.9—doesn't work at this time.
Additionally, this change reformats the notifications to include more
details on the GoCD event, such as lists of passed and failed jobs.
Fixes#21224.
Co-authored-by: Pieter CK <pieterceka123@gmail.com>
[1]:
https://docs.gocd.org/current/integration/#integration-with-bug-tracking-and-story-management-tools
[2]:
https://github.com/getsentry/gocd-webhook-notification-plugin/releases/tag/v0.0.6
'do_deactivate_user' is used inside an outer db transaction
created in 'sync_user_from_ldap'.
`transaction.atomic()` block in 'do_deactivate_user' resulted
in savepoint creation.
This commit adds `savepoint=False` to avoid that.
'bulk_remove_subscriptions' is used inside an outer db transaction
created in 'do_change_bot_owner'.
`transaction.atomic()` block in 'bulk_remove_subscriptions'
resulted in savepoint creation.
This commit adds `savepoint=False` to avoid that.
'do_activate_mirror_dummy_user' is used inside an outer db
transaction created in 'registration_helper'.
`transaction.atomic()` block in 'do_activate_mirror_dummy_user'
resulted in savepoint creation.
This commit adds `savepoint=False` to avoid that.
'get_or_create_direct_message_group' is used inside an outer
db transaction created in 'edit_scheduled_message'.
`transaction.atomic()` block in 'get_or_create_direct_message_group'
resulted in savepoint creation.
This commit adds `savepoint=False` to avoid that.
'compose_views' is used inside an outer db transaction created in
'update_user_group_backend'.
`transaction.atomic()` block in 'compose_views' resulted in
savepoint creation.
This commit adds `savepoint=False` to avoid that.
This commit adds `durable=True` to the outermost db transactions
created in the following:
* confirm_email_change
* handle_upload_pre_finish_hook
* deliver_scheduled_emails
* restore_data_from_archive
* do_change_realm_subdomain
* do_create_realm
* do_deactivate_realm
* do_reactivate_realm
* do_delete_user
* do_delete_user_preserving_messages
* create_stripe_customer
* process_initial_upgrade
* do_update_plan
* request_sponsorship
* upload_message_attachment
* register_remote_server
* do_soft_deactivate_users
* maybe_send_batched_emails
It helps to avoid creating unintended savepoints in the future.
This is as a part of our plan to explicitly mark all the
transaction.atomic calls with either 'savepoint=False' or
'durable=True' as required.
* 'savepoint=True' is used in special cases.
Separate `avatars/<email_or_id>/medium?` endpoints into distinct
endpoints for email-based and user ID-based access. This change aligns
avatar endpoints with Zulip’s existing API path conventions (e.g., the
`users/` endpoint).
As we would like to send similar notifications for other billing
state changes (for all BillingSession types), it makes sense to
move the logic for creating and sending these admin realm internal
messages to the BillingSession framework in the corporate app.
In the case that a channel with the specified name does not exist,
we now send direct messages to the admin realm administrators with
the channel, topic and message so that the information is not lost
and so that the channel for these messages can be created.
This commit adds a --reset-level optional argument to
send_zulip_update_announcements management command to
reset all the active realms to a given level.
Links to zulip messages can now be written as
`#**channel_name > topic_name @ message_id**.`
The `message_id` is replaced with `💬` in the rendered
message.
Fixes part of #31920
The classes StreamPattern and StreamTopicPattern both had a
separate copy of the function `find_stream_id` which did the same
thing. Since another Pattern will be added as a part of #31920, it
is a good idea to move that function into a superclass which is then
inherited by all the related patterns.
Fixes part of #31920
This adds `normalize_body_for_import` to normalize messages from
third-party importers by removing NUL bytes and also updates import
test files data to test this.
Fixes#31930.
add_subgroups_to_user_group and remove_subgroups_from_user_group
are already inside outer db transactions. This commit explicitly
adds 'savepoint=False' to avoid creating savepoints.
This commit adds 'durable=True' to the outermost transaction
in 'remote_server_post_analytics'.
It also adds 'savepoint=False' to inner transaction.atomic
decorator to avoid creating savepoint.
This is as a part of our plan to explicitly mark all the
transaction.atomic decorators with either 'savepoint=False' or
'durable=True' as required.
* 'savepoint=True' is used in special cases.
This commit adds 'durable=True' to the outermost transaction
in 'delete_message_backend'.
It also adds 'savepoint=False' to inner transaction.atomic
decorator to avoid creating savepoint.
This is as a part of our plan to explicitly mark all the
transaction.atomic decorators with either 'savepoint=False' or
'durable=True' as required.
* 'savepoint=True' is used in special cases.
This commit adds 'durable=True' to the outermost transactions
of the following functions:
* do_create_multiuse_invite_link
* do_revoke_user_invite
* do_revoke_multi_use_invite
* sync_ldap_user_data
* do_reactivate_remote_server
* do_deactivate_remote_server
* bulk_handle_digest_email
* handle_customer_migration_from_server_to_realm
* add_reaction
* remove_reaction
* deactivate_user_group
It helps to avoid creating unintended savepoints in the future.
This is as a part of our plan to explicitly mark all the
transaction.atomic decorators with either 'savepoint=False' or
'durable=True' as required.
* 'savepoint=True' is used in special cases.
This commit adds 'durable=True' to the outermost transaction
in 'do_invite_users'.
It also adds 'savepoint=False' to inner transaction.atomic
decorators to avoid creating savepoints.
Note about the documentation: There are currently two "Save changes"
buttons on the Airbyte "Notifications" settings page, so the
instructions specify which one to use for clarity.
Currently, the GitHub webhook sends activity from both public and private
repositories, which could lead to unintended disclosure of sensitive
information from private repositories.
This commit introduces a ignore_private_repositories parameter to the
webhook URL. When set to true, the webhook ignore processing activity from
private repositories, ensuring that such activities are not posted to
Zulip streams. By default, if the parameter is omitted or set to false,
activities from both public and private repositories are processed
normally. This provides users with the flexibility to control the
visibility of private repository activities without altering the default
behavior.
More importantly, this introduces a cleaner mechanism for individual
incoming webhooks to declare support for settings not common to all
webhook integrations.
Fixes#31638.
During account creation when a user opted to import settings
from an existing account, the "Mark visibility_policy_banner as
read" step was raising integrity error.
It is because 'copy_onboarding_steps' is already executed earlier
in the 'do_create_user' codeflow. If the source profile had already
marked 'visibility_policy_banner' as read, we were facing integrity
error.
This commit fixes the bug.
Previously, errors were returned using Zulip's default format,
which did not match Slack's expected response structure.
This change ensures that errors in the Slack incoming webhook handler
return JSON responses in Slack's expected format: {ok: false, error:
"error string"}.
Fixes: #31878.
Unfortunately, because this migration has already been run for many
installations, we need to ship another copy of the migration.
It should be a noop when repeated.
html2text mangles Unicode by default, with a --unicode-snob option to
disable it. If I have to get called a “snob” for wanting to correctly
support non-English languages, then uh, I’ll take one for the team.
https://github.com/Alir3z4/html2text/blob/2024.2.26/html2text/config.py#L111-L150
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Currently, Slack messages containing hyperlinks
(e.g.,<http://foo.com|Foo!>) are converted like
normal links. This commit reformats Slack
hyperlinks into Zulip-friendly markdown
(e.g., [Foo!](http://foo.com)).
Part of #32165.
Previously, the `group_id` was not returned in the success
response of the user group creation API.
This commit updates the API to return a success response
containing the unique ID of the user group with the key
`group_id`. This enhancement allows clients to easily reference
the newly created user group.
Fixes: #29686
Removed `edit_topic_policy` property, as the permission
to move messages between topcis is now controlled by
`can_move_messages_between_topics_group` setting.
This is a somewhat hacky and fragile fix. Due to the order in which
imports seem to happen, the original ordering breaks
RealmImportExportTest: if one of the `use_s3_backend` tests runs before
test_import_realm, the latter will fail while processing thumbnailing,
as S3UploadBackend ends up leaking and
zerver.worker.thumbnail.upload_backend is still set to S3.
By making that mock.patch the first one that gets entered, and thus the
last one to get cleaned up, we fix the leak and upload_backend is set
back to LocalUploadBackend as it should.
Streams should not be marked as private, and subscribers
of the deactivated stream should not be removed.
Update the confirmation message when archiving a stream.
`is_archived` field is added to the stream and types.
Include a new `archived_channeels` client capability, to allow clients
to access data on archived channels, without breaking
backwards-compatibility for existing clients that don't know how to
handle these.
Also, included `exclude_archived` parameter to `/get-streams`,
which defaults to `true` as basic clients may not be interested
in archived streams.
As several archived streams may have the same new name,
it is essential to verify whether any stream, regardless
of its current status (active or archived), already has that name
before executing any renaming operation.
Previously, the hashing logic for static avatar files hashed the default
and medium files separately, which didn’t match how user-uploaded
avatars work—where you just add the "-medium.png" suffix to get the
medium version. Since we don’t have clear documentation for avatars yet,
this caused some issues for the mobile apps.
This commit makes sure the default and its medium variation share the
same hash.
PieterCK
Giovanni Silva
Mateusz Mandera
Harsh
swayam0322
Prakhar Pratyush
Lauryn Menard
Tim Abbott
Alya Abbott
Kislay Udbhav Verma
Harsh
aniebietafia
Aditya Kumar Kasaudhan
Anders Kaseorg
Vector73
sanchi-t