Commit Graph

19205 Commits

Author SHA1 Message Date
Sahil Batra f197d881ca streams: Fix handling empty groups when computing setting values.
Since "Nobody" and other user-defined empty groups has no
members or subgroups, we need a UserGroup query.

Since we are doing UserGroup query, updated the code to
check whether the setting is set to anonymous group or
not just after doing UserGroup query and not in membership
queries.

Also, added tests to check query count when setting is set
to anonymous group.
2024-11-22 10:48:41 -08:00
Prakhar Pratyush ded6bfd3f2 add_new_user_history: Remove the RECENT_MESSAGES_TIMEDELTA limit.
We give the new users some messages in their feed.

Earlier, we were including upto 1000 messages which were sent
within the last 12 weeks.

For realms with low-traffic it results in very few messages
being included in the new user's feed.

This commit removes the 12 week limit.
Now, we simply include upto 1000 recent messages.
2024-11-22 10:41:27 -08:00
Harsh c570194ff2 tests: Add assertion for database queries count.
This commit extends the test for the `/api/v1/register` endpoint
for spectator users by adding an assertion to check the number
of database queries.

Fixes: #32409
2024-11-21 19:02:09 -08:00
Prakhar Pratyush c5b3d2e434 custom_check: Add rule to avoid creating savepoints.
This commit adds a custom rule to check python files
and raise lint error if they have transaction.atomic used
without any argument or savepoint=True is used.

It helps to avoid creating unintended savepoints in the future.
2024-11-21 14:55:15 -08:00
Prakhar Pratyush 62b3e49075 transaction: Add 'savepoint=True' explicitly in tests, if used.
We create savepoint at a couple of places in backend tests
to avoid rollback due to error raised within test's transaction.

This commit explicitly adds 'savepoint=True' at those places.
2024-11-21 14:55:15 -08:00
Prakhar Pratyush fdcd518e26 test_user_groups: Add `durable=True` to the independent transaction.
This commit adds `durable=True` to the short independent db transaction
in `UserGroupRaceConditionTestCase.tearDown` method.

This is as a part of our plan to explicitly mark all the
transaction.atomic calls with either 'savepoint=False' or
'durable=True' as required.
2024-11-21 14:55:15 -08:00
Prakhar Pratyush b369177341 embed_links: Add `savepoint=False` to avoid creating savepoints.
It helps to avoid creating unintended savepoints in the future.

This is as a part of our plan to explicitly mark all the
transaction.atomic calls with either 'savepoint=False' or
'durable=True' as required.
2024-11-21 14:55:15 -08:00
Prakhar Pratyush 175104ea01 streams: Add 'ChannelEmailAddress' model.
This commit removes the 'email_token' field from the
'Stream' model, introduces a new model 'ChannelEmailAddress',
and backfills it.

This is a prep work towards our plan to generate unique
channel emails for different users, which can be used
to check post permissions at message send time.
2024-11-21 14:53:28 -08:00
Anders Kaseorg 801e267e00 integrations: Properly pass data-categories as a JSON array.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-11-21 14:41:36 -08:00
Sahil Batra b6f895f61c streams: Optimize DB queries for computing permission setting values.
This commit updates code to use union so that we can compute values
for permission settings using a single DB query.
2024-11-21 11:28:54 -08:00
Sahil Batra d1e76a9281 streams: Allow admins to unsubscribe others irrespective of setting. 2024-11-21 11:28:54 -08:00
Niloth P 619ada36ed integrations: Refine Mention example screenshot. 2024-11-21 11:09:40 -08:00
Niloth P 5de2a7115c integrations: Refine Gitea example screenshot. 2024-11-21 11:09:40 -08:00
Niloth P e1f256c905 integrations: Refine Basecamp example screenshot. 2024-11-21 11:09:40 -08:00
Niloth P 8e17ef3507 integrations: Refine CircleCI example screenshots.
- Removed the extra screenshot.
- Updated the fixture used to generate the screenshot to use a more
suitable topic name and commit message description.
2024-11-21 11:09:40 -08:00
Niloth P ab7d5ceacc integrations: Refine Zendesk example screenshot. 2024-11-21 11:09:40 -08:00
Niloth P 8582e78f02 integrations: Refine Front example screenshot. 2024-11-21 11:09:40 -08:00
Vector73 de44eb6a1b settings: Remove `invite_to_realm_policy` setting.
Removed `invite_to_realm_policy` property, as the permission
to invite other users to the organization is now controlled by
`can_invite_users_group` setting.
2024-11-20 13:35:05 -08:00
Vector73 4e89b4a88c settings: Add `can_invite_users_group` realm setting.
Added `can_invite_users_group` realm setting to replace
`invite_to_realm_policy`.
2024-11-20 13:35:05 -08:00
Sahil Batra a19287d581 streams: Optimize fetching setting values for web public subs.
This commit updates code to fetch the group setting values
efficiently for web public subscription objects.
2024-11-19 13:26:42 -08:00
Sahil Batra f4c00ce053 streams: Fix setting values passed in never subbed dicts.
This commit fixes the code to pass AnonymousSettingGroupDict
value instead of group ID when the setting is set to anonymous
groups in never subscribred streams subscription dicts.
2024-11-19 13:26:42 -08:00
oscar 9d1ca5476a integration-docs: Remove create channel for generate webhook URL docs.
Updates text of generate-webhook-url-basic.md to replace the use of
the "Create the channel" step.

Removes {!create-channel.md!} from all the integrations docs that
also include {!generate-webhook-url-basic.md!}.

Fixes #32269.
2024-11-19 13:02:01 -08:00
Anders Kaseorg c48ef926e1 mypy: Add types-defusedxml.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-11-19 11:09:34 -08:00
Anders Kaseorg 532aee926c requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-11-19 11:09:34 -08:00
jitendra-ky ca14366e38 import: Replace generic Exception with CommandError.
This change improves error handling in `do_import_realm` by replacing
the use of a generic Exception with CommandError. The updated approach
provides clearer, user-friendly error messages when there is a version
mismatch between the exported data and the Zulip server.

Fixes #32292.
2024-11-18 18:35:14 -08:00
Tim Abbott 18246ebd9f i18n: Update translation data from Transifex. 2024-11-18 16:12:31 -08:00
Sahil Batra d5a391a56b streams: Optmize code to send events on creating stream.
This commit updates code to effectively compute the setting
values when creating stream object to be sent in stream creation
events.
2024-11-18 11:55:19 -08:00
Sahil Batra 7adc83d2a0 streams: Optimize code for computing stream objects.
This commit updates code to optimize code for computing stream
objects to be sent with stream creation event and in response
for 'GET /streams/{stream_id}' endpoint by optimizing number
of queries while computing values for group permission settings.

This change does not benefit much currently as we only have one
stream group permission setting, but is important before we add
more stream permission settings.

There are a couple of places left where we can still optimize
the code and that would be done in further commits.
2024-11-18 11:55:19 -08:00
Sahil Batra 2cc0f482e1 streams: Refactor code to compute setting group values.
This commit adds a new function to compute setting group
values for a list of streams, so we can avoid having duplicate
code for computing setting group IDs from streams.
2024-11-18 11:55:19 -08:00
Sahil Batra b20c24c09d streams: Compute object only when needed.
This commit updates code to compute the values for group permission
settings in send_stream_creation_events_for_previously_inaccessible_streams
only when we need to send the events. This helps us in avoiding
unnecessary DB queries.
2024-11-18 11:55:19 -08:00
Sahil Batra f2158c42a7 streams: Do not call "locals()" inside loop.
There is no need to call "locals()" inside the loop for
stream permission settings. It should just be called once
to get values of all the settings passed to the endpoint.

It was fine as we only had one group permission setting
for stream, but is a good fix before we add more settings.
2024-11-18 11:55:19 -08:00
Shubham Padia b6ebf143cc streams: Backend changes to support anonymous groups.
can_remove_subscribers_group setting can now be set to
anonymous user groups.

Co-authored-by: Sahil Batra <sahil@zulip.com>
2024-11-16 17:11:08 -08:00
Sahil Batra b78ca79ccf user_groups: Define AnonymousSettingGroupDict in types.py.
AnonymousSettingGroupDict is now defined in types.py instead
of user_groups.py to avoid import cycles in future commits.
2024-11-15 16:36:37 -08:00
Sahil Batra 10ae4ffea4 subscription: Refactor code for creating subscription dicts.
This commit refactors code to create subscription dicts so
that we can efficiently compute can_remove_subscribers_group
setting when we add support to set the setting to anonymous
groups.
2024-11-15 16:36:37 -08:00
David Flanagan eec9b8e683
integrations: Avoid UUIDs in topics for linear.
The initial implementation for this feature used the issue UUID
to create the topics, as some of the webhook events only provide
this and it is the only way to group sub-issues with parents.

However, if we sacrifice this and allow sub-issues to be
their own topic then we can avoid using the UUID as the topic name.

I think the quality of life improvement to topics here warrants this change,
as UUID topic names can be come unruly very quickly.

Part of #23118.
2024-11-15 11:06:39 -08:00
whilstsomebody 8356a9d9e4
narrow_filter: Add negation support for in:home narrow filter.
The in:home narrow filter is used to filter messages that appear
in the home view, i.e., messages that are not muted. Conversely,
`-in:home` should filter messages that are not in the home view,
i.e., muted messages. However, `-in:home` did not work as expected
because this filter lacked negation support, unlike similar code
paths.

This commit adds negation support for the in:home filter.

For more information, see:
<https://chat.zulip.org/#narrow/channel/378-api-design/topic/mark.20muted-topic.20messages.20as.20read/near/1980534>.
2024-11-14 15:10:01 -08:00
Prakhar Pratyush 6820cbb980 realm_export: Change tarball_size_bytes to PositiveBigIntegerField.
This commit updates `tarball_size_bytes` field from
`PositiveIntegerField` to `PositiveBigIntegerField` to support
larger values.
2024-11-14 10:09:03 -08:00
Anders Kaseorg 2de648df02 makemessages: Write with orjson.
orjson’s use of Unicode is more consistent with what we get from
Transifex.  (We could alternatively use json’s ensure_ascii=False
flag.)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-11-13 17:40:51 -08:00
Anders Kaseorg 01540b45b0 web: Set "type": "module" and convert various CJS files to ESM.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-11-13 09:18:56 -08:00
Anders Kaseorg 687f1b1651 tests: Rename the Node tests to *.cjs.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-11-13 09:18:56 -08:00
Anders Kaseorg b3f18b805d ruff: Enable B008 function-call-in-default-argument.
https://docs.astral.sh/ruff/rules/function-call-in-default-argument/

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-11-10 22:43:55 -08:00
Mateusz Mandera 70fb590466 change_realm_subdomain: Revoke user sessions.
If we don't revoke sessions, this can lead to issues when we import a
realm (e.g. from Slack) into the original subdomain for the same
organization. Because then, users who already had sessions before the
move end up stuck:
- when they try to open Zulip on the original subdomain, which now has
the imported instance, they get redirected to the moved subdomain due to
their pre-existing session cookie.
- they can't even log out to get rid of the bugged cookie, because the
account they're logged into in that session is now on the moved
subdomain.
2024-11-10 19:13:14 -08:00
Mateusz Mandera 1f21b7437c test_import_export: Don't hard-code ZULIP_VERSION in fixtures.
Otherwise, these tests fail if ZULIP_VERSION is different locally from
what's hard-coded. Use a placeholder instead and replace dynamically in
a helper function.
2024-11-10 19:12:39 -08:00
PieterCK 0d7199b22e data_import: Add migration status file to converted exports.
This commit updates all third-party importer tools (Slack, Mattermost,
and Rocket Chat) in the `zerver/data_import` directory to also output a
migration_status.json file in their output tarball.

This is required because all importable tarball will be checked for
migration compatibility during import.

Fixes #28443.
2024-11-08 15:52:45 -08:00
PieterCK a9838d8089 import: Verify exported realm's migration compatibility.
When transferring a realm to a server that has a different set of
applied migrations (different Zulip versions), there is a chance that
the imported data formats appear to be compatible but data invariants
could still be violated.

This commit adds an assertion during the import process to verify
that both the exported realm and the importing server have matching
Zulip versions and have a compatible set of migrations.
2024-11-08 15:52:45 -08:00
PieterCK 40bcb4b42b export: Add migration status file to export tarball.
This commit updates the export process to write the migration status of
the realm as a JSON file to be included in the export tarball.

This is a preparatory step for adding an assertion to ensure that the
importing and exporting realms have a compatible set of applied
migrations.
2024-11-08 15:52:45 -08:00
Giovanni Silva 70d559cafa integrations: Update New Relic integration.
Newrelic has introduced a new update. This commit
updates the exisiting integration by adding support
for new types of payloads and cleaning up legacy
codes and fixtures.

Fixes #29729.

Co-authored-by: Pieter CK <pieterceka123@gmail.com>
Co-authored-by: Lauryn Menard <lauryn@zulip.com>
2024-11-08 15:49:56 -08:00
Mateusz Mandera 420849ff6a slack: Call the correct resize_* function when importing realm icon.
For resizing the icon.png files, we use resize_avatar, not resize_logo.
This is pretty confusing - sure, for icons we use the same function as
for avatars, but we should have a proper name for the function called in
the icon context. So this commit also adds resize_realm_icon, and
changes the calls to resize_avatar in icon contexts to
resize_realm_icon.
2024-11-08 15:43:18 -08:00
Mateusz Mandera a669fece3a email_mirror: Check that email gateway bot can send to the channel.
Otherwise this leads to an uncaught exception when the check happens
later in `check_message`.
2024-11-08 13:13:15 -08:00
Harsh e468818d2b import: Remove skipping of too-long messages during import.
This commit eliminates the skipping of messages longer than 10K characters during the import process.
2024-11-07 16:04:14 -08:00