Commit Graph

16809 Commits

Author SHA1 Message Date
Ujjawal Modi fd0434a052 realm_emoji: Refactor code for flushing realm_emoji cache.
Subsequent commits will add "on_delete=models.RESTRICT"
relationships, which will result in the RealmEmoji
objects being deleted after Realm has been deleted from
the database.

In order to handle this, we update get_realm_emoji_dicts,
get_realm_emoji_cache_key, get_active_realm_emoji_cache_key,
get_realm_emoji_uncached and get_active_realm_emoji_uncached
functions to accept realm_id as parameter instead of realm
object, so that the code for flushing the cache works even
after the realm is deleted. This change is fine because
eventually only realm_id is used by these functions and
there is no need of the complete realm object.
2023-06-28 18:03:32 -07:00
Ujjawal Modi 1be30d85ce import_realm: Create Stream, UserGroup and Realm in a transaction.
Make the import of `Realm`, `Stream` and `UserGroup` objects be
done in single transaction, to make the import process in general
more atomic.

This also removes the need to temporarily unset the Stream references
on the Realm object.  Since Django creates foreign key constraints
with `DEFERRABLE INITIALLY DEFERRED`, an insertion of a Realm row can
reference not-yet-existing Stream rows as long as the row is created
before the transaction commits.

Discussion - https://chat.zulip.org/#narrow/stream/101-design/topic/New.20permissions.20model/near/1585274.
2023-06-28 18:03:32 -07:00
Sahil Batra 74af803ec3 tests: Use check_add_user_group to create groups.
This commit changes the code in test_user_groups.py to use
check_add_user_group function to create user groups instead
of directly using django ORM to make sure that settings
would be set to the correct defaults in further commits.
2023-06-28 18:03:32 -07:00
Sahil Batra 138a67d97e types: Add default_group_name field to GroupPermissionSetting type.
This commit adds default_group_name field to GroupPermissionSetting
type which will be used to store the name of the default group for
that setting which would in most cases be one of the role-based
system groups. This will be helpful when we would have multiple
settings and we would need to set the defaults while creating
realm and streams.
2023-06-28 18:03:32 -07:00
Zixuan James Li b6d1e56cac queue_processors: Avoid queue worker timeouts in tests.
For tests that use the dev server, like test-api, test-js-with-puppeteer,
we don't have the consumers for the queues. As they eventually timeout,
we get unnecessary error messages. This adds a new flag, disable_timeout,
to disable this behavior for the test cases.
2023-06-28 11:06:24 -07:00
Lalit a686c0cc02 docs: Add documentation for delete emoji endpoint.
This endpoint was previously marked as `intentionally_undocumented`
but that was mistake.

Removed `intentionally_undocumented` and added proper documentation
with valid `python_example` for this Endpoint.

Fixes: #24084
2023-06-28 10:23:47 -07:00
Alex Vandiver bc3c2476d2 stripe: Nicknames are optional on plans. 2023-06-27 18:02:50 -07:00
Zixuan James Li 8b42f7ccfa test_user_groups: Check for updates/deletion of the user groups.
This verifies that updates of the user group name/description are
correctly done by doing additional queries. This also empathsizes on
checking that the state before and after API calls are indeed different.
2023-06-27 18:02:05 -07:00
Zixuan James Li 8493440049 test_user_groups: Check for subgroup membership changes.
This extracts a helper to test if changes are actually made to the
subgroups via the API.
2023-06-27 18:02:05 -07:00
Zixuan James Li d37f309a3c test_user_groups: Extract user memberships helper.
We extract the checks needed for user membership changes into a method,
verifying that the members of the user group are matching the expected
values exactly.
2023-06-27 18:02:05 -07:00
Zixuan James Li 4adb9dd2bc test_user_groups: Clean up typos. 2023-06-27 18:02:05 -07:00
Lauryn Menard 6f58994ef4 openapi-tests: Add coverage for validating example events.
Adds testing coverage for validating the documented examples for
each event in the `api/get-events` endpoint documentation.

This will help us catch basic typos / mistakes when adding new
event examples. And if fields / objects are removed or modified
for existing events in the API, then failing to update the
examples for those changes will also be caught by this additional
test coverage.

Adding new fields / objects to existing event schemas without
updating the example will not be caught unless the new field
is marked as required in the documentation.
2023-06-27 11:27:38 -07:00
Lauryn Menard b1b0296a21 api-docs: Update subscription peer_add and peer_remove events.
Updates the example for both of these events in the documentation
to be the current version. These were missed when the feature
level 35 updates were made to the API specification for these
events, see commit noted below.

Also, for completeness, adds Changes notes for feature level 35
and feature level 19, for these events.

The feature level 35 changes were made in commit 7ff3859136.
The feature level 19 changes were made in commit 00e60c0c91.
2023-06-27 11:27:38 -07:00
Lauryn Menard a810b58418 api-docs: Correct example for drafts remove event.
Corrects what seems to have been a typo when the documentation was
added for drafts in commit 6fee946a43.
2023-06-27 11:27:38 -07:00
Lauryn Menard 5ac37a9f75 api-docs: Correct example for realm_bot delete event.
Updates the example for the realm_bot delete event so that it does
not have a full_name field.

This was a pre-existing error in the documentation when the remove
and delete events shared the same event documentation. They were
separated in the documentation in commit fae3f1ca53.

The difference between these two events was noted when they were
added to `event_schema.py` in commit 385050de20.
2023-06-27 11:27:38 -07:00
Lauryn Menard 5939ffffee api-docs: Correct example/description for update_message_flags remove.
Updates the documented example for the update_message_flags remove
event so that the message ID that is the key for the object is
correctly shown as a string.

Also updates the description of these objects so that it is
rendered correctly in the documentation.
2023-06-27 11:27:38 -07:00
Lauryn Menard 2edcfb65bb api-docs: Remove sender_short_name from message event example.
Removes the `sender_short_name` from the example for the message
event in `/get-events`.

Also, to make this complete, adds Changes notes for the feature
level 26 changes that were made to the message objects returned
in the message events for `/get-events` and in the messages
array for the `/get-messages` response.

The field was originally removed from message objects in
commit b375581f58.
2023-06-27 11:27:38 -07:00
Lauryn Menard 89794ce47c api-docs: Update mute and unmute user endpoint documentation.
Updates the main descriptions for the mute a user and unmute a
user endpoint documentation. Also, revises the `muted_user_id`
parameter description and changes note for feature level 188.

The original feature level changes were made in #26005.
2023-06-27 11:24:30 -07:00
Alex Vandiver 21aeb4a040 slack: Handle the special case of permissions denied on team.info call.
This is a follow-up to 4c8915c8e4, for
the case when the `team:read` permission is missing, which causes the
`team.info` call itself to fail.  The error message supplies
information about the provided and missing permissions -- but it also
still sends the `X-OAuth-Scopes` header which we normall read, so we can
use that as normal.
2023-06-27 11:04:41 -07:00
Lauryn Menard e1c809a71e api-docs: Clarify what topics are returned by `get-stream-topics`.
Updates the main description for the `get-stream-topics` endpoint
so that it is clear that the topics for private streams with protected
history are limited to the topics / messages the user has access to.

And updates that documentation and the help center documentation for
bot permissions / abilities, to clarify that bots have the same
restriction and can only access messages / topics that are sent after
the bot (not the bot's owner) subscribed to the stream.
2023-06-26 20:26:30 -07:00
Satyam Bansal 92a1e3464d
integrations: Allow null pull request review body in GitHub Integration.
There are instances in which the pull request review body
can be null.

Fixes #26055.
2023-06-26 11:35:14 -07:00
Satyam Bansal 0945f2dee6 integrations: Separate issue labeled events in GitHub Integration.
This commit creates separate events for issue labeled and
unlabeled notifications. This allows the end-users to choose
whether they want these notifications or not.

Fixes #25789.
2023-06-26 11:04:32 -07:00
Satyam Bansal 842e9d1aca integrations: Improve GitHub issue labeled and unlabeled notifications.
Earlier, the notifications had no information about the labels
being added or removed.
2023-06-26 11:04:32 -07:00
Satyam Bansal 8fc28be8ca integrations: Add issue unlabeled fixture to GitHub Integration. 2023-06-26 11:04:32 -07:00
Satyam Bansal b4674892a4 integrations: Add issue labeled fixture to GitHub Integration. 2023-06-26 11:04:32 -07:00
David Rosa 04ead84c4d lib-markdown: Rename confusing variable name in `tabbed_sections.py`.
- Renames `data_language` -> `data_tab_key`.
- Renames `tab_name` -> `tab_key`.
2023-06-24 07:47:25 -07:00
David Rosa 0e0512df92 widgets: Rename confusing variable name in `tabbed_instructions.ts`.
The `tabbed_instructions` widget used for both language toggles in our
API documentation and app toggles in our Help Center documentation
misleadingly calls the identifier for the tab `language` in local
variables and its interface.

- Renames local variables `language` -> `tab_key`.
- Renames HTML data attributes `data-language` -> `data-tab-key`.

Fixes #24669.
2023-06-24 07:47:25 -07:00
Lauryn Menard 73fd729c4b message-flags: Rename classes for direct message dicts.
Renames `UnreadPrivateMessageInfo` and `RawUnreadPrivateMessageDict`
to be `UnreadDirectMessageInfo` and `RawUnreadDirectMessageDict`
instead.
2023-06-23 11:24:13 -07:00
Lauryn Menard 8c0d89a225 delete-user: Update management command help doc-string.
Updates the help doc-string for the delete_user management command
to use "direct message" instead of "private message".
2023-06-23 11:24:13 -07:00
Lauryn Menard d3f7cfccbc zerver: Update comments with "private message" or "PM".
Updates comments/doc-strings that use "private message" or "PM" in
files in the `/zerver` directory to instead use "direct message".
2023-06-23 11:24:13 -07:00
Lauryn Menard b75c99b893 webhooks: Update references to "private message" and "PM".
Updates references to "private message" and "PM" to instead be
"direct message".
2023-06-23 11:24:13 -07:00
Lauryn Menard 2eeeda7694 mattermost: Update references to "private message" and "PM".
Updates references to "private message" and "PM" in the data import
and related tests for Mattermost to be "direct message" or "DM"
instead.
2023-06-23 11:24:13 -07:00
Lauryn Menard d53b854a7c backend-tests: Update "private message" or "PM" to "direct message".
Updates comments and test strings/names with "private message" or
"PM" to use "direct message" instead.
2023-06-23 11:24:13 -07:00
Alex Vandiver 38d1b3314a convert_slack_data: Document "--token" more correctly. 2023-06-23 11:09:45 -07:00
Alex Vandiver 4c8915c8e4 slack: Provide more information when a Slack token fails to validate. 2023-06-23 11:09:45 -07:00
Alex Vandiver 1b2ba4e09d test_slack_importer: Switch to xoxb tokens, which is what we accept. 2023-06-23 11:09:45 -07:00
Lauryn Menard f461f28feb api-docs: Update endpoints in streams section for permissions info.
Updates the `api/subscribe` and `api/update-stream` endpoint docs
to note that streams' permissions impact whether a user/admin can
subscribe users and/or update a stream's permissions settings.

Updates the `api/archive-stream` and `api/delete-topic` endpoint
docs to note that they are only available to org admins.
2023-06-23 11:03:40 -07:00
Alex Vandiver 6c3969f893 name_restrictions: Reject anything with zulip or kandra in it.
This is primarily to prevent impersonation, such as `zulipteam`.  We
only enable these protections for CORPORATE_ENABLED, since `zulip` is
a reasonable test name for self-hosters.
2023-06-23 10:45:40 -07:00
Anders Kaseorg c09e7d6407 codespell: Correct “requestor” to “requester”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-06-20 16:17:55 -07:00
Alex Vandiver d30ab34e3d circleci: Support the "ping" event type. 2023-06-20 14:26:31 -04:00
Lauryn Menard 27878cce87 api-docs: Update `zulip.yaml` for "direct message".
Updates the API documentation in `zerver/openapi/zulip.yaml` to use
"direct message" instead of "private message".
2023-06-19 16:56:15 -07:00
Anders Kaseorg 98310f269b middleware: Do not consume StreamingHttpResponse.streaming_content.
streaming_content is an iterator. Consuming it within middleware
prevents it from being sent to the browser.

https://docs.djangoproject.com/en/4.2/ref/request-response/#streaminghttpresponse-objects

“The StreamingHttpResponse … has no content attribute. Instead, it has
a streaming_content attribute. This can be used in middleware to wrap
the response iterable, but should not be consumed.”

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-06-19 15:04:18 -07:00
Anders Kaseorg 7657cb4a0f django_api: Extract send_event_on_commit helper.
django-stubs 4.2.1 gives transaction.on_commit a more accurate type
annotation, but this exposed that mypy can’t handle the lambda default
parameters that we use to recapture loop variables such as

    for stream_id in public_stream_ids:
        peer_user_ids = …
        event = …

        transaction.on_commit(
            lambda event=event, peer_user_ids=peer_user_ids: send_event(
                realm, event, peer_user_ids
            )
        )

https://github.com/python/mypy/issues/15459

A workaround that mypy accepts is

        transaction.on_commit(
            (
                lambda event, peer_user_ids: lambda: send_event(
                    realm, event, peer_user_ids
                )
            )(event, peer_user_ids)
        )

But that’s kind of ugly and potentially error-prone, so let’s make a
helper function for this very common pattern.

        send_event_on_commit(realm, event, peer_user_ids)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-06-19 13:42:40 -07:00
Alex Vandiver 77c146b8b0 send_email: Delete ScheduledEmail objects with no recipients.
9d97af6ebb addressed the one major source of inconsistent data which
would be solved by simply re-attempting the ScheduledEmail row.  Every
other instance that we have seen since then has been a corrupt or
modified database in some way, which does not self-resolve.  This
results in an endless stream of emails to the administrator, and no
forward progress.

Drop this to a warning, and make it remove the offending row.  This
ensures we make forward progress.
2023-06-19 13:40:50 -07:00
Anders Kaseorg 92c83c1df4 tests: Remove assert_streaming_content helper in favor of getvalue.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-06-15 16:49:27 -07:00
Zixuan James Li 5901ffb0ab models: Fix typo in comment.
"This consistent ordering is important to prevent to prevent"
->
"This consistent ordering is important to prevent"
2023-06-15 17:50:34 -04:00
Prakhar Pratyush 134058b06d settings: Configure 'enable_followed_topic_audible_notifications'.
This commit makes it possible for users to control the
audible desktop notifications for messages sent to followed topics
via a global notification setting.

There is no support for configuring this setting through the UI yet.
2023-06-13 18:01:41 -07:00
Prakhar Pratyush a848c744c3 settings: Configure 'enable_followed_topic_desktop_notifications'.
This commit makes it possible for users to control the
visual desktop notifications for messages sent to followed topics
via a global notification setting.

There is no support for configuring this setting through the UI yet.
2023-06-13 18:01:41 -07:00
Prakhar Pratyush e71d3ada87 settings: Add wildcard mention notifications for the followed topics.
This commit makes it possible for users to control the wildcard
mention notifications for messages sent to followed topics
via a global notification setting.

There is no support for configuring this setting
through the UI yet.
2023-06-13 18:01:41 -07:00
Prakhar Pratyush d73c715dc2 settings: Add push notifications for the followed topics.
This commit makes it possible for users to control
the push notifications for messages sent to followed topics
via a global notification setting.

There is no support for configuring this setting
through the UI yet.
2023-06-13 18:01:41 -07:00