Commit Graph

16924 Commits

Author SHA1 Message Date
Alex Vandiver 1b2ba4e09d test_slack_importer: Switch to xoxb tokens, which is what we accept. 2023-06-23 11:09:45 -07:00
Lauryn Menard f461f28feb api-docs: Update endpoints in streams section for permissions info.
Updates the `api/subscribe` and `api/update-stream` endpoint docs
to note that streams' permissions impact whether a user/admin can
subscribe users and/or update a stream's permissions settings.

Updates the `api/archive-stream` and `api/delete-topic` endpoint
docs to note that they are only available to org admins.
2023-06-23 11:03:40 -07:00
Alex Vandiver 6c3969f893 name_restrictions: Reject anything with zulip or kandra in it.
This is primarily to prevent impersonation, such as `zulipteam`.  We
only enable these protections for CORPORATE_ENABLED, since `zulip` is
a reasonable test name for self-hosters.
2023-06-23 10:45:40 -07:00
Anders Kaseorg c09e7d6407 codespell: Correct “requestor” to “requester”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-06-20 16:17:55 -07:00
Alex Vandiver d30ab34e3d circleci: Support the "ping" event type. 2023-06-20 14:26:31 -04:00
Lauryn Menard 27878cce87 api-docs: Update `zulip.yaml` for "direct message".
Updates the API documentation in `zerver/openapi/zulip.yaml` to use
"direct message" instead of "private message".
2023-06-19 16:56:15 -07:00
Anders Kaseorg 98310f269b middleware: Do not consume StreamingHttpResponse.streaming_content.
streaming_content is an iterator. Consuming it within middleware
prevents it from being sent to the browser.

https://docs.djangoproject.com/en/4.2/ref/request-response/#streaminghttpresponse-objects

“The StreamingHttpResponse … has no content attribute. Instead, it has
a streaming_content attribute. This can be used in middleware to wrap
the response iterable, but should not be consumed.”

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-06-19 15:04:18 -07:00
Anders Kaseorg 7657cb4a0f django_api: Extract send_event_on_commit helper.
django-stubs 4.2.1 gives transaction.on_commit a more accurate type
annotation, but this exposed that mypy can’t handle the lambda default
parameters that we use to recapture loop variables such as

    for stream_id in public_stream_ids:
        peer_user_ids = …
        event = …

        transaction.on_commit(
            lambda event=event, peer_user_ids=peer_user_ids: send_event(
                realm, event, peer_user_ids
            )
        )

https://github.com/python/mypy/issues/15459

A workaround that mypy accepts is

        transaction.on_commit(
            (
                lambda event, peer_user_ids: lambda: send_event(
                    realm, event, peer_user_ids
                )
            )(event, peer_user_ids)
        )

But that’s kind of ugly and potentially error-prone, so let’s make a
helper function for this very common pattern.

        send_event_on_commit(realm, event, peer_user_ids)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-06-19 13:42:40 -07:00
Alex Vandiver 77c146b8b0 send_email: Delete ScheduledEmail objects with no recipients.
9d97af6ebb addressed the one major source of inconsistent data which
would be solved by simply re-attempting the ScheduledEmail row.  Every
other instance that we have seen since then has been a corrupt or
modified database in some way, which does not self-resolve.  This
results in an endless stream of emails to the administrator, and no
forward progress.

Drop this to a warning, and make it remove the offending row.  This
ensures we make forward progress.
2023-06-19 13:40:50 -07:00
Anders Kaseorg 92c83c1df4 tests: Remove assert_streaming_content helper in favor of getvalue.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-06-15 16:49:27 -07:00
Zixuan James Li 5901ffb0ab models: Fix typo in comment.
"This consistent ordering is important to prevent to prevent"
->
"This consistent ordering is important to prevent"
2023-06-15 17:50:34 -04:00
Prakhar Pratyush 134058b06d settings: Configure 'enable_followed_topic_audible_notifications'.
This commit makes it possible for users to control the
audible desktop notifications for messages sent to followed topics
via a global notification setting.

There is no support for configuring this setting through the UI yet.
2023-06-13 18:01:41 -07:00
Prakhar Pratyush a848c744c3 settings: Configure 'enable_followed_topic_desktop_notifications'.
This commit makes it possible for users to control the
visual desktop notifications for messages sent to followed topics
via a global notification setting.

There is no support for configuring this setting through the UI yet.
2023-06-13 18:01:41 -07:00
Prakhar Pratyush e71d3ada87 settings: Add wildcard mention notifications for the followed topics.
This commit makes it possible for users to control the wildcard
mention notifications for messages sent to followed topics
via a global notification setting.

There is no support for configuring this setting
through the UI yet.
2023-06-13 18:01:41 -07:00
Prakhar Pratyush d73c715dc2 settings: Add push notifications for the followed topics.
This commit makes it possible for users to control
the push notifications for messages sent to followed topics
via a global notification setting.

There is no support for configuring this setting
through the UI yet.
2023-06-13 18:01:41 -07:00
Prakhar Pratyush 5e5538886f settings: Add email notifications for the followed topics.
This commit makes it possible for users to control
the email notifications for messages sent to followed topics
via a global notification setting.
Although there is no support for configuring this setting
through the UI yet.

Add five new fields to the UserBaseSettings class for
the "followed topic notifications" feature, similar to
stream notifications. But this commit consists only of
the implementation of email notifications.
2023-06-13 18:01:41 -07:00
Zixuan James Li 693b3679e3 muted users: Add support to muting bots.
We intentionally disallow muting bots previously upon
a pending design decision in #16915.
This lifts that constraint.

Fixes #22693.
2023-06-13 16:44:12 -07:00
Satyam Bansal 06c49969a0 integrations: Update links for various events in GitLab Integration.
After the release of GitLab 16.0, legacy URLs were modified to
include a "/-/" delimiter in the URLs.

Fixes #25643.
2023-06-12 17:14:18 -07:00
Satyam Bansal 45757ca7a1 integrations: Modify URL for note events in GitLab Integration.
Previosly, the URL was created manually by joining different
fields of the payload.
2023-06-12 17:14:18 -07:00
Satyam Bansal 508514c870 integrations: Update note fixtures for GitLab Integration. 2023-06-12 17:14:18 -07:00
Satyam Bansal 16563a3217 integrations: Add support for sample events in Sentry Integration.
Fixes #25778.
2023-06-12 17:05:12 -07:00
Satyam Bansal 580d8c4dfe integrations: Add exception fixture for Vue in Sentry Integration. 2023-06-12 17:05:12 -07:00
Satyam Bansal 3bdb806fba integrations: Check for Raven SDK only on python in Sentry Integration.
Fixes part of #25778.
2023-06-12 17:05:12 -07:00
Satyam Bansal 142e455d81 integrations: Add support for Rails backend in Sentry Integration. 2023-06-12 17:05:12 -07:00
Satyam Bansal 9e793c37e6 integrations: Update documentation for Sentry Integration. 2023-06-12 17:05:12 -07:00
Alex Vandiver 76d7a5a53a dev_settings: Remove `THUMBNAIL_IMAGES` from test_extra_settings.
THUMBNAIL_IMAGES was previously set to true as there were tests on a new
thumbnail functionality. The feature was never stable enough to remain in
the codebase and the setting was left enabled. This setting also doesn't
reflect how the production deployments are and it has been decided that we
should drop setting from test_extra_settings altogether.

Co-authored-by: Joseph Ho <josephho678@gmail.com>
2023-06-12 16:26:55 -07:00
Alex Vandiver fbb831ff3b uploads: Allow access to the /download/ variant anonymously.
This was mistakenly left off of b799ec32b0.
2023-06-12 12:55:27 -07:00
Alex Vandiver 0dbe111ab3 test_helpers: Switch add/remove_ratelimit to a contextmanager.
Failing to remove all of the rules which were added causes action at a
distance with other tests.  The two methods were also only used by
test code, making their existence in zerver.lib.rate_limiter clearly
misplaced.

This fixes one instance of a mis-balanced add/remove, which caused
tests to start failing if run non-parallel and one more anonymous
request was added within a rate-limit-enabled block.
2023-06-12 12:55:27 -07:00
Sahil Batra ea1357be66 user_groups: Prevent cycles when adding subgroups for a user group.
The user group depedency graph should always be a DAG.
This commit adds code to make sure we keep the graph DAG
while adding subgroups to a user group.

Fixes #25913.
2023-06-12 11:06:49 -07:00
Anders Kaseorg 85681546ce openapi: Convert deprecated Spec.create to Spec.from_dict.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-06-11 13:51:17 -07:00
Tim Abbott f63973c0cf integrations: Remove ancient legacy trello plugin docs.
This has been "legacy" since 2016, so surely nobody is using it.
2023-06-09 15:08:24 -07:00
Zixuan James Li b67c354826 user_groups: Make system groups creation atomic.
We want to make sure that the system groups, once created, will always
have the GroupGroupMemberships fully set up.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-06-09 15:07:37 -07:00
Prakhar Pratyush 79e5d32ef6 mention: Refactor 'possible_mentions' to return a dataclass.
This prep commit refactors 'possible_mentions' to
return a dataclass instead of a tuple for better readability.
2023-06-07 16:55:31 -07:00
Prakhar Pratyush d78bdc092f mention: Refactor 'extract_mention_text' to return dataclass.
This prep commit refactors 'extract_mention_text' to
return a dataclass instead of a tuple for better readability.
2023-06-07 16:55:31 -07:00
Prakhar Pratyush 0ea999b348 refactor: Convert the 'wildcards' list to a frozenset.
This commit converts the 'wildcards' list to a frozenset,
as we never intend to mutate 'wildcards'.
2023-06-07 16:55:30 -07:00
Zixuan Li e39e04c3ce
migration: Add `extra_data_json` for audit log models.
Note that we use the DjangoJSONEncoder so that we have builtin support
for parsing Decimal and datetime.

During this intermediate state, the migration that creates
extra_data_json field has been run. We prepare for running the backfilling
migration that populates extra_data_json from extra_data.

This change implements double-write, which is important to keep the
state of extra data consistent. For most extra_data usage, this is
handled by the overriden `save` method on `AbstractRealmAuditLog`, where
we either generates extra_data_json using orjson.loads or
ast.literal_eval.

While backfilling ensures that old realm audit log entries have
extra_data_json populated, double-write ensures that any new entries
generated will also have extra_data_json set. So that we can then safely
rename extra_data_json to extra_data while ensuring the non-nullable
invariant.

For completeness, we additionally set RealmAuditLog.NEW_VALUE for
the USER_FULL_NAME_CHANGED event. This cannot be handled with the
overridden `save`.

This addresses: https://github.com/zulip/zulip/pull/23116#discussion_r1040277795

Note that extra_data_json at this point is not used yet. So the test
cases do not need to switch to testing extra_data_json. This is later
done after we rename extra_data_json to extra_data.

Double-write for the remote server audit logs is special, because we only
get the dumped bytes from an external source. Luckily, none of the
payload carries extra_data that is not generated using orjson.dumps for
audit logs of event types in SYNC_BILLING_EVENTS. This can be verified
by looking at:

`git grep -A 6 -E "event_type=.*(USER_CREATED|USER_ACTIVATED|USER_DEACTIVATED|USER_REACTIVATED|USER_ROLE_CHANGED|REALM_DEACTIVATED|REALM_REACTIVATED)"`

Therefore, we just need to populate extra_data_json doing an
orjson.loads call after a None-check.

Co-authored-by: Zixuan James Li <p359101898@gmail.com>
2023-06-07 12:14:43 -07:00
evykassirer aa270bcef0 search: Remove support for experimental search pills.
This in-progress feature was started in 2018 and hasn't
been worked on much since. It's already in a broken state,
which makes it hard to iterate on the existing search bar
since it's hard to know how those changes will affect search
pills.

We do still want to add search pills eventually, and when
we work on that, we can refer to this diff to readd the
changes back.
2023-06-06 18:36:02 -07:00
Anders Kaseorg b7909db987 ruff: Fix PLC0208 Use a sequence type when iterating over values.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-06-06 14:58:11 -07:00
Anders Kaseorg b907ad0dcb ruff: Fix more of RUF010 Use conversion in f-string.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-06-06 14:58:11 -07:00
Anders Kaseorg 92db6eba78 test_helpers: Convert TypedDict from queries_captured to dataclass.
An implicit coercion from an untyped dict to the TypedDict was hiding
a type error: CapturedQuery.sql was really str, not bytes.  We should
always prefer dataclass over TypedDict to prevent such errors.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-06-06 14:58:11 -07:00
Zixuan James Li 28ec7baaef zilencer: Make analytics bouncer forward-compatible with JSONField.
This adds support to accepting extra_data being dict from remote
servers' RealmAuditLog entries. So that it is forward-compatible with
servers that have migrated to use JSONField for RealmAuditLog just in
case. This prepares us for migrating zilencer's audit log models to use
JSONField for extra_data.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-06-05 17:38:10 -07:00
Zixuan James Li 71ab77db9a zilencer: Use more realistic audit log extra_data.
This prepares for the audit log migration which requires us to populate
a JSONField from the extra_data field. "data" is not representative of
the actual extra_data field for RealmAuditLog entries of event types
in SYNC_BILLING_EVENTS.

We intentionally leave the test cases unchanged without bothering to
verify if the extra_data arrives as-is to keep this change minimal.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-06-05 17:38:10 -07:00
Mateusz Mandera 06143943d8 migrations: Handle duplicate fk constraint in 0443.
It turns out that for some some deployments, there exists a second,
duplicate, foreign key constraint for user_profile_id. The logic below
would try to rename both to the same name, which would fail on the
second:

```
psycopg2.errors.DuplicateObject: constraint "zerver_userpresenceo_user_profile_id_d75366d6_fk_zerver_us" for relation "zerver_userpresence" already exists
```

Eliminate the duplicate constraint, rather than attempting to rename
it.  Also add a block, in case of future reuse of this pattern, which
caveats that this approach will not work in the presence of
explicitly-named indexes.  UserPresence happens to not have any, so
this technique is safe in this instance.

Co-authored-by: Alex Vandiver <alexmv@zulip.com>
2023-06-05 17:34:17 -07:00
Zixuan James Li a5cc3c5d45 users: Use a less generic response for unauthorized user creation.
This reduces confusion when an admin user tries to create users.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-06-05 17:33:26 -07:00
Lalit 0ced086821 docs: Fix data type for `submessages` field on `message` type.
Before this commit our docs mentioned `string[]` data type for
`submessages` field on the `message` object. This commit changes the
type to `object[]` and correctly mentions all fields of the `submessage`
object.
2023-06-05 16:21:01 -07:00
dhruv302003 34e16c1553 integrations: Refactor Github Integration to use get_sender_name. 2023-06-05 12:01:27 -07:00
dhruv302003 67125acaa6 integrations: Add user url for star event in Github Integration.
Fixes #25672.
2023-06-05 12:01:27 -07:00
Mateusz Mandera a55901aa67 migrations: Fix bug in migration 0439.
This code clearly meant to return host and returning realm.host is a
mistake. realm.host is not accessible in a migration due to being a
@property-decorated method. The code constructs the host var value just
above this line.
2023-06-05 12:16:41 -04:00
Anders Kaseorg b5e5728112 coverage: Clean up coverage configuration.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-31 13:53:04 -07:00
Anders Kaseorg 2db9c0bc21 openapi: Remove unused prune_schema_by_type function.
It’s unused since commit a881918a05
(#24979).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-31 13:53:04 -07:00
Anders Kaseorg 0eff862459 openapi: Remove unused get_event_type function.
It’s unused since commit 8108acbdfd
(#16000).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-31 13:53:04 -07:00
Lauryn Menard bca7683f58 api-docs: Revisions for feature level 161 changes.
Revises the API changelog entry for feature level 161 to document
the changes to `DELETE /users/me/subscriptions` and to explain
more clearly what the new `can_remove_subscribers_group_id`
parameter does.

Updates the feature level 161 changes notes and related descriptions
to include links and also more clearly explain the updates.

Also, updates the `GET /user_groups` example to better reflect what
is returned for system groups since this is now referenced in the
`can_remove_subscribers_group_id` parameter description.

The original API feature level 161 API documentation changes were
made in commit c3759814be and commit 73f11853ec.
2023-05-31 08:16:22 -07:00
Lauryn Menard b0b3230917 scheduled-messages: Update scheduled_message_id parameter for DELETE.
Updates the scheduled_message_id parameter for deleting scheduled
messages to use the to_non_negative_int converter function for
validation, which is used in other endpoints/views with an ID in
the request path.
2023-05-31 08:15:03 -07:00
rht 1c84f02f57 slack import: Convert threads to nicely named Zulip topics.
Fixes #9006.
2023-05-30 16:35:19 -07:00
sbansal1999 2d6e6369f3 integrations: Add Linear webhook integration.
Fixes part of #23118.
2023-05-30 10:48:19 -07:00
Prakhar Pratyush dcb163bbe3 user_topics: Update API documentation.
This commit adds the missing 'UNMUTED' visibility policy
to the documentation for 'api/get-events' and 'api/register-queue'.

It replaces INHERIT with NONE for a clearer name
in the 'api/update-user-topic' documentation.

Other smaller changes in wording to improve readability.
2023-05-30 10:46:05 -07:00
Sahil Batra a143d6b62c message_edit: Add period to error message raised due to time limit. 2023-05-29 14:49:32 -07:00
Satyam Bansal caf6506811 integrations: Simplify regex checks in GitHub Integration.
The regex check is replaced with a simpler "startswith" function.
2023-05-29 10:55:31 -07:00
Tim Abbott dce4a3c98e markdown: Remove most of Twitter integration.
Twitter removed their v1 API. We take care to keep the existing cached
results around for now, and to not poison that cache, since we might
be able replace this with something that can still use the existing
cache.
2023-05-29 10:43:35 -07:00
Alya Abbott 3ca20e3ea8 help: Document quick way to check Zulip version.
Version number was added to top section of gear menu in
4df8c6610f.

Version number was added to "About Zulip" in
668b5137b0.
2023-05-29 08:39:01 -07:00
Mateusz Mandera db7a7d589c actions: Handle cross-realm messages in do_scrub_realm.
This is necessary to properly scrub all data from the realm.
2023-05-28 15:24:31 -07:00
Anders Kaseorg 66501a8eae requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-26 22:09:18 -07:00
Anders Kaseorg 9797de52a0 ruff: Fix RUF010 Use conversion in f-string.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-26 22:09:18 -07:00
Lauryn Menard 0c09a4fffe scheduled-messages: Add API documentation for new endpoint.
Part of splitting creating and editing scheduled messages.
Final commit. Should be merged with previous commits in series.

Updates the API documentation for the new endpoint for editing
scheduled messages.
2023-05-26 18:05:55 -07:00
Lauryn Menard 957382253a scheduled-messages: Split out edit scheduled message endpoint.
Part of splitting creating and editing scheduled messages.
Should be merged with final commit in series. Breaks tests.

Splits out editing an existing scheduled message into a new
view function and updated `edit_scheduled_message` function.
2023-05-26 18:05:55 -07:00
Lauryn Menard 154af5bb6b scheduled-messages: Remove ID from create scheduled message.
Part of splitting creating and editing scheduled messages.
Should be merged with final commit in series. Breaks tests.

Removes `scheduled_message_id` parameter from the create scheduled
message path.
2023-05-26 18:05:55 -07:00
Lauryn Menard 7af5ceb1c5 scheduled-messages: Add direct scheduled message to populate_db.
Prep commit for splitting create/edit endpoint for scheduled
messages.

Because of `test-api` runs the tests in alphabetical order based on
the `operationId`, we need two scheduled messages in the test database.
The first for the curl example delete (delete-scheduled-message) and
the second for the curl example update (update-scheduled-message).
2023-05-26 18:05:55 -07:00
Lauryn Menard 2f196bff19 api-docs: Add when moving messages fields/parameters were added.
Adds API changelog feature level 1 and associated Changes notes
for when the `stream_id` parameter in the `PATCH /messages/message_id`
was added, and for when the `prev_stream` field was added to edit
history information for messages.

We're adding these to the Zulip 3.0 feature level 1 because
commit 843345dfee that introduced this field and this parameter
to the server / backend code was merged before the commit that added
the API feature level tracking, commit e3b90a5ec8, at level 1.
2023-05-26 11:57:08 -07:00
Lauryn Menard bb735460e5 api-docs: Update avatar_url descriptions for feature level 163 change.
Updates the descriptions of the `avatar_url` field in message and
user objects to be clear that the current user must have access
to the other user's real email address in order for the value to
ever be `null`.

Also adds a bullet point to the API changelog feature level 163
entry about this change.
2023-05-26 11:51:05 -07:00
Lauryn Menard 389e45625a api-docs: Clarify if "email" is the "Zulip API email".
Clarifies additional areas of the API documentation where a user's
email is mentioned / used where it could be useful to clarify
that the email in question is the "Zulip API email".
2023-05-26 11:51:05 -07:00
Lauryn Menard 8c7e18fd75 api-docs: Use reference link in `/register-queue` main description. 2023-05-26 11:51:05 -07:00
Lauryn Menard ee58910f1a api-docs: Update `/get-own-user` to consistently use "requesting user". 2023-05-26 11:51:05 -07:00
Lauryn Menard 7a4565e833 api-changelog: Update feature level 163 entry and changes notes.
Original commits for this feature level's changes were part
of #22994.
2023-05-26 11:51:05 -07:00
Lauryn Menard 526fd7f24b api-docs: Add "delivery_email" field to example responses. 2023-05-26 11:51:05 -07:00
Lauryn Menard 58a64b33c9 api-docs: Revise main description for /get-user-by-email endpoint. 2023-05-26 11:51:05 -07:00
Lauryn Menard 87e9482672 api-docs: Update "Zulip display email" to "Zulip API email".
In some instances, the email was not clearly defined as
Zulip display email, but the Zulip API email is what is being
returned.
2023-05-26 11:51:05 -07:00
Sahil Batra 48e99657ad events: Remove realm_community_topic_editing_limit_seconds.
This commit removes realm_community_topic_editing_limit_seconds
field from register response since topic edit limit is now
controlled by move_messages_within_streams_limit_seconds
setting.
We also remove DEFAULT_COMMUNITY_TOPIC_EDITING_LIMIT_SECONDS
constant since it is no longer used.
2023-05-25 17:26:21 -07:00
Lauryn Menard e41c235657 api-docs: Link client_capabilities parameter in property descriptions.
In the register response properties deprecated at feature level 89,
update the descriptions to link to the client_capabilities parameter
when referenced.

Also, moves the enter_send property to be in the same section of the
register response as other properties deprecated at this feature level.

These descriptions were originally added to these properties in
commit e6f828a8e2.
2023-05-25 12:04:03 -07:00
Satyam Bansal cae02dbca4 integrations: Ignore merge queue push events for GitHub Integration.
Previosuly, the server would send a 500 whenever a merge queue push
event was encountered.

Fixes #25629.
2023-05-24 15:33:50 -07:00
Lauryn Menard 1beebb9fa1 api-changelog: Update feature level 168 entry and changes notes.
Original API changelog entry from commit ae72777c77.
2023-05-24 11:38:00 -07:00
Mateusz Mandera dcbcb05655 saml: Make SP-initiated SLO work after signup. 2023-05-23 13:01:15 -07:00
Mateusz Mandera 04f5358a76 tests: Flush session in verify_desktop_flow_end_page in social auth.
As the relevant comment elaborates - what happens next in the test in
simulating the step that happens in the desktop app. Thus a new session
needs to be used. Otherwise, the old session created normally in the
browser pollutes the state and can give falsely passing tests.

This should be happening for all social auth tests using this, not just
in that one SAML test, thus moving it inside the helper method.
2023-05-23 13:01:15 -07:00
Mateusz Mandera 8fb0fe96c6 saml: Save SessionIndex in session and use when making a LogoutRequest.
This is a useful improvement in general for making correct
LogoutRequests to Idps and a necessary one to make SP-initiated logout
fully work properly in the desktop application. During desktop auth
flow, the user goes through the browser, where they log in through their
IdP. This gives them a logged in  browser session at the IdP. However,
SAML SP-initiated logout is fully conducted within the desktop
application. This means that proper information needs to be given to the
the IdP in the LogoutRequest to let it associate the LogoutRequest with
that logged in session that was established in the browser. SessionIndex
is exactly the tool for that in the SAML spec.
2023-05-23 13:01:15 -07:00
Mateusz Mandera 5dd4dcdebb saml: Make SP-initiated SLO work in the desktop application. 2023-05-23 13:01:15 -07:00
Mateusz Mandera 3f55c10685 saml: Rework SP-initiated logout config to support IdP-level config.
This gives more flexibility on a server with multiple organizations and
SAML IdPs. Such a server can have some organizations handled by IdPs
with SLO set up, and some without it set up. In such a scenario, having
a generic True/False server-wide setting is insufficient and instead
being able to specify the IdPs/orgs for SLO is needed.
2023-05-23 13:01:15 -07:00
Mateusz Mandera 0bb0220ebb saml: Implement SP-initiated Logout.
Closes #20084

This is the flow that this implements:
1. A logged-in user clicks "Logout".
2. If they didn't auth via SAML, just do normal logout. Otherwise:
3. Form a LogoutRequest and redirect the user to
https://idp.example.com/slo-endpoint?SAMLRequest=<LogoutRequest here>
4. The IdP validates the LogoutRequest, terminates its own user session
and redirects the user to
https://thezuliporg.example.com/complete/saml/?SAMLRequest=<LogoutResponse>
with the appropriate LogoutResponse. In case of failure, the
LogoutResponse is expected to express that.
5. Zulip validates the LogoutResponse and if the response is a success
response, it executes the regular Zulip logout and the full flow is
finished.
2023-05-23 13:01:15 -07:00
Mateusz Mandera dda4603f94 auth: Extract EXPIRABLE_SESSION_VAR_DEFAULT_EXPIRY_SECS. 2023-05-23 13:01:15 -07:00
Lauryn Menard eba83b9d87 api-docs: Expand `/update-message` main description for realm settings.
Expands the main description for the `/update-message` documentation
to include a list of the realm settings in the API that are relevant
to when users can update a message's content, topic or stream.
2023-05-23 11:11:20 -07:00
Lauryn Menard 8ec9abe00d api-changelog: Clarify feature level 159 descriptive text.
Original API changelog entries: commit 02eee3a04f and
commit 815bf609fa.
2023-05-23 11:11:20 -07:00
Lauryn Menard e4ae8b79c3 api-changelog: Clarify feature level 162 descriptive text.
Original API changelog entries: commit 73f0eae394 and
commit 2c4e076fef and commit 891f83601d.
2023-05-23 11:11:20 -07:00
Lauryn Menard 0e9c34a975 api-changelog: Clarify feature level 172 descriptive text.
Original API feature level entries: commit 440f9e397a and
commit bd7f728796.
2023-05-23 11:11:20 -07:00
Lauryn Menard 6d6a335e32 api-docs: Make realm_linkifiers current API clear in description.
Adjusts the descriptions of realm_linkifiers (and deprecated
realm_filters) events and register response fields so that the
description of the current API is complete without the feature
level 176 **Changes** notes.
2023-05-22 15:20:07 -07:00
Lauryn Menard fd02648b0e api-changelog: Update feature level 175 entry and related changes notes. 2023-05-22 13:21:03 -07:00
Lauryn Menard 3cb6c9aea9 api-docs: Add examples to `realm_filters` event prose description.
Adds examples of the regex pattern and old URL string format to
the deprecated `realm_filters` event and register response field.
The examples are in the prose description since the events are
no longer sent and therefore no longer tested.
2023-05-19 13:23:07 -07:00
Lauryn Menard 03a2c2da6a api-changelog: Update and clarify docs for feature level 176 entry.
Revises API changelog entry for missing endpoint method and to
clarify the overall text.

Updates Changes notes for feature level 176 to not have repetitive
text, so that the updates were clearer and more concise.

The original commit with the changes related to this API changelog
entry is commit 268f858f39.
2023-05-19 13:23:07 -07:00
Sahil Batra 4c4caa7be4 CVE-2023-32677: Check permission to subscribe other users in invites.
This commit updates the API to check the permission to subscribe other
users while inviting.  The API will error if the user passes the
"stream_ids" parameter (even when it contains only default streams)
and the calling user does not having permission to subscribe others to
streams.

For users who do not have permission to subscribe others, the
invitee will be subscribed to default streams at the time of
accepting the invite.

There is no change for multiuse invites, since only admins are allowed
to send them, and admins always have the permission to subscribe
others to streams.
2023-05-19 16:13:32 -04:00
Mateusz Mandera a23b077b79 CVE-2023-28623: Prevent unauthorized signup with ldap + external auth.
Since 74dd21c8fa in Zulip Server 2.1.0, if:
- ZulipLDAPAuthBackend and an external authentication backend (any aside
  of ZulipLDAPAuthBackend and EmailAuthBackend) are the only ones
  enabled in AUTHENTICATION_BACKENDS in /etc/zulip/settings.py
- The organization permissions don't require invitations to join

...then an attacker can create a new account in the organization with
an arbitrary email address in their control that's not in the
organization's LDAP directory.

The impact is limited to installations which have the specific
combination of authentication backends described above, in addition to
having the "Invitations are required for joining this organization
organization" permission disabled.
2023-05-19 16:13:00 -04:00
Mateusz Mandera 3ed2a30e01 maybe_send_to_registration: Remove password_required arg.
This argument was added with the default incorrectly set to `True` in
bb0eb76bf3 - despite
`maybe_send_to_registration` only ever being called in production code
in a single place, with `password_required=False` explicitly. And then
it just got carried forward through refactors.

`maybe_send_to_registration` was/is also called twice in tests, falling
back to the default, but the `password_required` value is irrelevant to
the tests - and if anything letting it use the `True` has been wrong,
due to not matching how this function is actually used.
2023-05-19 16:13:00 -04:00
Mateusz Mandera b55adbef3d export: Handle RealmAuditLog with .acting_user in different realm. 2023-05-19 11:12:19 -07:00
Alex Vandiver c978bfaa32 models: Add a unique index on UserProfile.api_key.
This prevents `get_user_profile_by_api_key` from doing a sequential
scan.

Doing this requires moving the generation of initial api_key values
into the column definition, so that even bare calls to
`UserProfile.objects.create` (e.g. from tests) call appropriately
generate a random initial value.
2023-05-19 11:11:04 -07:00
Lauryn Menard 145d6915c1 api-changelog: Add 2.1.0 entry for realm_default_external_accounts.
Adds an API changelog note to 2.1 for the addition of
realm_default_external_accounts to the `/register-queue` response.

Also adds a Changes note to the field in the endpoint's response
API documentation.

The original commit that added it to that endpoint's response was
commit d7ee2aced1.
2023-05-19 10:50:17 -07:00
Alex Vandiver b312a86ada sentry: Add the observed user's IP address before forwarding.
The default for Javascript reporting is that Sentry sets the IP
address of the user to the IP address that the report was observed to
come from[^1].  Since all reports come through the Zulip server, this
results in all reports being "from" one IP address, thus undercounting
the number of affected unauthenticated users, and making it difficult
to correlate Sentry reports with server logs.

Consume the Sentry Envelope format[^2] to inject the submitting
client's observed IP address, when possible.  This ensures that Sentry
reports contain the same IP address that Zulip's server logs do.

[^1]: https://docs.sentry.io/platforms/python/guides/logging/enriching-events/identify-user/
[^2]: https://develop.sentry.dev/sdk/envelopes/
2023-05-18 16:25:54 -07:00
Lauryn Menard 5802f7775f api-changelog: Update and clarify docs for feature level 178 entry.
Updates the descriptions and examples for there only being two key
values: "website" and "aggregated".

Also, clarifies that email keys are the Zulip display email.

And removes any descriptive text that says presence objects have
information about the clients the user is logged into.
2023-05-18 11:57:02 -07:00
Alex Vandiver 1184bdc934 push_notifications: Lock message while we mark it pending for push.
Deleting a message can race with sending a push notification for it.
b47535d8bb handled the case where the Message row has gone away --
but in such cases, it is also possible for `access_message` to
succeed, but for the save of `user_message.flags` to fail, because the
UserMessage row has been deleted by then.

Take a lock on the Message row over the accesses of, and updates to,
the relevant UserMessage row.  This guarantees that the
message's (non-)existence is consistent across that transaction.

Partial fix for #16502.
2023-05-18 11:53:21 -07:00
Anders Kaseorg 4b19863065 test_timeout: Skip test_timeout_warn on Python 3.11 for coverage issue.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-18 11:52:22 -07:00
Lauryn Menard 2f96b1756e api-changelog: Add notes for feature level 3 `zulip_version` change.
Original API changelog note was made in commit 2c63130195.
2023-05-18 08:51:46 -07:00
Lauryn Menard 3023091caa scheduled-messages: Add help center link to fetch / delete descriptions. 2023-05-18 08:45:51 -07:00
Lauryn Menard 67e1889817 scheduled-messages: Add explicit test with emails for direct message. 2023-05-18 08:45:51 -07:00
Lauryn Menard a2a50e1408 scheduled-messages: Simplify message_to type in check_schedule_message.
As of commit 38f6807af1, we accept only stream and user IDs for
the recipient information for scheduled messages, which means we
can simplify the type for `message_to` in `check_schedule_message`.
2023-05-18 08:45:51 -07:00
Lauryn Menard 1ad0ad8ece tests-scheduled-messages: Use "direct" instead of "private". 2023-05-18 08:45:51 -07:00
Lauryn Menard 8f4de3afc0 api-docs: Update create/edit scheduled message errors for IDs.
In commit 38f6807af1, we updated the `POST /scheduled_messages`
endpoint to only accept user IDs for direct messages. The endpoint
alread only accepted a stream ID for stream messages.

But the API documentation was not updated for the errors returned
when either a stream or user with the specified ID does not exist.

Updates the API documentation for the correct error responses.
2023-05-16 15:14:54 -07:00
Alex Vandiver 24c3e25f86 middleware: Redirect non-canonical realm domain names.
If a host is in REALM_HOSTS, it has its own domain name.  Redirect
access from other domain names to that name.
2023-05-16 15:13:51 -07:00
Alex Vandiver 724de9cd49 rocketchat: Treat users with "bot" roles as bots when importing.
We previously relied on `type`, but we have observed bots typed with a
`bot` role as well.
2023-05-16 15:10:58 -07:00
Alex Vandiver 34394cec9a rocketchat: Handle users with no email address set.
Fixes: #25596.
2023-05-16 15:10:58 -07:00
Alex Vandiver 3160c3cce0 realm_export: Return export id from POST which create it. 2023-05-16 14:05:01 -07:00
Alex Vandiver 7811e99548 realm_export: Handle hard head-of-queue failures.
Realm exports may OOM on deployments with low memory; to ensure
forward progress, log the start time in the RealmAuditLog entry, and
key off of the existence of that to prevent re-attempting an export
which was already tried once.
2023-05-16 14:05:01 -07:00
Alex Vandiver 4a43856ba7 realm_export: Do not assume null extra_data is special.
Fixes: #20197.
2023-05-16 14:05:01 -07:00
Alex Vandiver 5eeb616666 realm_export: Test failures during export. 2023-05-16 14:05:01 -07:00
Alex Vandiver 362177b788 workers: Run realm export with one thread if in low-memory environment.
We previously hard-coded 6 threads for the realm export; in low-memory
environments, spawning 6 threads for an export can lean to an OOM,
which kills the process and leaves a partial export on disk -- which
is then tried again, since the export was never completed.  This leads
to excessive disk consumption and brief repeated outages of all other
workers, until the failing export job is manually de-queued somehow.

Lower the export to only use on thread if it is already running in a
multi-threaded environment.  Note that this does not guarantee forward
progress, it merely makes it more likely that exports will succeed in
low-memory deployments.
2023-05-16 14:05:01 -07:00
Alex Vandiver 9f231322c9 workers: Pass down if they are running multi-threaded.
This allows them to decide for themselves if they should enable
timeouts.
2023-05-16 14:05:01 -07:00
Alex Vandiver 5329fed387 test_queue_worker: Do a full recursion on subclasses.
This makes it less likely we will accidentally fail to include a class
if the subclassing of QueueProcessingWorker changes, and lets mypy
more accurately understand the typing.
2023-05-16 14:05:01 -07:00
Sahil Batra 007a51f277 accounts: Allow user to change email visibility during first login.
We now allow users to change email address visibility setting
on the "Terms of service" page during first login. This page is
not shown for users creating account using normal registration
process, but is useful for imported users and users created
through API, LDAP, SCIM and management commands.
2023-05-16 13:52:56 -07:00
Sahil Batra 7f01b3fb63 users: Set tos_version to -1 for users who have not logged-in yet.
We now set tos_version to "-1" for imported users and the ones
created using API or using other methods like LDAP, SCIM and
management commands. This value will help us to allow users to
change email address visibility setting during first login.
2023-05-16 13:52:56 -07:00
Lauryn Menard 8d06fa7e38 api-docs: Update descriptions for display_emoji_reaction_users. 2023-05-16 12:31:03 -07:00
Lauryn Menard 8962565154 api-docs: Fix capitalization in descriptions with int values lists. 2023-05-16 12:31:03 -07:00
Lauryn Menard 239458a173 api-docs: Remove instances of "=>" in API documentation descriptions. 2023-05-16 12:31:03 -07:00
Lauryn Menard 63d51e8114 api-docs: Clarify uses of "=" in API documentation descriptions.
For cases of `name=value` in descriptions in the API documentation,
update to either use JSON style of `"name": value` when correct or
revise the descriptive text.
2023-05-16 12:31:03 -07:00
Lauryn Menard 09ab1be34f api-docs: Use backticks for descriptions referencing "null" values. 2023-05-16 12:31:03 -07:00
Lauryn Menard c43ea96c06 api-docs: Clean up instances of "None" that should be "null". 2023-05-16 12:31:03 -07:00
Lauryn Menard 4fcb243769 api-docs: Standardize on lowercase true, false and null.
Creates a custom linter rule for `zerver/openapi/zulip.yaml` to
only allow lowercase versions of "true", "false" and "null".

Updates existing documentation for new rules.
2023-05-16 12:31:03 -07:00
Alex Vandiver ffaccb8af2 outgoing_webhook: Respect settings.OUTGOING_WEBHOOK_TIMEOUT_SECONDS.
The use of the setting was accidentally removed in b88d7a741e, and
replaced with a static 10 seconds.
2023-05-16 07:00:37 -07:00
David Rosa be14ec2cab help: Rename "Starting a new private thread" to "... new direct message".
With the private messages -> direct messages migration, we should
rename the "Starting a new private thread" help center article.

- Renames article to "Starting a new direct message"
- Updates relevant section in /help/getting-started-with-zulip
- Fixes typo in /help/send-group-dm
- Updates file names and adds URL redirect.

Fixes #25506.
2023-05-15 16:13:55 -07:00
Alex Vandiver ca2ca030d2 migrations: Backfill missing RealmAuditLog entries for subscriptions.
Backfill subscription realm audit log SUBSCRIPTION_CREATED events for
users which are currently subscribed but don't have any subscription
events, presumably due to some historical bug.  This is important
because those rows are necessary when reactivating a user who is
currently soft-deactivated.

For each stream, we find the subscribed users who have no
subscription-related realm audit log entries, and create a
`backfill=True` subscription audit log entry which is the latest it
could have been, based on UserMessage rows.  We then optionally insert
a `DEACTIVATION` if the current subscription is not active.
2023-05-15 16:09:44 -07:00
Aman Agrawal c2145a8993 scheduled_message: Send `update` event on failed delivery. 2023-05-14 16:46:20 -07:00
Aman Agrawal 4dd32a4d85 scheduled_messages: Extract notify_update_scheduled_message. 2023-05-14 16:46:20 -07:00
Ujjawal Modi 2a6146110c subscriptions: Change in API used for adding new subscriptions.
Earlier when a user who is not allowed to add subscribers to a
stream because of realm level setting "Who can add users to streams"
is subscribing other users while creating a new stream than new stream
was created but no one is subscribed to stream.

To fix this issue this commit makes changes in the API used
for adding subscriptions. Now stream will be created only when user
has permissions to add other users.

With a rewrite of the test by Tim Abbott.
2023-05-14 11:19:05 -07:00
Ujjawal Modi a47569bf47 backend_tests: Add a test for subsribing others to public streams.
Earlier there was no backend test for subscribing others to
public streams in zephyr realm.

This commit adds a backend test for it.
2023-05-14 11:19:04 -07:00
Tim Abbott 2d3d3f6072 message_send: Don't mark scheduled messages to self as read.
The only reasonable intent for such a scheduled message is to remind
oneself of something at that time, which requires it being unread.

Fixes #25523.
2023-05-12 17:55:46 -07:00
Tim Abbott daf6fb17bf do_send_messages: Use mandatory kwargs pattern. 2023-05-12 17:55:46 -07:00
Mateusz Mandera 254ea4b0c8 social_auth: Save authentication method information in the session.
The immediate application of this will be for SAML SP-initiated logout,
where information about which IdP was used for authenticating the
session needs to be accessed. Aside of that, this seems like generally
valuable session information to keep that other features may benefit
from in the future.
2023-05-12 16:21:26 -07:00
Mateusz Mandera af9d1a7dfb register_remote_user: Use explicit kwargs list.
This is nicer that .pop()ing specified keys - e.g. we no longer will
have to update this chunk of code whenever adding a new key to
ExternalAuthDataDict.
2023-05-12 16:21:25 -07:00
Lauryn Menard 90cc2716f0 scheduled-messages: Update API dicts for `failed` boolean field.
Adds the `failed` boolean from the ScheduledMessage to the API dict
returned by scheduled message events and register response, and by
fetching the user's scheduled messages.

`failed` will only be true when the server has tried to send the
scheduled message and failed due to an error.
2023-05-12 15:48:59 -07:00
Lauryn Menard c7c67c01ce scheduled-messages: Update failure to send message fields for edits.
In the case of a user editing a scheduled message that the server
had failed to send at the scheduled time due to an error, we want
to update the `failed` and `failure_message` fields as the intent
is for the server to retry to send the scheduled message based on
the updated information provided by the user.
2023-05-12 15:48:59 -07:00
Lauryn Menard cc648a2c19 scheduled-messages: Send notification if send scheduled message fails.
In the case that there is an error when sending a scheduled message,
we now send a message from the notification bot to the user who
scheduled the message about the failure/error.

The notification message is not sent if the error when sending the
scheduled message was due to the realm or sender being deactivated.
2023-05-12 15:48:59 -07:00
Alex Vandiver a2ed0302ce streams: Prevent already-deactivated streams from being deactivated. 2023-05-12 13:26:43 -07:00
Alex Vandiver 5be7bc58fe upload: Use content_disposition_header from Django 4.2.
The code for this was merged in Django 4.2:
https://code.djangoproject.com/ticket/34194
2023-05-11 14:51:28 -07:00
Sahil Batra ddb0bb58ed tests: Add tests to update visibility policy when target topic is empty.
This commit adds a new test to check how the visibility policy updates
when moving messages to a topic that didn't exist previously.

This test also helps us adding coverage for the code which just
skips setting visibility_policy if there is no need to update the
value because both previous and new value of visibility policy
is INHERIT. The "actions/message_edit.py" file has 100% coverage
now and thus is removed from "not_yet_fully_covered" list.
2023-05-11 12:13:50 -07:00
Sahil Batra d645d5c0ec message_edit: Fix code to set visibility policy on moving messages.
The code for updating visibility policy values on moving messages
had two bugs.

- There was a typo in elif condition where "user_profile" was being
used instead of "user_profile_with_policy".

This commit fixes the typo.

- It was assumed that there would be no UserTopic rows for target
topic if the target topic didn't exist. But there can be such case
where some messages were sent to that topic and the user muted
the topic. But then the messages in that topic was deleted. In
such case there can be UserTopic rows for a stream-topic pair
that does not exist.

This commit fixes the code to handle such case as well and set
the visibility policy of new topic to what was set for the original
topic. This change simplifies the condition to just check whether
new_visibility_policy is equal to target_topic_visibility_policy
and skip if so, and update the visibility policy otherwise.

Due to this change, we now do not try to mute the already muted
topic if the topic is moved to a topic which didn't exist
previously and thus we modify the existing test to not expect
any INFO logs.
2023-05-11 12:13:50 -07:00
Sahil Batra a2600a2b97 tests: Add coverage to actions/message_edit.py.
This commit adds tests to cover the case of message editing
not allowed due to allow_message_editing set to False and
the case when there is no limit set when moving all messages
in a topic.

The "actions/message_edit.py" file does not have 100% coverage
still and it will be addressed in the next commit.
2023-05-11 12:13:50 -07:00
Sahil Batra 9fa67f0fa9 tests: Add coverage to actions/create_realm.py.
This commit adds test coverage to actions/create_realm.py.
The file is also removed from not_yet_fully_covered list
since it has 100% coverage now.
2023-05-11 12:13:50 -07:00
Sahil Batra 5e7d49d129 create_realm: Remove unused argument from do_create_realm.
We do not pass "email_address_visibility" to do_create_realm
anymore. It was passed before to set the setting for realms in
development database, but it has been changed since we changed
email_address_visibility to be a user-level setting instead
of realm-level setting since now it is set on RealmUserDefault
table.
2023-05-11 12:13:50 -07:00
Alex Vandiver 6b58f9f9fa integrations: Add support for MovieAdded Radarr hook. 2023-05-11 12:08:25 -07:00
Alex Vandiver 1d5bccf4c3 integrations: Add support for MovieFileDelete Radarr hook. 2023-05-11 12:08:25 -07:00
Alex Vandiver 79c1123700 validator: Generalize type of check_string_in argument. 2023-05-11 12:08:25 -07:00
Alex Vandiver 01241f0e72 integrations: Add support for MovieDelete Radarr hook. 2023-05-11 12:08:25 -07:00
Alex Vandiver 84b440f0e4 integrations: Add support for ApplicationUpdate Radarr hook. 2023-05-11 12:08:25 -07:00
Alex Vandiver 5931787f11 integrations: Support scans of untagged images in Harbor. 2023-05-11 11:42:54 -07:00
Alex Vandiver 6e842f54c1 webhooks: Add support for gmail and rules sources to Front integration. 2023-05-11 11:36:58 -07:00
Lauryn Menard 1d209220dd tests: Add coverage for error when editing a sent scheduled message.
Adds test coverage for the error sent for editing a scheduled
message that was successfully sent.

`zerver/actions/scheduled_messages.py` now has 100% test coverage
again.
2023-05-11 10:52:01 -07:00
Anders Kaseorg e88b2caeef Revert "db: Force use of TimeTrackingCursor to work around Django 4.2 bug."
This reverts commit f1925487e8.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-10 19:44:47 -07:00
Anders Kaseorg d0481be3e5 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-10 19:44:47 -07:00
Tim Abbott 2aae32f642 events: Fix apply_events when raw_unread_msgs not present.
We were missing a few checks for raw_unread_msgs being present before
trying to parse and update it.

The test only covers 2/3 of the cases, but I wasn't convinced it was
worth adding another test just for the corner case of removing a
message flag; this seems fairly unlikely to regress.
2023-05-10 13:44:35 -07:00
Alya Abbott a9d0656605 help: Document stream/DM switcher on "Mastering the compose box" page. 2023-05-10 11:37:36 -07:00
Sahil Batra 4ca887bade invites: Allow users to invite without specifying any stream to join.
We now allow users to invite without specifying any stream to join.
In such cases, the user would join the default streams, if any, during
the process of account creation after accepting the invite.

It is also fine if there are no default streams and user isn't
subscribed to any stream initially.
2023-05-09 17:05:17 -07:00
Sahil Batra 875bcb183e invites: Do not add user to default streams if streams list is empty.
We do not add user to the default streams if the streams list passed
while sending the invite (both email and multi-use) was empty since
invite explicitly selected to not subscribe the user to default
streams.
2023-05-09 16:59:06 -07:00
Sahil Batra ded6b7ede4 test_invite: Add test to verify stream subscriptions.
This commit adds test to verify whether the user was
subscribed to the streams selected when inviting them.
2023-05-09 16:59:06 -07:00
Sahil Batra aebf7bf0e1 test_invite: Fix typo in comment. 2023-05-09 16:59:06 -07:00
Aman Agrawal f025415f2b scheduled_messages: Add tests for delivery of scheduled messages. 2023-05-09 13:48:28 -07:00
Tim Abbott 602e4c2aa3 scheduled_messages: Add focused scheduled message indexes. 2023-05-09 13:48:28 -07:00
Tim Abbott 835f62617e scheduled_messages: Store the final delivered message ID.
This could be useful for debugging problems with the system
operationally.
2023-05-09 13:48:28 -07:00
Tim Abbott fe9656237e scheduled_message: Move remaining logic out of management command.
While this doesn't include tests yet, it will make it much easier to
write tests for the core server logic here.
2023-05-09 13:48:28 -07:00
Tim Abbott 20be0759b5 scheduled_message: Avoid delivering scheduled messages late. 2023-05-09 13:48:28 -07:00
Tim Abbott a065576db7 scheduled messages: Call check_message at send time. 2023-05-09 13:48:28 -07:00
Tim Abbott 7051d3416b scheduled_messages: Add reasonable failure handling.
Previously, it seemed possible for the scheduled messages API to try
to send infinite copies of a message if we had the very poor luck of a
persistent failure happening after a message was sent.

The failure_message field supports being able to display what happened
in the scheduled messages modal, though that's not exposed to the API
yet.
2023-05-09 13:48:28 -07:00
Tim Abbott 147e296e0a scheduled_messages: Add defensive assert for double sends. 2023-05-09 13:48:28 -07:00
Tim Abbott 24bb03c89f scheduled_messages: Add logging on successful send. 2023-05-09 13:48:28 -07:00
Tim Abbott 1fc6ed224a scheduled_messages: Remove batching logic.
The previous logic would attempt to send a large number of unrelated
messages in a single transaction, which is just asking for trouble in
the event that one of the attempts fails.
2023-05-09 13:48:28 -07:00
Tim Abbott d1dfab5b70 scheduled_messages: Live update #scheduled view after sending.
We apparently had neglected to send the actual event to clients that
the message was successfully sent.
2023-05-09 13:48:28 -07:00
Tim Abbott c8f3443fb6 schedules_messages: Move logic out of management command. 2023-05-09 13:48:28 -07:00
Tim Abbott 611cee177d deliver_scheduled_messages: Rename misleading message variable. 2023-05-09 13:48:28 -07:00
Alex Vandiver 1285c39098 zerver: Remove now-unused report/ endpoints. 2023-05-09 13:16:28 -07:00
Lauryn Menard e44520c8fe scheduled-messages: Limit `to` parameter to user and stream IDs.
For scheduled stream messages, we already limited the `to`
parameter to be the stream ID, but here we return a JsonableError
in the case of a ValueError when the passed value is not an integer.

For scheduled direct messages, we limit the list for the `to`
parameter to be user IDs. Previously, we accepted emails like
we do when sending messages.
2023-05-09 12:45:11 -07:00
sbansal1999 63fcfd543a integrations: Add documentation for Notion-Zapier Integration.
Fixes #25314.
2023-05-09 12:25:18 -07:00
sbansal1999 6e2423fe64 integrations: Update text in Zapier Documentation.
This commit updates the text to match the current version
displayed on the website.
2023-05-09 12:25:18 -07:00
Lauryn Menard df17a1bf28 test-events: Test do_delete_message with no messages specified.
Test coverage for `zerver/actions/message_delete.py`.

Both callers of this function would already return if there were
no Messages specified to delete, which is why existing tests did
not cover this.
2023-05-09 09:58:33 -07:00
Alex Vandiver 89d4737ab2 message_edit: Do not reuse user_profile variable in "for" loop.
Doing so causes the "username resolved this topic" or "this topic was
moved by username" notifications to be attributed to a random user who
had a visibility policy on the topic.
2023-05-09 09:38:38 -07:00
Lauryn Menard 02fafb0376 models: Update the references for API dicts for scheduled messages. 2023-05-09 07:36:05 -07:00
Mateusz Mandera 682a4d6f9e import_realm: Improve comment about sequencing of imported models.
The comment was outdated, currently we import UserProfiles before
realm_tables - because some models in realm_tables have a dependency on
UserProfile.

Also makes sense to elaborate a bit more in the comment that it's just
an outline of the ordering, not an exhaustive list.
2023-05-08 15:55:06 -07:00
Mateusz Mandera 0abf60fd01 scheduled_message: Make export/import work.
Closes #25130 by addressing the import/export part of it.
2023-05-08 15:55:06 -07:00
Mateusz Mandera 780ef71891 export: Fix typo in variable name. 2023-05-08 15:55:06 -07:00
Mateusz Mandera 414658fc8e scheduled_message: Handle attachments properly.
Fixes #25414.

We add Attachment.scheduled_messages relation to track ScheduledMessages
which reference the attachment.

The import bits can be done after merging this, by updating #25345.
2023-05-08 09:56:02 -07:00
Mateusz Mandera 4598607a46 test_uploads: Fix two typos. 2023-05-08 09:56:02 -07:00
sbansal1999 2a3d4041e0 integrations: Improve GitHub force push notifications.
Previously, we didn't explicitly display something special for force-pushes.

Fixes #21969.
2023-05-04 14:31:25 -07:00
sbansal1999 dba6f84b97 integrations: Add force push fixtures for GitHub Integration.
One fixture is just removing commits; the other removes some commits
but adds others.
2023-05-04 14:31:25 -07:00
Mateusz Mandera d0e9a77a57 migrations: Fix performance of migration 0436.
This was doing bulk_create in a loop for each realm, which is too slow
on very large servers. Just do a single bulk_create with a reasonable
batch_size at the end.
2023-05-03 11:09:23 -07:00
Tim Abbott 9bdb044ff4 i18n: Update translation data from Transifex. 2023-05-02 13:16:25 -07:00
Lauryn Menard b5c4064b7b api-docs: Clarify topic parameter for create-or-edit-scheduled-message. 2023-05-02 12:46:41 -07:00
Lauryn Menard a5b527f321 onboarding: Specialize Welcome Bot message for education organizations.
Because education organizations and users have slightly specialized
use cases, we update the Welcome Bot message content sent to new
users and new organization owners for these types of organizations
to link to help center articles/guides geared toward these users
and organizations.

Also, updates the demo organization warning to only go to the new
demo organization owner because the 30 day deletion text is only
definitely accurate when the organization is created.

Fixes #21694.
2023-05-01 16:48:48 -07:00
Lauryn Menard 06dd7a3a68 emails: Add corporate_enabled to context for emails.
In commit fc58c35c0, we added a check in various emails for the
settings.CORPORATE_ENABLED value, but that context is only always
included for views/templates with a request.

Here we add that to common_context, which is often used when there
is not a request (like with emails). And we manually add it to the
email context in various cases when there is not a user account to
call with common_context: new user invitations, registration emails,
and realm reactivation emails.
2023-05-01 10:32:43 -07:00
Lauryn Menard d41f12495d api-docs: Document create/edit scheduled message endpoint and events.
Adds API changelog update for the new endpoint added for creating
or editing scheduled messages, for the events sent for scheduled
messages (add, update, delete), and for the `scheduled_messages`
field in the register response.

Adds a specific API markdown documentation file for the create
or update scheduled messages endpoint, so that the parameters in
the cURL examples are accurate for the type of message (direct or
stream) and action (create or edit).

Bumps the API feature level, adds Changes notes to API documentation
and adds the scheduled message endpoints to the API documentation
sidebar.
2023-04-28 17:25:00 -07:00