Commit Graph

3013 Commits

Author SHA1 Message Date
Alex Vandiver abed174b12 uploads: Add an endpoint which forces a download.
This is most useful for images hosted in S3, which are otherwise
always displayed in the browser.
2022-03-22 15:05:02 -07:00
Tim Abbott 6b00c748fd onboarding: Deduplicate realm creation initial user logic.
We now call this function inside do_create_user(...,
realm_creation=True), which generally improves readability and
robustness of the codebase.

This fixes a bug where this onboarding content was not correctly done
when creating a realm via LDAP, and also will be important as we add
new code paths that might let you create a realm.
2022-03-21 12:05:59 -07:00
Tim Abbott a117b224a7 onboarding: Refactor setup_realm_internal_bots call.
This improves robustness of any code paths calling do_create_realm,
which previously needed to call this correctly to achieve the same
results as creating a user via the UI.

This also fixes a bug where this code was not called if a realm were
created using the LDAP code path.
2022-03-21 12:05:59 -07:00
Tim Abbott fddd83394e create_user: Specify tos_version=None explicitly in automation.
This is an important design detail, so we document this aspect of
creating users via both the management command and API code paths with
an explicit parameter value and comment.
2022-03-21 12:05:59 -07:00
Felix 780bda0d40
integrations: Fix BigBlueButton password length.
The BigBlueButton integration had a problem with generating
the random password with only 12 characters. This would
cause the attendeePW to be the same as the moderatorPW,
which might be fine but seems like something that could be an
error in a future version of BigBlueButton.
2022-03-20 16:09:36 -07:00
strifel a967a86b10 integration: Generate dynamic name for BigBlueButton video calls.
The name for a BigBlueButton meeting is now generated from the stream
name and topic name.

The createTime option is used to have the user redirected to a link
that is only valid for this meeting.

Even if the same link in Zulip is used again, a new createTime
parameter will be created, as the Meeting on the BigBlueButton server
has to be recreated.

Fixes #16498.
Fixes #20509.
Fixes #20804.
2022-03-18 17:27:39 -07:00
Anders Kaseorg f21842e920 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-16 10:43:23 -07:00
Sahil Batra b86b9bdc02 realm: Create RealmAuditLog entry when removing realm playgrounds.
This commit also adds 'acting_user' parameter to
do_remove_realm_playground function.

Fixes a part of #21268.
2022-03-15 15:58:36 -07:00
Sahil Batra dea3389045 realm: Create RealmAuditLog entry when adding realm playgrounds.
This commit also adds 'acting_user' parameter to
do_add_realm_playground function.

Fixes a part of #21268.
2022-03-15 15:58:36 -07:00
Anders Kaseorg b70a071124 report: Strengthen report_csp_violations type using WildValue.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-15 13:02:02 -07:00
Anders Kaseorg 5f92078d07 request: Add a var_name parameter to converter.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-15 13:02:02 -07:00
Anders Kaseorg 5caeb8444b validator: Replace converter=orjson.loads with json_validator.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-15 13:02:02 -07:00
Anders Kaseorg c3788ab847 validator: Replace converter=int with json_validator=check_int.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-15 13:02:02 -07:00
Sahil Batra f31bb6754b user_groups: Pass list of user ids to remove_members_from_user_group.
We pass list of user ids instead of user profile objects to
remove_members_from_user_group. We still need to call user_id_to_users
in the views function instead of directly passing the ids to
remove_members_from_user_group to make sure we check whether all
ids are valid or not.
2022-03-14 18:52:47 -07:00
Sahil Batra 303fca2ce7 user_groups: Pass list of user ids to bulk_add_members_to_user_group.
We pass list of user ids instead of user profile objects to
bulk_add_members_to_user_group. We still need to call user_id_to_users
in the views function instead of directly passing the ids to
bulk_add_members_to_user_group to make sure we check whether all
ids are valid or not.
2022-03-14 18:52:47 -07:00
Kartik Srivastava eefaa9120f user_topic: Rename topic_mutes.py to user_topics.py. 2022-03-11 14:26:55 -08:00
Aman Agrawal 82837304ec api: Send full message in GET /messages/{message_id} response.
Previously, this URL just returned the `raw_content` field. It seems
cleanest to just make it a single-message variant of GET /messages,
deprecating the only format.
2022-03-11 10:25:22 -08:00
Sahil Batra ab5567e8c5 realm: Create RealmAuditLog entry while changing an allowed domain.
This commit also adds 'acting_user' argument for do_change_realm_domain
function.

Fixes a part of #21268.
2022-03-10 17:48:02 -08:00
Sahil Batra 5ef8da40a9 realm: Create RealmAuditLog entry while adding a new allowed domain.
This commit also adds 'acting_user' argument for do_add_realm_domain
function.

Fixes a part of #21268.
2022-03-10 17:48:02 -08:00
Steve Howell a90d9ef536 unread: Remove unused client parameter. 2022-03-10 10:04:51 -05:00
Tim Abbott d693a6717c i18n: Remove quote syntax from stream description notification.
Translators found it confusing, since it's not at all obvious that the
word "quote" should not be translated.

I'm not altogether happy with the code formatting for this.

While we're changing this, also standardize on the "```` quote" style
of quote blocks to ensure code/quote blocks in stream descriptions are
unlikely to conflict with this syntax.
2022-03-09 15:22:57 -08:00
NerdyLucifer 4b9770e270 stream_settings: Show stream privacy & description in stream events.
Provide stream privacy and description in stream notification events
when stream is created.
In function "send_messages_for_new_subscribers" for when stream is
created, put policy name and description of the stream.

Fixes #21004
2022-03-07 11:53:49 -08:00
Lauryn Menard 072051f81e api: Add additional fields to `edit_history` entries.
Since we've changed the database to contain these new fields, we just
need to stop dropping them in the API code.

This also changes the public API to match the database format again
by removing `prev_subject` from edit history API.

Adds an API changelog feature update for the renamed `prev_subject`
field (to `prev_topic`) and new fields (`topic` and `stream`)
in the message `edit_history`.

Also, documents said `edit_history` in the `MessagesBase` schema
in the api documentation, which is used by the `/get-messages`,
`/get-events` and `/zulip-outgoing-webhooks` endpoints.

Fixes #21076.

Co-authored-by: Lauryn Menard <lauryn.menard@gmail.com>
2022-03-04 10:25:48 -08:00
Tim Abbott 4e91d03d56 edit_history: Remove prev_subject from EditHistoryEvent type.
It is no longer possible for this field to appear in actual entries.
2022-03-04 10:25:48 -08:00
Tim Abbott 5e819cdebc edit_history: Support prev_subject not being present in database.
This commit serves as a checkpoint that can be deployed to production
before running the big Message format migration in the next commit.
2022-03-04 10:25:48 -08:00
Tim Abbott 85222b790d edit_history: Refactor to use FormattedEditHistoryEvent type.
We fix the mutation of caller and other bad patterns, as well as
adding explicit typing to make the code readable.

We also update the OpenAPI documentation for previously
undocumented `prev_strem` field in the `/get-message-history`
endpoint for API validation testing.

Co-authored-by: Lauryn Menard <lauryn.menard@gmail.com>
2022-03-04 10:25:48 -08:00
Mateusz Mandera 3c5ea830a9 create_preregistration_user: Add additional hardening assertion.
TestMaybeSendToRegistration needs tweaking here, because it wasn't
setting the subdomain for the dummy request, so
maybe_send_to_registration was actually running with realm=None, which
is not right for these tests.

Also, test_sso_only_when_preregistration_user_exists was creating
PreregistrationUser without setting the realm, which was also incorrect.
2022-02-25 14:02:24 -08:00
Mateusz Mandera 7b795b6338 registration: Change create_preregistration_user to take realm as arg.
create_preregistration_user is a footgun, because it takes the realm
from the request. The calling code is supposed to validate that
registration for the realm is allowed
first, but can sometimes do that on "realm" taken from something else
than the request - and later on calls create_preregistration_user, thus
leading to prereg user creation on unvalidated request.realm.

It's safer, and makes more sense, for this function to take the intended
realm as argument, instead of taking the entire request. It follows that
the same should be done for prepare_activation_url.
2022-02-25 14:02:24 -08:00
Mateusz Mandera 88917019f0 CVE-2022-21706: Prevent use of multiuse invites to join other orgs.
The codepaths for joining an organization via a multi-use invitation
(accounts_home_from_multiuse_invite and maybe_send_to_registration)
weren't validating whether
the organization the invite was generated for matches the organization
the user attempts to join - potentially allowing an attacker with access
to organization A to generate a multi-use invite and use it to join
organization B within the same deployment, that they shouldn't have
access to.
2022-02-25 14:02:24 -08:00
Sahil Batra 392b17da5f invite: Add backend support for "Never expires" option.
The database value for expiry_date is None for the invite
that will never expire and the clients send -1 as value
in the API similar to the message retention setting.

Also, when passing invite_expire_in_days as an argument
in various functions, invite_expire_in_days is passed as
-1 for "Never expires" option since invite_expire_in_days
is an optional argument in some functions and thus we cannot
pass "None" value.
2022-02-24 16:32:19 -08:00
Lauryn Menard 8e1f62af89 api: Include realm_web_public_access_enabled in get-server-settings.
Adds `realm_web_public_access_enabled` as a realm-specific server
setting potentially returned by the `/get-server-settings` endpoint
so that clients that support browsing of web-public stream content
without an account can generate a login page that supports that
type of access.
2022-02-22 10:11:11 -08:00
Sahil Batra 630b37f2d8 users: Refactor get_user_by_email.
This commit refactors get_user_by_email function
to use access_user_by_email which is similar to
already existing access_user_by_id and thus using
get_user_data function added recently.

We also remove the unnecessary check for email as
email will always be passed to this endpoint.

Preparatory commit for #10970.
2022-02-21 06:47:54 -08:00
Sahil Batra 5471584fa7 users: Add get_user_data function.
This commit adds get_user_data which is called by
get_members_backend to compute the client_gravatar
value and then return the data of a single user or
all accessible users.

This function will also be used by get_user_by_email
in further commtis.
2022-02-21 06:47:54 -08:00
Anders Kaseorg 1629d6bfb3 python: Reformat with Black 22 (stable).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-18 18:03:13 -08:00
Alya Abbott e93320c40d portico: Add a self-hosting page.
Co-authored-by: Aman Agrawal <amanagr@zulip.com>
2022-02-17 12:43:13 -08:00
Alex Vandiver a40b3e1118 realm_emoji: Stop swallowing all exceptions from upload_emoji_image.
Putting all of the logic in a `finally` block is equivalent to a bare
`except` block, which silently consumes all exceptions.

Move only the most-necessary parts into the except; this lets
`BadImageError` exceptions from `zerver/lib/upload.py` to escape,
allowing better the generic "Image file upload failed" to be replaced
with a more specific message.

It also allows unexpected exceptions, as the previous commit resolved,
to escape and 500.  This lets them be detected and resolved, rather
than give users a silently bad experience.
2022-02-17 12:19:47 -08:00
Anders Kaseorg 2a5adf1453 auth: Remove unused kwargs for saml_sp_metadata.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-16 13:01:34 -08:00
Anders Kaseorg ce4261cbfb auth: Fix incorrect kwargs forwarding for logout_then_login.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-16 13:01:34 -08:00
Anders Kaseorg 12da3ac0ad pgroonga: Re-enable PGroonga in development.
This partially reverts commit fdabf0b357
(#21104).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-13 19:11:49 -08:00
Tim Abbott fdabf0b357 pgroonga: Disable PGroonga in development to fix CI.
A recent Postgres upstream release appears to have broken PGroonga.
While we wait for https://github.com/pgroonga/pgroonga/issues/203 to
be resolved, disable PGroonga in our automated tests so that Zulip
CI passes.
2022-02-11 11:35:43 -08:00
Anders Kaseorg 29330c180a sqlalchemy_utils: Make get_sqlalchemy_connection a context manager.
Although our NonClosingPool prevents the SQLAlchemy connection from
closing the underlying Django connection, we still want to properly
dispose of the associated SQLAlchemy structures.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-10 11:59:41 -08:00
Anders Kaseorg 8e5ae4e829 message_fetch: Use Row._mapping.
Fixes these warnings with SQLALCHEMY_WARN_20=1:

RemovedIn20Warning: Using non-integer/slice indices on Row is
deprecated and will be removed in version 2.0; please use
row._mapping[<key>], or the mappings() accessor on the Result
object. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9)

RemovedIn20Warning: Using the 'in' operator to test for string or
column keys, or integer indexes, in a :class:`.Row` object is
deprecated and will be removed in a future release. Use the
`Row._fields` or `Row._mapping` attribute, i.e. 'key in
row._fields' (Background on SQLAlchemy 2.0 at:
https://sqlalche.me/e/b8d9)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-10 11:59:41 -08:00
Anders Kaseorg fbb8f54d80 message_fetch: Use modern select() calling style.
Fixes this warning with SQLALCHEMY_WARN_20=1:

RemovedIn20Warning: The legacy calling style of select() is deprecated
and will be removed in SQLAlchemy 2.0.  Please use the new calling
style described at select(). (Background on SQLAlchemy 2.0 at:
https://sqlalche.me/e/b8d9)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-10 11:59:41 -08:00
Anders Kaseorg 8769c0c485 message_fetch: Replace Select.column() with Select.add_columns().
Fixes “SADeprecationWarning: The Select.column() method is deprecated
and will be removed in a future release.  Please use
Select.add_columns() (deprecated since: 1.4)”.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-10 11:59:41 -08:00
Anders Kaseorg d5a5b0b5ac message_fetch: Replace SELECT coercion with .subquery().
Fixes “SADeprecationWarning: Implicit coercion of SELECT and textual
SELECT constructs into FROM clauses is deprecated; please call
.subquery() on any Core select or ORM Query object in order to produce
a subquery object.”

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-10 11:59:41 -08:00
madrix01 a1e71e8639 topic: Return JsonableError for race condition in topic mute.
To avoid an uncaught IntegrityError causing a 500 HTTP response in a
race between two processes trying to mute a topic, we catch the
integrity error and raise the error exception with status 400 we'd
have gotten if the second request had been a bit later.

Fixes #21011.
2022-02-09 14:35:12 -08:00
Anders Kaseorg b0ce4f1bce docs: Fix many spelling mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-07 18:51:06 -08:00
Tim Abbott de411be576 home: Simplify and comment terms of service notice implementation. 2022-02-04 15:48:38 -08:00
Aman Agrawal 37d7d6ccd4 tos: Adjust migration tos to look same as our new standard tos. 2022-02-04 15:48:38 -08:00
Lauryn Menard 3be622ffa7 backend: Add request as parameter to json_success.
Adds request as a parameter to json_success as a refactor towards
making `ignored_parameters_unsupported` functionality available
for all API endpoints.

Also, removes any data parameters that are an empty dict or
a dict with the generic success response values.
2022-02-04 15:16:56 -08:00