Commit Graph

7268 Commits

Author SHA1 Message Date
Anders Kaseorg 02be415122 settings_type: Extract new module for types used in settings.
This breaks an import cycle that prevented django-stubs from inferring
types for django.conf.settings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-26 12:42:36 -07:00
Anders Kaseorg 9198fe4fac scim: Downgrade SCIMClient from a model to an ephemeral dataclass.
SCIMClient is a type-unsafe workaround for django-scim2’s conflation
of SCIM users with Django users.  Given that a SCIMClient is not a
UserProfile, it might as well not be a model at all, since it’s only
used to satisfy django-scim2’s request.user.is_authenticated queries.

This doesn’t solve the type safety issue with assigning a SCIMClient
to request.user, nor the performance issue with running the SCIM
middleware on non-SCIM requests.  But it reduces the risk of potential
consequences worse than crashing, since there’s no longer a
request.user.id for Django to confuse with the ID of an actual
UserProfile.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-26 11:36:48 -07:00
Lauryn Menard 37aca4ac67 user-status: Stop updating the UserStatus model for `away` updates.
Fourth step in making user status `away` a deprecated way to access
`presence_enabled` for clients supporting older servers, and
checkpoint commit prior to deleting the `status` field from the
UserStatus model.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard e36cfdb0a7 user-status: Send `away=True` if `!presence_enabled` for user status.
We stop sending the `away=True` based on the user's `UserStatus`
object having `status=AWAY`, and instead send that value if
`!presence_enabled` for the user.

Third step in making user status `away` a deprecated way to access
`presence_enabled` for clients supporting older servers.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 47c1dbaa7d user-status: Refactor function names with "user_info" for clarity.
Rename functions that refer to "user_info" without a reference to
"status" to help clarify in the backend between UserPresence
and UserStatus models.

Prep commit for migrating "unavailable" user status feature to
"invisible" user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 32381f8678 user-presence: Refactor function names with "status" for clarity.
Rename functions that refer to "status" without a reference to
"presence" to help clarify in the backend between UserPresence
and UserStatus models.

Prep commit for migrating "unavailable" user status feature to
"invisible" user presence feature.
2022-09-23 12:27:54 -07:00
David Rosa c6abb7bedf help: Restructure "Mastering the compose box" article.
This help center article should include more features rather than just
focusing on the "go to conversation" button. We should broaden and
restructure this page to cover other advanced features.

Refactors the "Go to conversation" section as step-by-step instructions,
and adds a `keyboard_tip`.

Adds new section "Toggle between Ctrl+Enter and Enter".
Deletes the "Enable Enter to send" help center article, and adds its
content as a new subheading in this section.
Updates existing links accordingly and adds a URL redirect.
Documents "Enable Control + Enter to send".
Tweaks intro paragraph of "Mastering the compose box".

Fixes: #22817.
2022-09-22 15:20:37 -07:00
yogesh sirsat 543f36b7da custom_profile_fields: Add "display_in_profile_summary" field in model.
To allow `custom_profile_field` to display in user profile popover,
added new boolean field "display_in_profile_summary" in its model class.

In `custom_profile_fields.py`, functions are edited as per conditions,
like currently we can display max 2 `custom_profile_fields` except
`LONG_TEXT` and `USER` type fields.

Default external account custom profile fields made updatable for only
this new field, as previous they were not updatable.

Fixes part of: #21215
2022-09-20 17:03:57 -07:00
Anders Kaseorg 2e9cd20380 timezone: Improve tzdata parser’s compatibility with zic(8).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-20 16:58:31 -07:00
Anders Kaseorg f929050230 external_accounts: Make ExternalAccount dataclass to fix typing.
Commit 1a426fa6be (#22977) changed name
to be a StrPromise rather than a str.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-20 15:14:33 -07:00
Yogesh Sirsat 1a426fa6be i18n: Add translation tags to default external accounts name. 2022-09-20 10:42:33 -07:00
Zixuan James Li 3193afffe8 test_runner: Remove monkey-patched SubSuiteList.
This monkey-patching approach is not meaningful when what we really need
is just the names of the test, that can already be done in
get_test_names.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-09-20 10:18:22 -07:00
Anders Kaseorg 23d9dd3163 streams: Fix can_remove_subscribers_from_stream type.
Added by commit c3fe8420fd (#22589).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-19 13:53:44 -07:00
Mateusz Mandera 29589a69eb bots: Replace some redundant logic with is_cross_realm_bot_email.
is_cross_realm_bot_email is just
`email.lower() in settings.CROSS_REALM_BOT_EMAILS` which is the same,
aside of looking at .lower() - which is actually more correct.
2022-09-17 17:52:01 -07:00
Anders Kaseorg 403837e52d python: Use ‘not in’ for negated membership tests
Fixes “E713 Test for membership should be `not in`” found by
ruff (https://github.com/charliermarsh/ruff).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-17 11:48:33 -07:00
Jason Hildebrand 7efd59b6d7 email_mirror: Remove limits (expiry, max uses) to improve usability.
These limits don't appear to provide useful security benefits, and
they do impact usability because they prevented email-based users from
replying more than once, or from replying to message more than 5 days
old.

Fixes #2755.
Fixes #19994.
2022-09-16 18:07:28 -07:00
Sahil Batra c3fe8420fd streams: Allow unsubscribing others based on can_remove_subscribers_group.
Current value of can_remove_subscribers_group field is admins system group
only so behavior is not changed. We would provide support to change this
setting from API and UI in further commits.
2022-09-14 16:14:15 -07:00
Sahil Batra b9248c75f4 stream: Add do_change_can_remove_subscribers_group and field to objects.
This commit adds do_change_can_remove_subscriber_group function for
changing can_remove_subscribers_group field of a stream. We also add
can_remove_subscribers_group_id field to stream and subscription
objects.

This function will be helpful for writing tests in next commit.
We would add API and UI support to change this setting in further
commits.
2022-09-14 16:03:11 -07:00
Sahil Batra b0de5c0f36 streams: Set can_remove_subscribers_group while creating streams.
This commit sets can_remove_subscribers_group to admins system
group while creating streams as it will be the default value
of this setting. In further we would provide an option to set
value of this setting to any user group while creating streams
using API or UI.
2022-09-14 16:03:11 -07:00
Sahil Batra 941bb5a372 import: Fix import related code for user groups.
We change the import order to import UserGroup objects before
Stream such that we can set can_remove_subscribers_group correctly.

We do not import UserGroupMembership objects here along with
UserGroup since UserProfile objects are not imported and
GroupGroupMembership are also imported later as these are not
required before.
2022-09-14 16:03:11 -07:00
Anders Kaseorg 61e149dbb5 subdomain: Remove impossible None case from user_matches_subdomain.
The only two callers pass get_subdomain(request) which can’t be None.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-14 09:35:36 -07:00
Lauryn Menard 09d94c62a5 integrations: Revise integration CATEGORY dict structure.
Adds a META_CATEGORY dict for categories that are not best described
as groups of 'tools', so that in the subsequent commit the PAGE_TITLE
can be set accordingly.

Also, removes 'tools' from the 'Miscellaneous' category text and
spells out 'Human resources' instead of using 'HR'.
2022-09-13 11:36:52 -07:00
Sahil Batra 1e55e7641e export: Do not export direct_members and direct_subgroups field.
We do not need direct_members and direct_subgroups field of
UserGroup objects in the export data since we already have
UserGroupMembership and GroupGroupMembership object data.

While importing we keep these fields empty when creating
UserGroup objects and direct_members and direct_subgroups
fields will get set when UserGroupMembership and
GroupGroupMembership objects are created.

This change will also help us in further changes when we
will change the order of importing to import UserGroup
objects just after Realm objects.
2022-09-13 11:07:09 -07:00
Yogesh Sirsat c355934179 custom_profile_fields: Change data of default external account type.
change the names of "github" and "twitter" external account fields to
"GitHub username" and "Twitter username" respectively and remove the
hints of them.
2022-09-12 14:29:11 -07:00
David Rosa 217915a50c integrations: Remove "Yo".
The Yo company shut down in 2016.
https://en.wikipedia.org/wiki/Yo_(app)#History

Removes `yo` instances from `zerver/lib/integrations.py`.
Removes `zerver/webhooks/yo`.
Removes `static/images/integrations/yo-app`.
2022-09-09 16:31:33 -07:00
Mateusz Mandera 95bc77ae4f help center: Update "SAML authentication" to follow help center guidelines.
Co-authored-by: Alya Abbott <alya@zulip.com>
2022-09-09 10:30:05 -07:00
Zixuan James Li 6b527dfa61 integrations: Add documentation for the event filtering system.
We create "event-filter-instruction.md" and add it to
"create-bot-construct-url.md". This allows the user to keep track of the
supported event types for most of the integrations that implement this
feature. Note that not all integrations use "create-bot-construct-url.md".

We also need to rename "function" to "view_function" to make this change
type-check.

This is relevant to #18392.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-09-08 13:37:35 -07:00
Lauryn Menard 7f738803f0 help-docs: Rename organization language help center file.
Renames article about organization language used for automated
messages and invitation emails. Creates URL redirect and updates
links in repository (web app, help center and api documentation).

Prior to this change, the article was named:
'change-the-default-language-for-your-organization'.

Fixes #21949.
2022-09-07 09:45:37 -07:00
Christopher Chong 28173cafc8 message_flags: Fix deadlocks when updating message flags.
Previously, an active production Zulip server would experience a class
of deadlocks caused by two or more concurrent bulk update operations
on the UserMessage table.

This is because UPDATE ... SET ... WHERE statements that execute in
parallel take row-level UPDATE locks as they get results; since the
query plans may result in getting rows in different orders between two
queries, this can result in deadlocks.

Some databases allow ORDER BY on their UPDATE ... WHERE statements;
PostgreSQL does not. In PostgreSQL, the answer is to do a sub-select
with an ORDER BY ... FOR UPDATE to ensure consistent ordering on row
locks.

We do this all code paths using bitand or bitor as part of bulk
editing message flags, which should ensure that these concurrent
operations obtain row level locks on the table in the same order.

Fixes #19054.
2022-09-06 16:06:58 -07:00
Christopher Chong f1a3cdc4c6 integrations: Add Azure DevOps webhook integration. 2022-09-01 14:01:13 -07:00
Lauryn Menard 5a7aa8228a urls: Extend documentation URL redirects system to corporate landing pages.
Extends the URL redirect system used for documentation pages to corporate
landing pages. This makes it easier and consistent for contributors who
work on both areas to create new URL redirects when needed.
2022-08-31 15:01:29 -07:00
Lauryn Menard 878b46e758 urls: Simplify process to add URL redirects for documentation pages.
Creates `zerver.lib.url_redirects.py` to record old and new URLs
for documentation pages that have been renamed/moved and need URL
redirects.

This file is then used by `zproject.urls.py` to redirect links and
by `zerver.test.test_urls.py` to test that all of the old URLs
return a success response with a common page header/text depending
on the type of redirect (help center, policy, or API).

Adds a section to contributor docs on writing documentation for
how to use this redirect system when renaming a help center or api
documentation page.

Fixes #21946. Fixes #17897.
2022-08-31 15:01:29 -07:00
Mateusz Mandera 10a1596d96 send_analytics_to_remote_server: Log connection errors with traceback.
It seems helpful for this to get logged with the traceback rather than
just the general
"<exception name>  while trying to connect to push notification bouncer."
2022-08-31 14:43:15 -07:00
Khushiyant 8609c441ff images: Move Giphy images into proper subdirectories.
The logo were only used in the integration documentation and belong in
static/images/integrations/giphy/; the in-app image is given its own
directory.

Fixes #22464.
2022-08-30 22:40:59 -07:00
Aman Agrawal 09572fe09b templates: Provide proper error message if entrypoint is not defined. 2022-08-30 16:02:06 -07:00
Mateusz Mandera d5b450c1e0 import: Fix the import of authentication_methods BitField.
The ordering of bits that fix_realm_authentication_bitfield was making
was incorrect, it needs to be reversed.
2022-08-29 11:10:18 -07:00
Zixuan James Li bb9e80d7a2 i18n: Deal with lazy strings more carefully.
This uses a more specific type `_StrPromise` to replace `Promise`
providing typing information for lazy translation strings.

In places where the callee evaluates the `_StrPromise` object in all
cases we simply force the evaluation with `str()`. This includes
`JsonableError` that ends up handled by the error handler middleware,
and `internal_send_stream_message` that depends on `check_stream_topic`,
requiring the `topic` to be evaluated anyway. In other siuations, the
callee is expected to be able to handle `StrPromise` explicitly.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-29 10:52:45 -07:00
Zixuan James Li ab9279aabe models: Fix type annotation with StrPromise.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-29 10:52:45 -07:00
Mateusz Mandera ac9e927989 import_realm: Validate emails of users being imported.
Having users with malformed email addresses in our database can lead to
hard to debug issues and we really should try to prevent that.
2022-08-29 10:39:14 -07:00
evykassirer efee77b41f emoji: Add which emoji are supported to the /register call.
Fixes #21037.

This is part of fixing #19371. To bulk-add new emoji regularly,
mobile needs to know which servers support which emoji.
`staticfiles_storage.url` generates a unique URL with a hash
based on the file content, which lets mobile know if it needs
to update its locally stored data.
2022-08-26 17:58:31 -07:00
sahil839 decc8d41fa emails: Add quote prefix ">" in plain text missed message emails.
We add quote prefix ">" to each line of the message in the plain text
missed message emails, which are then rendered as quotes by email
clients. We also move the message content in the next line after sender.

This helps us in clearly showing the message authors in missed message
emails especially in emails with multiple messages and senders.

Fixes #15836.
2022-08-26 11:48:29 -07:00
Alya Abbott ff82c69480 help center: Rewrite Import from Slack documentation. 2022-08-25 15:17:43 -07:00
Tim Abbott c98f9bcb8e email_notifications: Use assert to verify only one recipient.
This ValueError had no test coverage, because the code path wasn't
actually possible with how the caller is constructed.

Rather than writing a highly artificial test for this as proposed in
2022-08-25 12:26:02 -07:00
Zixuan James Li c8d5959689 settings: Make AVATAR_SALT mandatory.
This also allows us to remove some assertions as we now know that
AVATAR_SALT will never be None.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-25 12:13:03 -07:00
Anders Kaseorg 4a61e36def CVE-2022-36048: Rewrite only specific local links to relative.
Due to mismatches between the URL parsers in Python and browsers, it
was possible to hoodwink rewrite_local_links_to_relative into
generating links that browsers would interpret as absolute.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-08-24 16:29:09 -07:00
Mateusz Mandera c1c9024af5 email_mirror: Replace disallowed characters in incoming email subject.
These characters are not allowed and trying to create a Zulip message
with those characters throws a JsonableError in check_stream_topic.

We don't want to reject emails with those chars in the subject, so
it's best to just modify it appropriately.
2022-08-22 17:16:20 -07:00
Zixuan James Li 32a77fc0bf test_helpers: Move patch_queue_publish to test_service_bot_system.
This prepares us to tighten the type annotations for this helper.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-22 15:46:16 -07:00
Zixuan James Li 21fd62427d typing: Remove ViewFuncT.
This removes ViewFuncT and all the associated type casts with ParamSpec
and Concatenate. This provides more accurate type annotation for
decorators at the cost of making the concatenated parameters
positional-only. This change does not intend to introduce any other
behavioral difference. Note that we retype args in process_view as
List[object] because the view functions can not only be called with
arguments of type str.

Note that the first argument of rest_dispatch needs to be made
positional-only because of the presence of **kwargs.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-22 15:46:16 -07:00
Israel Ebonko 1f286ab283 integration: Add wekan integration.
Fixes #13086

Co-authored-by: Ganesh Pawar (ganpa3) <pawarg256@gmail.com>
2022-08-22 15:37:22 -07:00
Zixuan James Li 965ed728e1 type_debug: Remove type_debug.
This module was originally introduced in 2016 to assist adding mypy
annotations to the project. Back then static type checking was not that
established throughout the codebase, so it was helpful to be able to
print out the types for type checking purposes.

This workflow is no longer helpful for improving type annotations right
now, and it has been unused for a while.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-22 13:19:09 -07:00