Commit Graph

15742 Commits

Author SHA1 Message Date
Hari Prashant Bhimaraju e9e1c1d6b1 zapier: Strengthen types using WildValue. 2022-09-01 14:02:01 -07:00
Hari Prashant Bhimaraju 76b49459f8 zabbix: Strengthen types using WildValue.
This commit strengthens types by typing the webhook's incoming
payload as WildValue, which eradicates the use of Any within the
Zabbix incoming webhook integration.

The KeyError exception has been replaced to catch a ValidationError
instead now, since the incoming payload's keys will be tamed before
usage and the non-existence of the key is raised as a
ValidationError in the taming function.
2022-09-01 14:02:01 -07:00
Christopher Chong f1a3cdc4c6 integrations: Add Azure DevOps webhook integration. 2022-09-01 14:01:13 -07:00
Alya Abbott 87a295677e portico: Fix title and description for /attribution. 2022-09-01 11:33:35 -07:00
Lauryn Menard 5a7aa8228a urls: Extend documentation URL redirects system to corporate landing pages.
Extends the URL redirect system used for documentation pages to corporate
landing pages. This makes it easier and consistent for contributors who
work on both areas to create new URL redirects when needed.
2022-08-31 15:01:29 -07:00
Lauryn Menard 878b46e758 urls: Simplify process to add URL redirects for documentation pages.
Creates `zerver.lib.url_redirects.py` to record old and new URLs
for documentation pages that have been renamed/moved and need URL
redirects.

This file is then used by `zproject.urls.py` to redirect links and
by `zerver.test.test_urls.py` to test that all of the old URLs
return a success response with a common page header/text depending
on the type of redirect (help center, policy, or API).

Adds a section to contributor docs on writing documentation for
how to use this redirect system when renaming a help center or api
documentation page.

Fixes #21946. Fixes #17897.
2022-08-31 15:01:29 -07:00
Mateusz Mandera d21a1fe47f middleware: Log 5xx json_errors in JsonErrorHandler.
django.request logs responses with 5xx response codes (our configuration
of the logger prevents it from logging 4xx as well which it normally
does too). However, it does it without the traceback which results in
quite unhelpful log message that look like
"Bad Gateway:/api/v1/users/me/apns_device_token" - particularly
confusing when sent via email to server admins.

The solution here is to do the logging ourselves, using Django's
log_response() (which is meant for this purpose), and including the
traceback. Django tracks (via response._has_been_logged attribute) that
the response has already been logged, and knows to not duplicate that
action. See log_response() in django's codebase for these details.

Fixes #19596.
2022-08-31 14:43:15 -07:00
Mateusz Mandera 10a1596d96 send_analytics_to_remote_server: Log connection errors with traceback.
It seems helpful for this to get logged with the traceback rather than
just the general
"<exception name>  while trying to connect to push notification bouncer."
2022-08-31 14:43:15 -07:00
Khushiyant 8609c441ff images: Move Giphy images into proper subdirectories.
The logo were only used in the integration documentation and belong in
static/images/integrations/giphy/; the in-app image is given its own
directory.

Fixes #22464.
2022-08-30 22:40:59 -07:00
Anders Kaseorg 019c9527c1 test_message_flags: Fix “accessibility” spelling.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-08-30 17:36:21 -07:00
Aman Agrawal 379c6acbea portico: Add page to list open organizations.
Fixes #22020
2022-08-30 16:02:06 -07:00
Aman Agrawal 09572fe09b templates: Provide proper error message if entrypoint is not defined. 2022-08-30 16:02:06 -07:00
Mateusz Mandera 5bcf78e0cb import: Fix timestamp check in long_term_idle_helper.
This is supposed to be 60 days, but timestamps are in seconds.
2022-08-29 15:18:00 -07:00
Mateusz Mandera d350406991 gitter: Make imported Realm start with only GitHub auth enabled.
Users will only be able to login via GitHub, because imported users
get GitHub's generated noreply email addresses - so this should be the
only auth method enabled at first, to avoid confusion.
2022-08-29 11:10:18 -07:00
Mateusz Mandera d5b450c1e0 import: Fix the import of authentication_methods BitField.
The ordering of bits that fix_realm_authentication_bitfield was making
was incorrect, it needs to be reversed.
2022-08-29 11:10:18 -07:00
Zixuan James Li 3df8dd4b38 typing: Broaden parameter type for bulk_regenerate_api_keys.
In zerver.management.commands.logout_all_users,
we pass a values queryset containing the ids into
this function, which is not actually a list. This
broadens the type annotation so that the ValuesQuerySet
is accepted.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-29 11:06:41 -07:00
Mateusz Mandera eed8800573 long_term_idle_helper: Change all_user_ids arg to an Iterator. 2022-08-29 11:03:27 -07:00
Mateusz Mandera 4c7a9816ff gitter: Soft deactivate appropriate imported users.
We want to use the long_term_idle_helper logic for gitter imports just
like we do for slack.
2022-08-29 11:03:27 -07:00
Mateusz Mandera 75f26bb8ff long_term_idle_helper: Take list of user_ids as arg instead of dicts.
Only ["id"] is accessed on the dicts (representing the external tool
users). Given that for some tools the id may be under a different name
etc. due to different user dicts format, it's best to just pass those
ids to the function so that it can stay generalized and not reliant
on a specific user dict format.
2022-08-29 11:03:27 -07:00
Mateusz Mandera 7ac31223e8 gitter: Extract get_user_from_message helper. 2022-08-29 11:03:27 -07:00
Mateusz Mandera c4c270380a slack: Use get_timestamp_from_message helper function where relevant.
get_timestamp_from_message was extracted in the previous commit. We can
deduplicate and the code a bit cleaner by using it where appropriate
instead of message["ts"].
2022-08-29 11:03:27 -07:00
Mateusz Mandera 9e56e71afe long_term_idle_helper: Take timestamp_from_message callable arg.
message["ts"] is slack-specific. For this to be a general util function
it needs to take a callable that will grab a timestamp from the message
dict (which has varying formats depending on what we're importing from).
2022-08-29 11:03:27 -07:00
Mateusz Mandera a86aa13e57 gitter: Extract get_timestamp_from_message function. 2022-08-29 11:03:27 -07:00
Alex Vandiver 1b1faa3907 import_util: Factor out long_term_idle_helper. 2022-08-29 11:03:27 -07:00
Alex Vandiver 842cff5975 gitter: Some users (e.g. from matrix.org) may not have avatar URLs. 2022-08-29 11:03:27 -07:00
Lauryn Menard d5f4a93d6c bots: Refer to notification bot messages as automated notices.
Updates in-app and documentation references to automated messages
sent by the notification bot as automated notices (or automated
messages where more appropriate/clear), instead of notifications.

Also, makes some small related revisions / general clean ups to
`resolve-a-topic.md`.

Fixes #22188.
2022-08-29 10:54:08 -07:00
Zixuan James Li bb9e80d7a2 i18n: Deal with lazy strings more carefully.
This uses a more specific type `_StrPromise` to replace `Promise`
providing typing information for lazy translation strings.

In places where the callee evaluates the `_StrPromise` object in all
cases we simply force the evaluation with `str()`. This includes
`JsonableError` that ends up handled by the error handler middleware,
and `internal_send_stream_message` that depends on `check_stream_topic`,
requiring the `topic` to be evaluated anyway. In other siuations, the
callee is expected to be able to handle `StrPromise` explicitly.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-29 10:52:45 -07:00
Zixuan James Li ab9279aabe models: Fix type annotation with StrPromise.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-29 10:52:45 -07:00
Mateusz Mandera f55ec8a026 tests: Replace some testserver.com dummy domains with zulip.example.com. 2022-08-29 10:39:14 -07:00
Mateusz Mandera 3ff0b64281 test_import_export: Use get_output_dir() helper.
That's much briefer and cleaner than repeating the os.path.join
expression and the helper already exists.
2022-08-29 10:39:14 -07:00
Mateusz Mandera ac9e927989 import_realm: Validate emails of users being imported.
Having users with malformed email addresses in our database can lead to
hard to debug issues and we really should try to prevent that.
2022-08-29 10:39:14 -07:00
Mateusz Mandera 43107fcdc3 scim: Upgrade scim2-filter-parser to 0.4.0 to fix case-sensitivity.
userName lookups are supposed to be case-insensitive per the SCIM2 RFC
and this was fixed upstream in
https://github.com/15five/scim2-filter-parser/pull/31
and included in 0.4.0 release.
2022-08-29 10:35:29 -07:00
evykassirer efee77b41f emoji: Add which emoji are supported to the /register call.
Fixes #21037.

This is part of fixing #19371. To bulk-add new emoji regularly,
mobile needs to know which servers support which emoji.
`staticfiles_storage.url` generates a unique URL with a hash
based on the file content, which lets mobile know if it needs
to update its locally stored data.
2022-08-26 17:58:31 -07:00
sahil839 bd1a27491e tests: Rename missed message email tests in test_email_notifications.py.
This commit changes the name of missed message email tests for
personal and huddle messages to be more clear:

- from *_personal_missed_stream_messages to *_missed_personal_messages

- from *_huddle_missed_stream_messages to *_missed_huddle_messages
2022-08-26 11:48:29 -07:00
sahil839 decc8d41fa emails: Add quote prefix ">" in plain text missed message emails.
We add quote prefix ">" to each line of the message in the plain text
missed message emails, which are then rendered as quotes by email
clients. We also move the message content in the next line after sender.

This helps us in clearly showing the message authors in missed message
emails especially in emails with multiple messages and senders.

Fixes #15836.
2022-08-26 11:48:29 -07:00
Alya Abbott ff82c69480 help center: Rewrite Import from Slack documentation. 2022-08-25 15:17:43 -07:00
Adam Sah c6130ce4b1 email_notifications: Complete test coverage for module.
Following the last commit, there was this WELCOME_EMAIL_SENDER code
without an existing automated test.
2022-08-25 13:04:51 -07:00
Adam Sah 637867dad1 settings: Send presence update event toggling presence_enabled.
Fixes #21180.
2022-08-25 12:30:12 -07:00
Tim Abbott c98f9bcb8e email_notifications: Use assert to verify only one recipient.
This ValueError had no test coverage, because the code path wasn't
actually possible with how the caller is constructed.

Rather than writing a highly artificial test for this as proposed in
2022-08-25 12:26:02 -07:00
Zixuan James Li c8d5959689 settings: Make AVATAR_SALT mandatory.
This also allows us to remove some assertions as we now know that
AVATAR_SALT will never be None.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-25 12:13:03 -07:00
Zixuan James Li 059d0e7be8 settings: Make SHARED_SECRET mandatory.
This implements get_mandatory_secret that ensures SHARED_SECRET is
set when we hit zerver.decorator.authenticate_notify. To avoid getting
ZulipSettingsError when setting up the secrets, we set an environment
variable DISABLE_MANDATORY_SECRET_CHECK to skip the check and default
its value to an empty string.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-25 12:13:03 -07:00
Adam Sah b1293a84f8 testing: 100% coverage for zerver/webhooks/{librato,pivotal}. 2022-08-25 12:04:24 -07:00
Anders Kaseorg 4a61e36def CVE-2022-36048: Rewrite only specific local links to relative.
Due to mismatches between the URL parsers in Python and browsers, it
was possible to hoodwink rewrite_local_links_to_relative into
generating links that browsers would interpret as absolute.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-08-24 16:29:09 -07:00
Mateusz Mandera c1c9024af5 email_mirror: Replace disallowed characters in incoming email subject.
These characters are not allowed and trying to create a Zulip message
with those characters throws a JsonableError in check_stream_topic.

We don't want to reject emails with those chars in the subject, so
it's best to just modify it appropriately.
2022-08-22 17:16:20 -07:00
Hari Prashant Bhimaraju fd0b013bcd slack_incoming: Handle optional attachment fields aptly.
This commit checks for null values for keys within "attachment" in
the Slack integration's incoming payloads. These keys were expected
to exist optionally previously, and the existence of null values for
these wasn't anticipated. Due to an issue report for such null
values in the payload, their handling is updated appropriately.

The checks for these values are truthiness checks since the strategy
for these values being null or falsy ("", 0) is the same; we don't
process that key-value pair. This is consistent with how Slack handles
this scenario.

For the case where all the attachment fields have null values, Slack
displays this as an empty block with no content, and therefore our
strategy for this is a no-op.

Tests updated.
2022-08-22 16:40:13 -07:00
Aman Agrawal ef21f9107c urls: Move /hello files to the corporate folder. 2022-08-22 15:53:43 -07:00
Aman Agrawal 21a2fd482e portico: Move to corporate folder.
Since the URLs that use these function are present in corporate/urls,
this file belongs in the corporate folder.
2022-08-22 15:53:43 -07:00
Aman Agrawal 4b6df8a678 home: Only serve landing page at root domain on corporate servers. 2022-08-22 15:53:43 -07:00
Zixuan James Li db92033b23 typing: Tighten type annotations for patch_queue_publish.
Since this decorator is only used for methods of
TestServiceBotEventTriggers, we can type the decorated method's
signature accurately without using ParamSpec.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-22 15:46:16 -07:00
Zixuan James Li 32a77fc0bf test_helpers: Move patch_queue_publish to test_service_bot_system.
This prepares us to tighten the type annotations for this helper.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-22 15:46:16 -07:00