Commit Graph

823 Commits

Author SHA1 Message Date
Max Zawisa ec00557962 docs: Updated New Relic documentation.
The docs are updated to work with the new webhook and new process on
https://one.newrelic.com.
2020-12-15 12:04:46 -08:00
Puneeth Chaganti 5dc3489166 webhooks/sentry: Fix URL generated in transform_webhook_payload.
The URL incorrectly had `event` in the URL path, instead of `events`.

Closes #16783
2020-12-02 12:28:45 -08:00
Puneeth Chaganti b7a08323aa webhooks/sentry: Use received key when timestamp key is absent. 2020-12-02 12:28:45 -08:00
Vishnu KS dabbc3445a webhooks: Properly format the currency amount for refunds.
By default all Stripe API amounts are in the currency's smallest unit.
It's upto us to convert it to a bigger unit and show it to the end user.
And refund event used to show the currency in the smallest unit which makes
the output wrong when it comes to most currencies like USD, Europ, INR etc
which uses a bigger unit(eg Dollar instead of Cents) as the standard.
2020-11-29 18:11:24 -08:00
Max Zawisa f05a04e000
webhooks: Update NewRelic webhook for new format.
Update the New Relic webhook and tests to match the format specified
in the New Relic documentation. The new format sends a json body
instead of using url parameters. The old format is no longer supported
by New Relic according to their support staff; as a result, the fixtures for 
the old test cases were removed. Added fixtures for new test cases.

Fixes: #16393.
2020-11-18 16:19:08 -08:00
Vishnu KS 5eb63ddb7a webhooks: Handle dispute events with object IDs prefixed with du.
Sometimes the dispute object IDs are prefixed with `du` instead of `dp`.

https://freenode.logbot.info/stripe/20200605#c4059469

The correct long-term fix here would be to stop using object IDs to
detect the object type of these events and instead maybe make use of
"object" key instead.

https://stripe.com/docs/api/disputes/object#dispute_object-object
2020-11-16 17:05:54 -08:00
Puneeth Chaganti 358f1f9ba7 webhooks/sentry: Support integration configured as webhook.
Sentry allows adding simple webhooks without going through the process
of creating an Internal Integration in Sentry's Integration
Platform[1] (which our docs recommend).

The payload from sent from such a (simple) webhook integration is
slightly different from the payload sent by an Internal Integration
webhook. This commit tries to wrangle this payload into a form that is
usable by our webhook handler to send a notification message.

[1]: https://sentry.io/integration-platform/
2020-11-09 12:02:49 -08:00
Anders Kaseorg 72d6ff3c3b docs: Fix more capitalization issues.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-23 11:46:55 -07:00
Steve Howell f29b2884ca bitbucket2: Format user info consistently.
We now use get_user_info() to format all of our
users in messages.
2020-09-23 15:31:38 -07:00
Steve Howell e0b6619dac bitbucket2: Simplify how we display user for fork events.
Even before GDPR changes, it was strange that we displayed
users differently for fork events vs. all other events.

After GDPR, we don't even get the `username` field any
more.

So now we simply use `display_name` if available, and then
we try `nickname`.

See https://developer.atlassian.com/cloud/bitbucket/bitbucket-api-changes-gdpr/
for more context.
2020-09-23 15:31:38 -07:00
Steve Howell 1ef8d79352 bitbucket: Decouple BITBUCKET_FORK_BODY between versions.
We were trying to share the same format string between
the two different versions of bitbucket, but this only
creates confusion, as the two versions are only close
enough to be confusing.

The format string might be the same, but the semantics
are different, as well as the eventual outputs.

For example, the {username} piece here is simple in version
2, but in version 3 we append a url to the user's name.
2020-09-23 15:31:38 -07:00
Hemanth V. Alluri f57196b1d0 webhooks/gitlab: Add fixture for MR opened with multiple assignees.
Fixes #16173.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-09-13 20:54:14 -07:00
Hemanth V. Alluri 54aba8d402 webhooks/gitlab: Use information about all assignees.
Previously, the GitLab webhook code, namely the `get_objects_assignee`
method first tried to get a single assignee and if that failed then it
looks for multiple assignees and then it would return the first
assignee that it found (there's actually a code smell here - a loop
which would always return on the first iteration).

Instead, this commit will change that behavior to first check for
multiple assignees first then for a single assignee if we can't find
multiple assignees. Ultimately it will return a list of all of the
assignees (however many that might be [0, n]). This method has then
aptly been renamed to `get_assignees`.

Finally, we tweked the code using this method to always use it's
output as an "assignees" parameter to templates (there's also an
assignee parameter which we want to avoid here for consistency).

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-09-13 20:54:14 -07:00
Hemanth V. Alluri bbe7a54171 webhooks/gitlab: Make permissions consistent across all fixtures.
For some reasons, some of the fixtures had the +x bit set, while
some didn't. What this commit does is make sure that no fixture
is marked as "executable" (for anyone).

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-09-13 20:54:14 -07:00
Anders Kaseorg f39d4cf1f0 sentry: Pass format arguments to logging.
semgrep 0.23 correctly flags this as a violation of our logging-format
rule.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-13 15:37:28 -07:00
Alex Vandiver 3f6e4ff303 webhooks: Move the extra logging information into a formatter.
This clears it out of the data sent to Sentry, where it is duplicative
with the indexed metadata -- and potentially exposes PHI if Sentry's
"make this issue public" feature is used.
2020-09-11 16:43:29 -07:00
Alex Vandiver 4917391133 webhooks: Derive payload from request itself. 2020-09-10 17:47:22 -07:00
Alex Vandiver a1f5f6502c webhooks: In logger, pull user from request, rather than parameter.
request.user is set by validate_api_key, which is called by
webhook_view and authenticated_rest_api_view.
2020-09-10 17:47:22 -07:00
Alex Vandiver d04db7c5fe webhooks: Remove repetitive argument to UnsupportedWebhookEventType.
The name of the webhook can be added by the webhook decorator.
2020-09-10 17:47:21 -07:00
Alex Vandiver cf6ebb9c8d webhooks: Rename api_key_only_webhook_view to webhook_view.
There are no other types of webhook views; this is more concise.
2020-09-10 17:47:21 -07:00
Alex Vandiver 9ea9752e0e webhooks: Rename UnexpectedWebhookEventType to UnsupportedWebhookEventType.
Any exception is an "unexpected event", which means talking about
having an "unexpected event logger" or "unexpected event exception" is
confusing.  As the error message in `exceptions.py` already explains,
this is about an _unsupported_ event type.

This also switches the path that these exceptions are written to,
accordingly.
2020-09-10 17:47:21 -07:00
Alex Vandiver 8016769613 webhooks: Move UnexpectedWebhookEventType into zerver.lib.exceptions.
8e10ab282a moved UnexpectedWebhookEventType into
`zerver.lib.exceptions`, but left the import into
`zserver.lib.webhooks.common` so that webhooks could continue to
import the exception from there.

This clutters things and adds complexity; there is no compelling
reason that the exception's source of truth should not move alongside
all other exceptions.
2020-09-10 17:47:21 -07:00
Anders Kaseorg f91d287447 python: Pre-fix a few spots for better Black formatting.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 17:51:09 -07:00
Anders Kaseorg bef46dab3c python: Prefer kwargs form of dict.update.
For less inflation by Black.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 17:51:09 -07:00
Steve Howell dad0073331 trello webhook: Ignore copyCard actions.
See https://github.com/zulip/zulip/issues/16185,
which encourages folks to fill in for these
missing actions.
2020-09-03 10:44:39 -07:00
Steve Howell 5bff66b450 github webhook: Always send messages for team edits.
If there are unsupported keys, we still log an error,
but we now also send a message to the stream.  (This
is a good tradeoff for the github webhook, since users
can just turn off notifications if they find it spammy.
Also, we intend to support "repository" soon.)

This is a bit of an experiment to see how this plays
in the field:

    * will customers notice the change?
    * will Sentry reports look any different?
2020-09-03 10:44:39 -07:00
Steve Howell c6b9a23c17 github webhooks: Fix message for unsupported team payloads.
The main thing fixed here is that we weren't turning
on our keys into a list.  And then I refined the message
a bit more, including sorting the keys.

I also avoid the unnecessary "else".
2020-09-03 10:44:39 -07:00
Steve Howell 8785790a27 github tests: Add test_team_edited_error_handling.
This doesn't test much interesting yet, but it
will soon.
2020-09-03 10:44:39 -07:00
Steve Howell cc2dbefc60 mypy: Use better types for EVENT_FUNCTION_MAPPER.
The EVENT_FUNCTION_MAPPER maps a string event name
to a function handler.  Before this we circumvented
mypy checks with a call to get_body_function_based_on_type,
which specified Any as the type of our event function.

Now the types are rigorous.

This change was impossible without the recent commit
to introduce the Helper class.
2020-09-03 10:44:39 -07:00
Steve Howell 4de2b78c25 github refactor: Add Helper class.
The Helper class will soon grow, but the immediate
problem it solves is the need to jankily inspect
the parameters of our get_*_body function.

Most of the changes were handled by an ad hoc
munge.py script.

The substantive changes were adding the Helper
class and passing it in.

And then the linter discovered a place where
the optional include_title parameter wasn't used
(which is one of the reasons to avoid the janky
inspect-signature technique).

As a side note, none of the include_title parameters
needed a default value of False, as we always passed
in an explicit value.

We test cover both sides of include_title, which
you can verify by hard coding it to either True or
False (and seeing the relevant failures), although I
suspect most individual codepaths
only test one value, based on whether "topic" is in
the fixture or not.

Finally, I know Helper is not a great name, but I
intend to evolve the class a bit before deciding
whether a more descriptive name is helpful here.
(For example, an upcoming commit will add a
log_unexpected helper method.)
2020-09-03 10:44:39 -07:00
Steve Howell ead7cbea40 github refactor: Handle header_event explicitly.
We get the header_event one level up the call
stack now, too.

It's somewhat annoying that we have our own
concept of "event" here, instead of just returning
our event handlers directly, or just calling them
directly, but it's a bit non-trivial to fix that
right away.

In passing, I remove the strange OR for "ping",
which is already a key in EVENT_FUNCTION_MAPPER.
2020-09-03 10:44:39 -07:00
Steve Howell 0d9b1817f9 github refactor: Use early-exit idiom.
We also comment a bit more explicitly about the
None case.
2020-09-03 10:44:39 -07:00
Steve Howell 5c916135c9 github webhooks: Avoid string interpolation.
We know the event explicitly here.
2020-09-03 10:44:39 -07:00
Steve Howell 425db931a8 github webhook: Explicitly ignore team actions. 2020-09-03 10:44:39 -07:00
Steve Howell 294fd59983 github webhook: Ignore more pull_request actions.
See https://github.com/zulip/zulip/issues/16258 for
possible follow up here.

We now ignore the following two new pull_request
actions (as well as the three existing ones
from before):

    approved
    converted_to_draft

As the issue above indicates, we may want to actually
support "approved" if we can find somebody to work
on the webhook.  (And then the issue goes a little
broader than what changed here.)
2020-09-03 10:44:39 -07:00
Steve Howell 5dea85a186 github tests: Extract test_ignored_pull_request_actions.
We consolidate the tests and remove the fixtures, which
just have a lot of noisy fields that we ignore.  Also,
pull_request__request_review_removed was named improperly.
2020-09-03 10:44:39 -07:00
Steve Howell 4c2787c35f github tests: Add test_ignored_events. 2020-09-03 10:44:39 -07:00
Steve Howell 040bf82122 github webhook: Remove unused exception class. 2020-09-03 10:44:39 -07:00
Anders Kaseorg a276eefcfe python: Rewrite dict() as {}.
Suggested by the flake8-comprehensions plugin.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:15:41 -07:00
Anders Kaseorg ab120a03bc python: Replace unnecessary intermediate lists with generators.
Mostly suggested by the flake8-comprehension plugin.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:15:41 -07:00
Anders Kaseorg 72d2e5df15 isort: Enable black profile.
Our isort configuration was almost Black-compatible, but we were
missing ensure_newline_before_comments.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:00:07 -07:00
Alex Vandiver cec594e525 trello: Remove unnecessary "is not supported" verbiage.
UnexpectedWebhookEventType already encapsulates that it is not
supported, and having extra wording here disrupts how the exception
stringifies.
2020-08-31 14:24:35 -07:00
Alex Vandiver a350a9b3d8 trello: Simplify logic for ignored card actions.
Rather than catching, checking action type, and possibly re-raising,
instead return None explicitly from `get_subject_and_body`, which
already signals for a blank success result.  This collocates the logic
of the action types in one place, and removes the complexity of the
re-raise.
2020-08-31 14:24:35 -07:00
Alex Vandiver 398102e215 sentry: When there are multiple stacktraces, take the most recent.
Sentry may get reported multiple exceptions stacks, in the case where
a `raise ...` was caught, and a new exception was `raise`d.  In this
case, the `filename` is the most recent exception -- but the
exceptions are stored in the `exception` key in the order in which
they occurred.  As such, taking the first value with a `stacktrace`
will result in showing the wrong line, or in no stack trace being
resolved at all.

Look from the last `exception` backwards, for matching stacks.
2020-08-31 14:14:20 -07:00
Steve Howell f429df3401 github tests: Extract verify_post_is_ignored. 2020-08-24 12:34:46 -07:00
Steve Howell dfe6960584 minor: Format IGNORED_EVENTS for github. 2020-08-24 12:34:46 -07:00
Steve Howell 0863874050 jira: Ignore worklog_updated event. 2020-08-24 12:34:46 -07:00
Steve Howell 7b344044b4 jira: Ignore version_released event. 2020-08-24 12:34:46 -07:00
Steve Howell 52c17fc731 minor: Format IGNORED_EVENTS for jira. 2020-08-24 12:34:46 -07:00
Steve Howell 56bfae3fac jira tests: Test ignored events. 2020-08-24 12:34:46 -07:00