Commit Graph

78 Commits

Author SHA1 Message Date
Alex Vandiver 3f6e4ff303 webhooks: Move the extra logging information into a formatter.
This clears it out of the data sent to Sentry, where it is duplicative
with the indexed metadata -- and potentially exposes PHI if Sentry's
"make this issue public" feature is used.
2020-09-11 16:43:29 -07:00
Alex Vandiver 4917391133 webhooks: Derive payload from request itself. 2020-09-10 17:47:22 -07:00
Alex Vandiver a1f5f6502c webhooks: In logger, pull user from request, rather than parameter.
request.user is set by validate_api_key, which is called by
webhook_view and authenticated_rest_api_view.
2020-09-10 17:47:22 -07:00
Alex Vandiver d04db7c5fe webhooks: Remove repetitive argument to UnsupportedWebhookEventType.
The name of the webhook can be added by the webhook decorator.
2020-09-10 17:47:21 -07:00
Alex Vandiver cf6ebb9c8d webhooks: Rename api_key_only_webhook_view to webhook_view.
There are no other types of webhook views; this is more concise.
2020-09-10 17:47:21 -07:00
Alex Vandiver 9ea9752e0e webhooks: Rename UnexpectedWebhookEventType to UnsupportedWebhookEventType.
Any exception is an "unexpected event", which means talking about
having an "unexpected event logger" or "unexpected event exception" is
confusing.  As the error message in `exceptions.py` already explains,
this is about an _unsupported_ event type.

This also switches the path that these exceptions are written to,
accordingly.
2020-09-10 17:47:21 -07:00
Alex Vandiver 8016769613 webhooks: Move UnexpectedWebhookEventType into zerver.lib.exceptions.
8e10ab282a moved UnexpectedWebhookEventType into
`zerver.lib.exceptions`, but left the import into
`zserver.lib.webhooks.common` so that webhooks could continue to
import the exception from there.

This clutters things and adds complexity; there is no compelling
reason that the exception's source of truth should not move alongside
all other exceptions.
2020-09-10 17:47:21 -07:00
Steve Howell 5bff66b450 github webhook: Always send messages for team edits.
If there are unsupported keys, we still log an error,
but we now also send a message to the stream.  (This
is a good tradeoff for the github webhook, since users
can just turn off notifications if they find it spammy.
Also, we intend to support "repository" soon.)

This is a bit of an experiment to see how this plays
in the field:

    * will customers notice the change?
    * will Sentry reports look any different?
2020-09-03 10:44:39 -07:00
Steve Howell c6b9a23c17 github webhooks: Fix message for unsupported team payloads.
The main thing fixed here is that we weren't turning
on our keys into a list.  And then I refined the message
a bit more, including sorting the keys.

I also avoid the unnecessary "else".
2020-09-03 10:44:39 -07:00
Steve Howell 8785790a27 github tests: Add test_team_edited_error_handling.
This doesn't test much interesting yet, but it
will soon.
2020-09-03 10:44:39 -07:00
Steve Howell cc2dbefc60 mypy: Use better types for EVENT_FUNCTION_MAPPER.
The EVENT_FUNCTION_MAPPER maps a string event name
to a function handler.  Before this we circumvented
mypy checks with a call to get_body_function_based_on_type,
which specified Any as the type of our event function.

Now the types are rigorous.

This change was impossible without the recent commit
to introduce the Helper class.
2020-09-03 10:44:39 -07:00
Steve Howell 4de2b78c25 github refactor: Add Helper class.
The Helper class will soon grow, but the immediate
problem it solves is the need to jankily inspect
the parameters of our get_*_body function.

Most of the changes were handled by an ad hoc
munge.py script.

The substantive changes were adding the Helper
class and passing it in.

And then the linter discovered a place where
the optional include_title parameter wasn't used
(which is one of the reasons to avoid the janky
inspect-signature technique).

As a side note, none of the include_title parameters
needed a default value of False, as we always passed
in an explicit value.

We test cover both sides of include_title, which
you can verify by hard coding it to either True or
False (and seeing the relevant failures), although I
suspect most individual codepaths
only test one value, based on whether "topic" is in
the fixture or not.

Finally, I know Helper is not a great name, but I
intend to evolve the class a bit before deciding
whether a more descriptive name is helpful here.
(For example, an upcoming commit will add a
log_unexpected helper method.)
2020-09-03 10:44:39 -07:00
Steve Howell ead7cbea40 github refactor: Handle header_event explicitly.
We get the header_event one level up the call
stack now, too.

It's somewhat annoying that we have our own
concept of "event" here, instead of just returning
our event handlers directly, or just calling them
directly, but it's a bit non-trivial to fix that
right away.

In passing, I remove the strange OR for "ping",
which is already a key in EVENT_FUNCTION_MAPPER.
2020-09-03 10:44:39 -07:00
Steve Howell 0d9b1817f9 github refactor: Use early-exit idiom.
We also comment a bit more explicitly about the
None case.
2020-09-03 10:44:39 -07:00
Steve Howell 5c916135c9 github webhooks: Avoid string interpolation.
We know the event explicitly here.
2020-09-03 10:44:39 -07:00
Steve Howell 425db931a8 github webhook: Explicitly ignore team actions. 2020-09-03 10:44:39 -07:00
Steve Howell 294fd59983 github webhook: Ignore more pull_request actions.
See https://github.com/zulip/zulip/issues/16258 for
possible follow up here.

We now ignore the following two new pull_request
actions (as well as the three existing ones
from before):

    approved
    converted_to_draft

As the issue above indicates, we may want to actually
support "approved" if we can find somebody to work
on the webhook.  (And then the issue goes a little
broader than what changed here.)
2020-09-03 10:44:39 -07:00
Steve Howell 040bf82122 github webhook: Remove unused exception class. 2020-09-03 10:44:39 -07:00
Steve Howell dfe6960584 minor: Format IGNORED_EVENTS for github. 2020-08-24 12:34:46 -07:00
Till Schneidereit 6d6d43188d webhooks/github: Include full repository name in notification messages.
This changes the notification messages for events that currently just
include the string `"the repository"` to also include the full (`org/repo`)
name of the affected repository. Messages for the following events are
changed:
- `public`
- `star`
- `watch`
- `repository`
- `team_add`

Background: we're using the GitHub integration for org-wide notifications
for the [Bytecode Alliance Zulip](bytecodealliance.zulipchat.com/), and
having all messages just say "the repository" isn't ideal. Even now one
can hover over the link to see the repo's url, but it'd be much nicer if
the message just contained the full name.

I also changed the message for `star` to include a link to the repository,
same as the `watch` notification.
2020-06-29 13:12:16 -07:00
Anders Kaseorg 69c0959f34 python: Fix misuse of Optional types for optional parameters.
There seems to have been a confusion between two different uses of the
word “optional”:

• An optional parameter may be omitted and replaced with a default
  value.
• An Optional type has None as a possible value.

Sometimes an optional parameter has a default value of None, or None
is otherwise a meaningful value to provide, in which case it makes
sense for the optional parameter to have an Optional type.  But in
other cases, optional parameters should not have Optional type.  Fix
them.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:31:27 -07:00
Anders Kaseorg 365fe0b3d5 python: Sort imports with isort.
Fixes #2665.

Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.

Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start.  I expect this change will increase pressure for us to split
those files, which isn't a bad thing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:45:32 -07:00
Anders Kaseorg 69730a78cc python: Use trailing commas consistently.
Automatically generated by the following script, based on the output
of lint with flake8-comma:

import re
import sys

last_filename = None
last_row = None
lines = []

for msg in sys.stdin:
    m = re.match(
        r"\x1b\[35mflake8    \|\x1b\[0m \x1b\[1;31m(.+):(\d+):(\d+): (\w+)", msg
    )
    if m:
        filename, row_str, col_str, err = m.groups()
        row, col = int(row_str), int(col_str)

        if filename == last_filename:
            assert last_row != row
        else:
            if last_filename is not None:
                with open(last_filename, "w") as f:
                    f.writelines(lines)

            with open(filename) as f:
                lines = f.readlines()
            last_filename = filename
        last_row = row

        line = lines[row - 1]
        if err in ["C812", "C815"]:
            lines[row - 1] = line[: col - 1] + "," + line[col - 1 :]
        elif err in ["C819"]:
            assert line[col - 2] == ","
            lines[row - 1] = line[: col - 2] + line[col - 1 :].lstrip(" ")

if last_filename is not None:
    with open(last_filename, "w") as f:
        f.writelines(lines)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 16:04:12 -07:00
Anders Kaseorg 6480deaf27 python: Convert more "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, with more
restrictions patched out.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 14:48:09 -07:00
Anders Kaseorg 8dd83228e7 python: Convert "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, but with the
NamedTuple changes reverted (see commit
ba7906a3c6, #15132).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 15:31:20 -07:00
Hemanth V. Alluri ab03659095 webhooks/github: Provide more detailed info for unexpected events.
Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-05-21 12:31:43 -07:00
Hemanth V. Alluri c8f4177872 webhooks/github: Add a support for team/edited events.
Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-05-16 14:59:08 -07:00
cestrell 297185cc12 integrations: Use new function in Github releases.
Use get_release_event_message from webhooks/git.py to format release
events using the newly implemented release message template.

Tweaked by tabbott to handle name=None.

Builds on #14746. Proposed in #14934.
2020-05-12 17:42:49 -07:00
Hemanth V. Alluri cd69da26dc webhooks/github: Ignore the label event. 2020-05-12 09:32:17 -07:00
Hemanth V. Alluri 4dd957faa2 webhooks/github: Add support for pull request ready for review.
GitHub supports opening a draft/WIP pull request and then marking it
as ready for review later on. This PR supports the ready_for_review
action for pull_request events.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-05-12 09:32:17 -07:00
Anders Kaseorg c734bbd95d python: Modernize legacy Python 2 syntax with pyupgrade.
Generated by `pyupgrade --py3-plus --keep-percent-format` on all our
Python code except `zthumbor` and `zulip-ec2-configure-interfaces`,
followed by manual indentation fixes.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-09 16:43:22 -07:00
Stefan Weil d2fa058cc1
text: Fix some typos (most of them found and fixed by codespell).
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2020-03-27 17:25:56 -07:00
Sourabh Singh 1b3cfecf2a
webhooks: Add team reviewers support in github webhook.
The github webhook implementation previously ignored the "team reviewers"
part of pull_request events, resulting in inaccurate output.

Fixes: #14096.
2020-03-10 16:29:59 -07:00
Tim Abbott c10cc24ee8 python: Sort webhooks imports with isort. 2020-01-14 13:07:47 -08:00
Anders Kaseorg cafac83676 request: Tighten type checking on REQ.
Then, find and fix a predictable number of previous misuses.

With a small change by tabbott to preserve backwards compatibility for
sending `yes` for the `forged` field.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 12:35:55 -08:00
Brutus5000 a386e3aebb github webhook: For release events show actions and tag name. 2019-08-17 12:35:16 -07:00
Eeshan Garg 1579dc2fee webhooks/github: Add support for star events. 2019-07-22 18:20:53 -07:00
Eeshan Garg badaf28f21 webhooks/github: Ignore 'meta' events.
This event occurs when its corresponding webhook is deleted. This
isn't very useful information, so we should ignore it.
2019-07-22 18:20:53 -07:00
Hemanth V. Alluri ef52aa0fc1 webhooks: Eliminate the usage of a headers.py file.
For storing HTTP headers as a function of fixture name, previously
we required that the fixture_to_headers method should reside in a
separate module called headers.py.

However, as in many cases, this method will only take a few lines,
we decided to move this function into the view.py file of the
integration instead of requiring a whole new file called headers.py

This commit introduces the small change in the system architecture,
migrates the GitHub integration, and updates the docs accordingly.
2019-06-26 10:35:14 -07:00
Eeshan Garg ef070ef019 webhooks/github: Improve punctuation. 2019-05-09 13:05:49 -07:00
Eeshan Garg ab8aae6d0c git_webhooks: Use proper punctuation for PR/issue messages. 2019-05-07 16:45:01 -07:00
Eeshan Garg 7b6a37780a webhooks/github: Ignore organization and milestone events.
These events are not super useful and were cluttering up our
webhook logs.
2019-03-10 14:13:17 -07:00
Eeshan Garg 6afd02bef5 webhooks/github: Restrict membership event scope to teams.
According to GitHub's webhook docs, the scope of a membership
event can only be limited to 'teams', which holds true when a
new member is added to a team. However, we just found a payload
in our logs that indicates that when a user is removed from a
team, the scope of the membership is erroneously set to
'organization', not 'team'. This is most likely a bug on
GitHub's end because such behaviour is a direct violation of
their webhook API event specifications. We account for this
by restricting membership events to teams explicitly, at least
till GitHub's docs suggest otherwise.
2019-03-10 14:13:17 -07:00
Eeshan Garg fa29006311 webhooks/github: Ignore check_suite events.
A check suite is a collection of check runs. We care a lot more
about the outcomes of check runs in this case because check_run
payloads are a lot more informative than check_suite payloads.

(And in any case, the check_suite events are primarily for notifying
tools like CI to run checks).
2019-02-20 16:32:42 -08:00
Eeshan Garg a0717e4424 webhook/github: Support check_run events.
We only support notifications for events where a check run has
completed. Notifications for when a check run has been queued or
is in progress are not very informative and may be too noisy.
2019-02-20 16:32:39 -08:00
Eeshan Garg c78c3f423c webhooks/github: Ignore project_card events.
The payloads for this event are missing some important details
about the Project's changes, such as the name of the project,
the card's column name, etc. Without such details, the resultant
notifications would not be useful at all!
2019-02-19 14:25:38 -08:00
Eeshan Garg ec81410b03 webhooks/github: Ignore repository_vulnerability_alert event.
This event isn't incredibly common/useful and errors for this
event were cluttering up our webhook logs.
2019-02-19 18:01:05 -03:30
Anders Kaseorg 39ac378220 webhooks: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:05:20 -08:00
Eeshan Garg ab7d0de782 webhooks/github: Enable PM notifications for non-JSON payloads. 2018-11-14 22:35:57 -08:00
Steve Howell ea98a44db3 webhooks: Replace SUBJECT_WITH_* with TOPIC_WITH_*. 2018-11-12 15:47:11 -08:00