Commit Graph

66 Commits

Author SHA1 Message Date
Anders Kaseorg 0043c0b6b2 django: Use HttpRequest.headers.
Fixes #14769.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-13 20:42:20 -07:00
Anders Kaseorg a2825e5984 python: Use Python 3.8 typing.{Protocol,TypedDict}.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-27 12:57:49 -07:00
Anders Kaseorg 6528538188 bitbucket2: Strengthen types using WildValue.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-15 13:02:02 -07:00
Anders Kaseorg af67eee3ea bitbucket2: Cast append_punctuation argument to str.
Commit ab8aae6d0c (#12161) incorrectly
assumed that ‘new’ is a string.  In the case of change == "links",
it’s a dict.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-15 13:02:02 -07:00
Anders Kaseorg a34e09215d bitbucket2: Remove function signature inspection nonsense.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-15 13:02:02 -07:00
Anders Kaseorg cd2fc198a9 bitbucket2: Untangle totally different control flow for push events.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-15 13:02:02 -07:00
Lauryn Menard 3be622ffa7 backend: Add request as parameter to json_success.
Adds request as a parameter to json_success as a refactor towards
making `ignored_parameters_unsupported` functionality available
for all API endpoints.

Also, removes any data parameters that are an empty dict or
a dict with the generic success response values.
2022-02-04 15:16:56 -08:00
Eeshan Garg 2393342e03 webhooks/jira: Handle anomalous payloads properly.
We recently ran into a payload in production that didn't contain
an event type at all. A payload where we can't figure out the event
type is quite rare. Instead of letting these payloads run amok, we
should raise a more informative exception for such unusual payloads.
If we encounter too many of these, then we can choose to conduct a
deeper investigation on a case-by-case basis.

With some changes by Tim Abbott.
2021-12-28 10:56:25 -08:00
PIG208 5359d20a13 webhooks: Add support to event filtering system for webhooks.
This add support to event filtering system for most webhooks that
require trivial changes to adapt this feature.
2021-07-24 15:10:09 -07:00
Anders Kaseorg 544bbd5398 docs: Fix capitalization mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-10 09:57:26 -07:00
Anders Kaseorg 6e4c3e41dc python: Normalize quotes with Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 11741543da python: Reformat with Black, except quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 5028c081cb python: Merge concatenated string literals that Black would uglify.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
ritik 50b0496a3a bitbucket webhook: Fully deprecate username field. 2021-02-05 09:46:27 -05:00
Steve Howell f29b2884ca bitbucket2: Format user info consistently.
We now use get_user_info() to format all of our
users in messages.
2020-09-23 15:31:38 -07:00
Steve Howell e0b6619dac bitbucket2: Simplify how we display user for fork events.
Even before GDPR changes, it was strange that we displayed
users differently for fork events vs. all other events.

After GDPR, we don't even get the `username` field any
more.

So now we simply use `display_name` if available, and then
we try `nickname`.

See https://developer.atlassian.com/cloud/bitbucket/bitbucket-api-changes-gdpr/
for more context.
2020-09-23 15:31:38 -07:00
Alex Vandiver d04db7c5fe webhooks: Remove repetitive argument to UnsupportedWebhookEventType.
The name of the webhook can be added by the webhook decorator.
2020-09-10 17:47:21 -07:00
Alex Vandiver cf6ebb9c8d webhooks: Rename api_key_only_webhook_view to webhook_view.
There are no other types of webhook views; this is more concise.
2020-09-10 17:47:21 -07:00
Alex Vandiver 9ea9752e0e webhooks: Rename UnexpectedWebhookEventType to UnsupportedWebhookEventType.
Any exception is an "unexpected event", which means talking about
having an "unexpected event logger" or "unexpected event exception" is
confusing.  As the error message in `exceptions.py` already explains,
this is about an _unsupported_ event type.

This also switches the path that these exceptions are written to,
accordingly.
2020-09-10 17:47:21 -07:00
Alex Vandiver 8016769613 webhooks: Move UnexpectedWebhookEventType into zerver.lib.exceptions.
8e10ab282a moved UnexpectedWebhookEventType into
`zerver.lib.exceptions`, but left the import into
`zserver.lib.webhooks.common` so that webhooks could continue to
import the exception from there.

This clutters things and adds complexity; there is no compelling
reason that the exception's source of truth should not move alongside
all other exceptions.
2020-09-10 17:47:21 -07:00
Anders Kaseorg 69c0959f34 python: Fix misuse of Optional types for optional parameters.
There seems to have been a confusion between two different uses of the
word “optional”:

• An optional parameter may be omitted and replaced with a default
  value.
• An Optional type has None as a possible value.

Sometimes an optional parameter has a default value of None, or None
is otherwise a meaningful value to provide, in which case it makes
sense for the optional parameter to have an Optional type.  But in
other cases, optional parameters should not have Optional type.  Fix
them.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:31:27 -07:00
Anders Kaseorg 365fe0b3d5 python: Sort imports with isort.
Fixes #2665.

Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.

Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start.  I expect this change will increase pressure for us to split
those files, which isn't a bad thing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:45:32 -07:00
Anders Kaseorg 69730a78cc python: Use trailing commas consistently.
Automatically generated by the following script, based on the output
of lint with flake8-comma:

import re
import sys

last_filename = None
last_row = None
lines = []

for msg in sys.stdin:
    m = re.match(
        r"\x1b\[35mflake8    \|\x1b\[0m \x1b\[1;31m(.+):(\d+):(\d+): (\w+)", msg
    )
    if m:
        filename, row_str, col_str, err = m.groups()
        row, col = int(row_str), int(col_str)

        if filename == last_filename:
            assert last_row != row
        else:
            if last_filename is not None:
                with open(last_filename, "w") as f:
                    f.writelines(lines)

            with open(filename) as f:
                lines = f.readlines()
            last_filename = filename
        last_row = row

        line = lines[row - 1]
        if err in ["C812", "C815"]:
            lines[row - 1] = line[: col - 1] + "," + line[col - 1 :]
        elif err in ["C819"]:
            assert line[col - 2] == ","
            lines[row - 1] = line[: col - 2] + line[col - 1 :].lstrip(" ")

if last_filename is not None:
    with open(last_filename, "w") as f:
        f.writelines(lines)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 16:04:12 -07:00
Anders Kaseorg 8dd83228e7 python: Convert "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, but with the
NamedTuple changes reverted (see commit
ba7906a3c6, #15132).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 15:31:20 -07:00
Anders Kaseorg fead14951c python: Convert assignment type annotations to Python 3.6 style.
This commit was split by tabbott; this piece covers the vast majority
of files in Zulip, but excludes scripts/, tools/, and puppet/ to help
ensure we at least show the right error messages for Xenial systems.

We can likely further refine the remaining pieces with some testing.

Generated by com2ann, with whitespace fixes and various manual fixes
for runtime issues:

-    invoiced_through: Optional[LicenseLedger] = models.ForeignKey(
+    invoiced_through: Optional["LicenseLedger"] = models.ForeignKey(

-_apns_client: Optional[APNsClient] = None
+_apns_client: Optional["APNsClient"] = None

-    notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
-    signup_notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
+    notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
+    signup_notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)

-    author: Optional[UserProfile] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE)
+    author: Optional["UserProfile"] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE)

-    bot_owner: Optional[UserProfile] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL)
+    bot_owner: Optional["UserProfile"] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL)

-    default_sending_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
-    default_events_register_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
+    default_sending_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
+    default_events_register_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)

-descriptors_by_handler_id: Dict[int, ClientDescriptor] = {}
+descriptors_by_handler_id: Dict[int, "ClientDescriptor"] = {}

-worker_classes: Dict[str, Type[QueueProcessingWorker]] = {}
-queues: Dict[str, Dict[str, Type[QueueProcessingWorker]]] = {}
+worker_classes: Dict[str, Type["QueueProcessingWorker"]] = {}
+queues: Dict[str, Dict[str, Type["QueueProcessingWorker"]]] = {}

-AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional[LDAPSearch] = None
+AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional["LDAPSearch"] = None

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-22 11:02:32 -07:00
Anders Kaseorg c734bbd95d python: Modernize legacy Python 2 syntax with pyupgrade.
Generated by `pyupgrade --py3-plus --keep-percent-format` on all our
Python code except `zthumbor` and `zulip-ec2-configure-interfaces`,
followed by manual indentation fixes.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-09 16:43:22 -07:00
Tim Abbott 0599273cfa bitbucket: Fix support for pushes that update a tag.
Previously, we threw an exception if created/closed were both unset;
apparently that can happen when pushing an update to a tag.
2020-04-07 17:00:20 -07:00
Tim Abbott c10cc24ee8 python: Sort webhooks imports with isort. 2020-01-14 13:07:47 -08:00
Eeshan Garg daea9a543b webhooks/bitbucket2: Account for missing username in PR payloads.
Certain PR payloads do not have a username for the reviewer, in
such cases, we just return the reviewer's nickname or display
name.
2019-06-24 23:19:06 -07:00
Eeshan Garg 0d519ab146 webhooks/bitbucket2: Account for missing username in user data.
Not all payloads contain the user's username. In such cases, we
should use the user's display name or nickname instead.
2019-06-03 20:00:17 -07:00
Eeshan Garg ab8aae6d0c git_webhooks: Use proper punctuation for PR/issue messages. 2019-05-07 16:45:01 -07:00
Anders Kaseorg 39ac378220 webhooks: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:05:20 -08:00
Eeshan Garg f9ee393e4f webhooks/common: Allow suppressing exceptions for missing headers.
This is useful in cases where an event type HTTP header isn't
crucial for inferring the payload's type.
2019-02-01 21:19:26 -03:30
Eeshan Garg d999e79a8b webhooks/bitbucket2: Decode improperly encoded stream names and topics. 2018-12-29 14:51:38 -08:00
Steve Howell 1fff890b71 webhooks: Change *_SUBJECT_TEMPLATE to *_TOPIC_TEMPLATE. 2018-11-12 15:47:11 -08:00
Steve Howell ea98a44db3 webhooks: Replace SUBJECT_WITH_* with TOPIC_WITH_*. 2018-11-12 15:47:11 -08:00
Eeshan Garg 9e625b881b webhooks/bitbucket2: Improve commit_status_changed type messages.
Instead of just referring to the commit with the raw URL, we
should use the commit ID as the text of the hyperlink.

Note that in commit_status_changed type messages, the name of the
commit isn't available.
2018-10-04 12:16:06 -07:00
Eeshan Garg b22254835e webhooks/bitbucket2: Fix broken commit URL.
The function that generates the body of the commit_status_changed
event messages generated an invalid commit URL.

Most likely, we missed this because this event type is fairly
vague and it is possible it was never tested by users much,
if at all.
2018-10-04 12:16:06 -07:00
Eeshan Garg d8101ca139 webhooks/bitbucket2: Improve test coverage.
The lack of coverage was due to:

* An unused function that was never used anywhere.
* get_commit_status_changed_body was using a regex where it didn't
  really need to use one. And there was an if statement that
  assumed that the payload might NOT contain the URL to the commit.
  However, I checked the payload and there shouldn't be any instances
  where a commit event is generated but there is no URL to the commit.
* get_push_tag_body had an `else` condition that really can't happen
  in any payload. I verified this by checking the BitBucket webhook
  docs.
2018-10-04 12:16:06 -07:00
Eeshan Garg 4a54e8ac17 webhooks/bitbucket2: Include title in message body if not in topic.
This is a follow-up in response to Tim's comments on #9951.

In instances where all messages from a BitBucket integration are
grouped under one user specified topic (specified in the URL), we
should include the title of the PR in the message body, since
the availability of a user-specified topic precludes us from
including it in the topic itself (which was the default behaviour).
2018-07-26 16:35:09 -07:00
Eeshan Garg e0ef831993 webhooks: Migrate to UnexpectedWebhookEventType.
This improves test coverage for a lot of our webhooks that relied
on ad-hoc methods to handle unexpected event types.

Note that I have deliberately skipped github_legacy, it isn't
advertised and is officially deprecated.

Also, I have refrained from making further changes to Trello, I
believe further improvements to test coverage should be covered
in separate per-webhook commits/PRs.
2018-05-22 08:30:19 -07:00
Aditya Bansal a40ae4cae5 zerver/webhooks: Change use of typing.Text to str. 2018-05-12 15:21:24 -07:00
Eeshan Garg 6817232a67 webhooks/bitbucket2: Support events of type "repo:updated". 2018-05-05 15:48:38 -07:00
Eeshan Garg 9fb9c0d901 webhooks: Migrate to validate_extract_webhook_http_header.
This is a part of our efforts to close #6213.
2018-05-05 15:48:38 -07:00
Eeshan Garg 026ff48020 webhooks/bitbucket2: Detect pushes that remove branches correctly.
This was a user-reported bug and a very subtle and painful one
to track down.

Previously, if payload['push']['changes'][i]['closed'] was True,
we assumed that a branch was removed. Looking at whether `closed`
was set to True or not was our way to tell whether a push removed
a branch or not.

However, this is wrong! `closed` being set to True can also mean
that the pull request associated with the branch was approved but
the branch itself was not deleted. According to the BitBucket docs,
the correct way to see if a branch is deleted is to check if `new`
is null.

This bug was leading to KeyErrors about not being able to find
the `commits` key, which shouldn't happen anymore!
2018-04-02 19:30:09 -02:30
Eeshan Garg 93678e89cd webhooks: Migrate 14 webhooks to use check_send_webhook_message.
These are the straightforward ones.

Note that there is a line in zerver.lib.test_classes.build_webhook_url
that lost test coverage. That's because most of our tests test using
stream messages so the webhook URLs being tested always have a query
parameter. So the line that accounts for there being no query
parameters never gets called, which is fine, but we should still
keep it.
2018-03-16 11:34:20 -07:00
picapi_ d038e7ea6b mypy: Use Python 3 type syntax in zerver/webhooks/bitbucket2/view.py. 2017-12-11 20:27:55 -08:00
Eeshan Garg 5b8f38f4e3 webhooks/bitbucket2: Ignore push events with no changes.
We now ignore payloads where payload['push']['changes'] is empty,
because an empty push doesn't really convey any useful information.

I couldn't find a way to replicate the action that would generate
such a payload, so I took one of our existing payloads and editted
out payload['push']['changes'] myself, so this payload is not
authentic.
2017-11-26 17:03:07 -08:00
Tim Abbott 73a668e7ae python: Sort imports in webhooks. 2017-11-15 15:43:10 -08:00
rht bafba95eb7 Text-wrap the remaining long lines exceeding 110. 2017-11-15 10:58:03 -08:00