Transifex's webhook documentation[^1] describes an `event` parameter
which is used to distinguish which event type was received. Dispatch
based on that, and pass that value to UnsupportedWebhookEventTypeError
if need be.
[^1]: https://developers.transifex.com/docs/webhooks
This kind of payload that's loaded from json in the body of the request
is not only used for webhooks, but also in the push bouncer, and may get
used elsewhere too - so a general name is better.
The type annotation for functools.partial uses unchecked Any for all
the function parameters (both early and late). returns.curry.partial
uses a mypy plugin to check the parameters safely.
https://returns.readthedocs.io/en/latest/pages/curry.html
Signed-off-by: Anders Kaseorg <anders@zulip.com>
The Librato webhook requires a mapping (which should be considered
immutable) with a default value. Ruff reports a false-positive due to
the Json wrapper.
Instead of a WildValue, the JSON/Sentry webhook expect the request body to be a
dict.
For the JSON webhook, json.dumps accepts other types of input as well and the
constraint is not necessary, but this serve as a good example of an alternative
use of WebhookPayload to describe a payload that is intended to be parsed from
the entire request body from JSON, into a type other than WildValue.
Transifex has parameters that need to be parsed from JSON and converted
to int. Note that we use Optional[Json[int]] instead of
Json[Optional[int]] to replicate the behavior of json_validator. This
caveat is explained in a new test called test_json_optional.
These webhooks have some URL query params that do not need additional
validation or parsing from JSON. So WebhookPaylaod is not applicable to
these webhooks.
This converts most webhook integration views to use @typed_endpoint instead
of @has_request_variables, rewriting REQ parameters. For these
webhooks, it simply requires switching the decorator, rewriting the
type annotation of payload/message to WebhookPayload[WildValue], and
removing the REQ default that defines the to_wild_value converter.
Previously, if a user tried to create a webhook using the Webhooks
plugin in Sentry and used the "Test plugin" to test the webhook,
the server would send a 500 error, even though the integration
worked perfectly. This led users to believe that the integration
was not working.
Fixes#26173.
Translators benefit from the extra information in the field names, and
need the reordering freedom that isn’t available with multiple
positional fields.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Pass the HttpRequest explicitly through the two webhooks that log to
the webhook loggers.
get_current_request is now unused, so remove it (in the same commit
for test coverage reasons).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Because the third party might not be expecting a 400 from our
webhooks, we now instead use 200 status code for unknown events,
while sending back the error to Sentry. Because it is no longer an error
response, the response type should now be "success".
Fixes#24721.
This commit creates separate events for issue milestoned and
demilestoned notifications. This allows the end-users to choose
whether they want these notifications or not.
Fixes#25793.
This commit creates separate events for issue labeled and
unlabeled notifications. This allows the end-users to choose
whether they want these notifications or not.
Fixes#25789.
In #23380 we want to change all occurrences of `uri` with `url`.
This commit changes the names of two variables `external_uri_scheme`
and `main_site_uri`, who are constructed using `settings` constants.
This commit changes the assignee info to the correct one.
Earlier, the current assignee info of the issue was used
to create the message but when an issue was unassigned
this would give incorrect information if there are still
assignees left on the issue. Now, it uses the details of
the user who has been unassigned from the issue.
This commit removes the description from the Issue Assigned
event message that is sent by the Github Integration.
Previously when an issue was assigned, the message also had the
description of the issue, which is redundant as the description
has been already sent through the issue created event. After
this commit, the description no longer appears in the message.
Fixes#24554
This swaps out url_format_string from all of our APIs and replaces it
with url_template. Note that the documentation changes in the following
commits will be squashed with this commit.
We change the "url_format" key to "url_template" for the
realm_linkifiers events in event_schema, along with updating
LinkifierDict. "url_template" is the name chosen to normalize
mixed usages of "url_format_string" and "url_format" throughout
the backend.
The markdown processor is updated to stop handling the format string
interpolation and delegate the task template expansion to the uri_template
library instead.
This change affects many test cases. We mostly just replace "%(name)s"
with "{name}", "url_format_string" with "url_template" to make sure that
they still pass. There are some test cases dedicated for testing "%"
escaping, which aren't relevant anymore and are subject to removal.
But for now we keep most of them as-is, and make sure that "%" is always
escaped since we do not use it for variable substitution any more.
Since url_format_string is not populated anymore, a migration is created
to remove this field entirely, and make url_template non-nullable since
we will always populate it. Note that it is possible to have
url_template being null after migration 0422 and before 0424, but
in practice, url_template will not be None after backfilling and the
backend now is always setting url_template.
With the removal of url_format_string, RealmFilter model will now be cleaned
with URL template checks, and the old checks for escapes are removed.
We also modified RealmFilter.clean to skip the validation when the
url_template is invalid. This avoids raising mulitple ValidationError's
when calling full_clean on a linkifier. But we might eventually want to
have a more centric approach to data validation instead of having
the same validation in both the clean method and the validator.
Fixes#23124.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This is a follow up to #24673, we want to modify every webhook events to
follow the same pattern and consistency where branch name should only
show on opened and merged events.
This commit passes the body of the PR Review as the message to
the helper function that generates the message to be sent by the
GitHub Integration.
Previously when a PR Review was done the message sent would just
include the link of the review but the message didn't include the
body the review. After this commit, the message also includes the
body of the review.
Fixes#24676
This is a follow up to #24673, we want to modify every webhook events to
follow the same pattern and consistency where branch name should only
show on opened and merged events.
This is a follow up to #24673, we want to modify every webhook events to
follow the same pattern and consistency where branch name should only
show on opened and merged events.
This is a prep commit to help make the changes to make changes to pull
event message easier. Our Bitbucket has been using a custom template to
render the reviewers. This means that values are fixed to how the templates
like it. These changes will allow `get_pull_request_event_message` to
support reviewer and allow for a easier and flexible adjustment to these
messages if needed.
Previously, the assignee message would stick around in the middle of the
event message. This doesn't look as good as if we put it to the end of
the event message. These changes does just that and move the assignee
messages towards the end of the event message to make it look better
and cleaner for the readers.
Previously, some call sites for the function provided optional
arguments as positional arguments. These changes will allow the
arguments to be passed as keyword arguments to the function and
fix up the call sites of the function to pass keyword arguments
instead.
Previously, some call sites for the function provided optional
arguments as positional arguments. These changes will allow the
arguments to be passed as keyword arguments to the function and
fix up the call sites of the function to pass keyword arguments
instead.
This is a follow up to #24673, we want to modify every webhook events to
follow the same pattern and consistency where branch name should only
show on opened and merged events.
This is a follow up to #24673, we want to modify every webhook events to
follow the same pattern and consistency where branch name should only
show on opened and merged events.
This is a follow up to #24673, we want to modify every webhook
events to follow the same pattern and consistency where branch name
should only show on opened and merged events.
Previously when Github bot receives an update pull request event,it
will produce the following message:
user updated PR #1 Start writing unit tests from test to main
"from test to main" is improper and causes unnecessary confusion.
These changes will update the logic to remove the phrase from
update events. These changes will also include the org: prefix to
the branch names to keep it consistent with Github and further
reduce confusions on branch names.
Fixes#24536.
The list of supported events for filtering itself does not document what
each of the events does. Adding a link to GitHub's documentation would
be a pointer to get people started. But ideally we need to establish a
better system to document the events in general.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Updates the Asana documentation, which was a detailed version
of the Zapier documentation with screenshots specifically for
Asana, to instead start with the basic incoming webhook steps
and then point to the general Zapier documentation to complete
the integration.
This will be easier to maintain moving forward in the short
term as ideally we'll migrate to a system that documents all
of the integrations with Zulip that are available via Zapier.
Also, updates the current Zapier documentation to mention
Asana as one of the apps that can be integrated with Zulip.
Black 23 enforces some slightly more specific rules about empty line
counts and redundant parenthesis removal, but the result is still
compatible with Black 22.
(This does not actually upgrade our Python environment to Black 23
yet.)
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Intercom sends a HEAD request to validate the webhook URL on their side,
which was not expected in the previous implementation.
This fixes the problem that we send out a confusing error message for it.
Fixes#23912.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This is a best-effort rendering of the "fields" of Slack incoming
hooks, which Slack renders in two columns. We approximate them in a
Markdown table, with some minor in-place replacements.
Fixes#22228.
`check_text_block` transformed its input, making the object it
returned not the same object it was passed; this invalidated it for
use in `check_list`. It is also, in general, unlike all other
validators.
Make it return a TypedDict cast of its input.
Since the removal of `CurlHttpRequest` in Rabbix 6.2, the old script for
setting up the Zabbix integration no longer works.
https://www.zabbix.com/documentation/6.2/en/manual/installation/upgrade_notes_620?hl=CurlHttpRequest#curlhttprequest-removed
This updates the documentation to use `HttpRequest` instead and keep it
up-to-date with the latest Zabbix server. We raise the minimum supported
version from 5.2 to 5.4 because `HttpRequest` was introduced in 5.4.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
The previous regular expression required a `[^\w]` at the start and
end of the match. This had two unintended effects -- it meant that it
could never match at the start or end of a string, and it meant
that *adjacent* words required *two* non-word characters between them,
as the pattern matches cannot overlap.
Switch to allowing string start/end to anchor the matches, and make
the trailing `[^\w]` be a zero-width look-ahead, to allow the patterns
to overlap. Also remove the spurious `^` within the inner character
classes, which prevented `*foo^bar*` from matching. Finally, add
tests to cover the functionality, which was previously untested.
Updates the Hello World integration documentation and the section
of the related tutorial on documenting the example integration
for the currently used shared macro `create-bot-construct-url.md`.
Also, updates them to use the numbered style currently used in
the majority of the integrations documentation pages.
CircleCI has updated its webhook format[1] for CircleCI Cloud,
Server version 3.x and 4.x. This commit rewrites the CircleCI
integration to parse the new webhook structure. The tests have also
been rewritten for the new format.
With this commit, we support webhooks from projects that use GitHub,
BitBucket and GitLab as VCS providers. The CircleCI integration doc
has been updated to mention the same. The doc has also been updated
with the latest instructions for configuring a webhook on the CircleCI
interface, and the new output screenshots.
References:
[1]: https://circleci.com/docs/webhooks
This commit adds support for Grafana's new alerting system, Grafana
Alerting. The existing Grafana integration has been modified to
detect the version of the notification through the structure of the
payload body, since the the structure varies by version. Support for
legacy alerting is been continued. Example fixtures have been added
for Grafana Alerting's webhooks.
Tests updated.
This unifies the length of the shortened SHA our integrations generate,
and ensures that they are long enough for projects of various sizes with
a chosen value defined in get_short_sha.
Fixes#23475
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
7 characters are not enough for large projects, so we change
it to reasonably longer. As an example, The Linux kernel needs
at least 11 characters of sha in its shortened form to identify
a revision. We pick 11 so it should work for most of the projects.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Several integration docs instruct the user to create a bot, but don't
specify that the type of bot should be "Incoming webhook".
Renames create-a-bot.md -> create-an-incoming-webhook.md for clarity,
and replaces all incomplete instructions with this macro.
Renames bot_types.png -> bot_types_incoming_webhook.png and updates
the image with a screenshot of the latest UI.
The Yo company shut down in 2016.
https://en.wikipedia.org/wiki/Yo_(app)#History
Removes `yo` instances from `zerver/lib/integrations.py`.
Removes `zerver/webhooks/yo`.
Removes `static/images/integrations/yo-app`.
Since the setup for uptimerobot does not deviate from what we have in
the "create-bot-construct-url" macro, we can reuse it and event
filtering instructions for uptimerobot will be automatically included.
TODO: Add event filtering documentation for buildbot when it supports
custom endpoint URL.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This commit strengthens types by typing the Solano webhook's incoming
payload as WildValue, which eradicates the use of Any within the
incoming webhook integration.
The KeyError exception has been replaced to catch a ValidationError
instead now, since the incoming payload's keys will be tamed before
usage and the non-existence of the key is raised as a
ValidationError in the taming function.
This commit strengthens types by typing the webhook's incoming
payload as WildValue and taming the values of the payload before
usage, which eradicates the use of Any within the UptimeRobot
incoming webhook integration.
The payload's values are now tamed, stored in variables and passed
into message templates as opposed to passing in the payload dict as
keyword arguments.
The KeyError exception has been replaced to catch a ValidationError
instead now, since the incoming payload's keys will be tamed before
usage and the non-existence of the key is raised as a
ValidationError in the taming function.
This commit strengthens types by typing the webhook's incoming
payload as WildValue, which eradicates the use of Any within the
Zabbix incoming webhook integration.
The KeyError exception has been replaced to catch a ValidationError
instead now, since the incoming payload's keys will be tamed before
usage and the non-existence of the key is raised as a
ValidationError in the taming function.
This commit checks for null values for keys within "attachment" in
the Slack integration's incoming payloads. These keys were expected
to exist optionally previously, and the existence of null values for
these wasn't anticipated. Due to an issue report for such null
values in the payload, their handling is updated appropriately.
The checks for these values are truthiness checks since the strategy
for these values being null or falsy ("", 0) is the same; we don't
process that key-value pair. This is consistent with how Slack handles
this scenario.
For the case where all the attachment fields have null values, Slack
displays this as an empty block with no content, and therefore our
strategy for this is a no-op.
Tests updated.
This is a follow-up to #19274. We map the supported event types to a
more suitable format for events.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Now that we can assume Python 3.6+, we can use the
email.headerregistry module to replace hacky manual email address
parsing.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Newrelic updated the payload that's sent via the webhook incoming call
causing a bug in the newrelic webhook endpoint.
This fixes the bug by updating the endpoint to respect the new format
of the payload as well as the old format. This should be updated once
the old format is EOLed.
Fixes#22338.
Updates `git-webhook-url-with-branches.md` (and two files that use
that file as an include link) for some of the follow-ups from #22315
to the Markdown parser. With this fix, all integrations docs that
reference this file as an include link should render the url as a
div element with `.codehilite` class.
This is a prep commit for tightening the types for our wrapped test
client.
The callers of the test client methods are refactored to either call
them without unpacking at all or create a TypedDict for the keyword
arguments to be unpacked. This allows the type checker to know exactly what
keys are present and their corresponding type.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
markdown-include is GPL licensed.
Also, rewrite it as a block processor, so that it works correctly
inside indented blocks.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Anders Kaseorg
Alex Vandiver
Alya Abbott
Mateusz Mandera
Zixuan James Li
Satyam Bansal
abdullahm1
Lauryn Menard
dhruv302003
AcKindle3
Joelute
Hari Prashant Bhimaraju
David Rosa
Matt Keller
Christopher Chong
Adam Sah
Israel Ebonko
rodwan-bakkar