2018-07-04 02:08:27 +02:00
|
|
|
# Incoming webhook integrations
|
|
|
|
|
2018-10-12 01:54:13 +02:00
|
|
|
An incoming webhook allows a third-party service to push data to Zulip when
|
2020-05-09 20:22:11 +02:00
|
|
|
something happens. There's several ways to do an incoming webhook in
|
2019-05-21 04:17:11 +02:00
|
|
|
Zulip:
|
2018-07-04 02:08:27 +02:00
|
|
|
|
2019-05-21 04:17:11 +02:00
|
|
|
* Use our [REST API](/api/rest) endpoint for [sending
|
|
|
|
messages](/api/send-message). This works great for internal tools
|
|
|
|
or cases where the third-party tool wants to control the formatting
|
|
|
|
of the messages in Zulip.
|
2020-05-09 20:22:11 +02:00
|
|
|
* Use one of our supported [integration
|
2021-05-07 02:55:31 +02:00
|
|
|
frameworks](/integrations/meta-integration), such as the
|
2020-05-09 20:22:11 +02:00
|
|
|
[Slack-compatible incoming webhook](/integrations/doc/slack_incoming),
|
|
|
|
[Zapier integration](/integrations/docs/zapier), or
|
|
|
|
[IFTTT integration](/integrations/doc/ifttt).
|
2019-05-21 04:17:11 +02:00
|
|
|
* Adding an incoming webhook integration (detailed on this page),
|
|
|
|
where all the logic for formatting the Zulip messages lives in the
|
|
|
|
Zulip server. This is how most of [Zulip's official
|
|
|
|
integrations](/integrations) work, because they enable Zulip to
|
|
|
|
support third-party services that just have an "outgoing webhook"
|
|
|
|
feature (without the third party needing to do any work specific to
|
|
|
|
Zulip).
|
|
|
|
|
|
|
|
In an incoming webhook integration, the third-party service's
|
|
|
|
"outgoing webhook" feature sends an `HTTP POST`s to a special URL when
|
|
|
|
it has something for you, and then the Zulip "incoming webhook"
|
|
|
|
integration handles that incoming data to format and send a message in
|
|
|
|
Zulip.
|
|
|
|
|
|
|
|
New official Zulip webhook integrations can take just a few hours to
|
|
|
|
write, including tests and documentation, if you use the right
|
|
|
|
process.
|
2018-07-04 02:08:27 +02:00
|
|
|
|
2018-10-12 01:54:13 +02:00
|
|
|
## Quick guide
|
|
|
|
|
|
|
|
* Set up the
|
|
|
|
[Zulip development environment](https://zulip.readthedocs.io/en/latest/development/overview.html).
|
|
|
|
|
2021-04-09 03:50:04 +02:00
|
|
|
* Use [Zulip's JSON integration](/integrations/doc/json),
|
|
|
|
<https://webhook.site/>, or a similar site to capture an example
|
2018-10-12 01:54:13 +02:00
|
|
|
webhook payload from the third-party service. Create a
|
2021-04-09 03:50:04 +02:00
|
|
|
`zerver/webhooks/<mywebhook>/fixtures/` directory, and add the
|
|
|
|
captured payload as a test fixture.
|
2018-10-12 01:54:13 +02:00
|
|
|
|
|
|
|
* Create an `Integration` object, and add it to `WEBHOOK_INTEGRATIONS` in
|
|
|
|
`zerver/lib/integrations.py`. Search for `webhook` in that file to find an
|
|
|
|
existing one to copy.
|
|
|
|
|
|
|
|
* Write a draft webhook handler under `zerver/webhooks/`. There are a lot of
|
|
|
|
examples in that directory that you can copy. We recommend templating off
|
|
|
|
a short one, like `stash` or `zendesk`.
|
|
|
|
|
|
|
|
* Add a test for your fixture at `zerver/webhooks/<mywebhook>/tests.py`.
|
|
|
|
Run the tests for your integration like this:
|
2018-07-04 02:08:27 +02:00
|
|
|
|
|
|
|
```
|
2018-10-12 01:54:13 +02:00
|
|
|
tools/test-backend zerver/webhooks/<mywebhook>/
|
2018-07-04 02:08:27 +02:00
|
|
|
```
|
|
|
|
|
2018-10-12 01:54:13 +02:00
|
|
|
Iterate on debugging the test and webhooks handler until it all
|
|
|
|
works.
|
|
|
|
|
|
|
|
* Capture payloads for the other common types of `POST`s the third-party
|
|
|
|
service will make, and add tests for them; usually this part of the
|
|
|
|
process is pretty fast.
|
|
|
|
|
|
|
|
* Document the integration (required for getting it merged into Zulip). You
|
|
|
|
can template off an existing guide, like
|
2021-09-01 00:15:31 +02:00
|
|
|
[this one](https://raw.githubusercontent.com/zulip/zulip/main/zerver/webhooks/github/doc.md).
|
2018-10-12 01:54:13 +02:00
|
|
|
This should not take more than 15 minutes, even if you don't speak English
|
|
|
|
as a first language (we'll clean up the text before merging).
|
2018-07-04 02:08:27 +02:00
|
|
|
|
2018-10-12 01:54:13 +02:00
|
|
|
## Hello world walkthrough
|
2018-07-04 02:08:27 +02:00
|
|
|
|
2018-10-17 05:08:27 +02:00
|
|
|
Check out the [detailed walkthrough](incoming-webhooks-walkthrough) for step-by-step
|
2018-10-12 01:54:13 +02:00
|
|
|
instructions.
|
2018-07-04 02:08:27 +02:00
|
|
|
|
2018-10-12 01:54:13 +02:00
|
|
|
## Checklist
|
|
|
|
|
|
|
|
### Files that need to be created
|
2018-07-04 02:08:27 +02:00
|
|
|
|
|
|
|
Select a name for your incoming webhook and use it consistently. The examples
|
2018-10-12 01:54:13 +02:00
|
|
|
below are for a webhook named `MyWebHook`.
|
|
|
|
|
|
|
|
* `zerver/webhooks/mywebhook/__init__.py`: Empty file that is an obligatory
|
|
|
|
part of every python package. Remember to `git add` it.
|
|
|
|
* `zerver/webhooks/mywebhook/view.py`: The main webhook integration function
|
|
|
|
as well as any needed helper functions.
|
2018-07-04 02:08:27 +02:00
|
|
|
* `zerver/webhooks/mywebhook/fixtures/messagetype.json`: Sample json payload data
|
|
|
|
used by tests. Add one fixture file per type of message supported by your
|
2018-10-12 01:54:13 +02:00
|
|
|
integration.
|
|
|
|
* `zerver/webhooks/mywebhook/tests.py`: Tests for your webbook.
|
2021-05-07 02:55:31 +02:00
|
|
|
* `zerver/webhooks/mywebhook/doc.md`: End-user documentation explaining
|
2018-10-12 01:54:13 +02:00
|
|
|
how to add the integration.
|
|
|
|
* `static/images/integrations/logos/mywebhook.svg`: A square logo for the
|
|
|
|
platform/server/product you are integrating. Used on the documentation
|
|
|
|
pages as well as the sender's avatar for messages sent by the integration.
|
|
|
|
* `static/images/integrations/mywebbook/001.svg`: A screenshot of a message
|
2021-05-07 02:55:31 +02:00
|
|
|
sent by the integration, used on the documentation page. This can be
|
|
|
|
generated by running `tools/generate-integration-docs-screenshot --integration mywebhook`.
|
|
|
|
* `static/images/integrations/bot_avatars/mywebhook.png`: A square logo for the
|
|
|
|
platform/server/product you are integrating which is used to create the avatar
|
|
|
|
for generating screenshots with. This can be generated automatically from
|
|
|
|
`static/images/integrations/logos/mywebhook.svg` by running
|
|
|
|
`tools/setup/generate_integration_bots_avatars.py`.
|
2018-10-12 01:54:13 +02:00
|
|
|
|
|
|
|
### Files that need to be updated
|
2018-07-04 02:08:27 +02:00
|
|
|
|
|
|
|
* `zerver/lib/integrations.py`: Add your integration to
|
2018-10-12 01:54:13 +02:00
|
|
|
`WEBHOOK_INTEGRATIONS`. This will automatically register a
|
|
|
|
URL for the incoming webhook of the form `api/v1/external/mywebhook` and
|
|
|
|
associate it with the function called `api_mywebhook_webhook` in
|
2021-05-07 02:55:31 +02:00
|
|
|
`zerver/webhooks/mywebhook/view.py`. Also add your integration to
|
|
|
|
`DOC_SCREENSHOT_CONFIG`. This will allow you to automatically generate
|
|
|
|
a screenshot for the documentation by running
|
|
|
|
`tools/generate-integration-docs-screenshot --integration mywebhook`.
|
2018-10-17 03:51:53 +02:00
|
|
|
|
2021-05-06 16:02:36 +02:00
|
|
|
## Common Helpers
|
|
|
|
|
|
|
|
* If your integration will receive a test webhook payload, you can use
|
|
|
|
`get_setup_webhook_message` to create our standard message for test payloads.
|
|
|
|
You can import this from `zerver/lib/webhooks/common.py`, and it will generate
|
|
|
|
a message like this: "GitHub webhook is successfully configured! 🎉"
|
|
|
|
|
2018-10-17 03:51:53 +02:00
|
|
|
## General advice
|
|
|
|
|
|
|
|
* Consider using our Zulip markup to make the output from your
|
2020-08-11 01:47:49 +02:00
|
|
|
integration especially attractive or useful (e.g. emoji, Markdown
|
markdown: Remove !avatar() and !gravatar() syntax.
This particular commit has been a long time coming. For reference,
!avatar(email) was an undocumented syntax that simply rendered an
inline 50px avatar for a user in a message, essentially allowing
you to create a user pill like:
`!avatar(alice@example.com) Alice: hey!`
---
Reimplementation
If we decide to reimplement this or a similar feature in the future,
we could use something like `<avatar:userid>` syntax which is more
in line with creating links in markdown. Even then, it would not be
a good idea to add this instead of supporting inline images directly.
Since any usecases of such a syntax are in automation, we do not need
to make it userfriendly and something like the following is a better
implementation that doesn't need a custom syntax:
`![avatar for Alice](/avatar/1234?s=50) Alice: hey!`
---
History
We initially added this syntax back in 2012 and it was 'deprecated'
from the get go. Here's what the original commit had to say about
the new syntax:
> We'll use this internally for the commit bot. We might eventually
> disable it for external users.
We eventually did start using this for our github integrations in 2013
but since then, those integrations have been neglected in favor of
our GitHub webhooks which do not use this syntax.
When we copied `!gravatar` to add the `!avatar` syntax, we also noted
that we want to deprecate the `!gravatar` syntax entirely - in 2013!
Since then, we haven't advertised either of these syntaxes anywhere
in our docs, and the only two places where this syntax remains is
our game bots that could easily do without these, and the git commit
integration that we have deprecated anyway.
We do not have any evidence of someone asking about this syntax on
chat.zulip.org when developing an integration and rightfully so- only
the people who work on Zulip (and specifically, markdown) are likely
to stumble upon it and try it out.
This is also the only peice of code due to which we had to look up
emails -> userid mapping in our backend markdown. By removing this,
we entirely remove the backend markdown's dependency on user emails
to render messages.
---
Relevant commits:
- Oct 2012, Initial commit c31462c2782a33886e737cf33424a36a95c81f97
- Nov 2013, Update commit bot 968c393826f8846065c5c880427328f6e534c2f5
- Nov 2013, Add avatar syntax 761c0a0266669aca82d134716a4d6b6e33d541fc
- Sep 2017, Avoid email use c3032a7fe8ed49b011e0d242f4b8a7d756b9f647
- Apr 2019, Remove from webhook 674fcfcce1fcf35bdc57031a1025ef169d495d36
2020-07-06 23:01:38 +02:00
|
|
|
emphasis or @-mentions).
|
2018-10-17 03:51:53 +02:00
|
|
|
|
|
|
|
* Use topics effectively to ensure sequential messages about the same
|
|
|
|
thing are threaded together; this makes for much better consumption
|
|
|
|
by users. E.g. for a bug tracker integration, put the bug number in
|
|
|
|
the topic for all messages; for an integration like Nagios, put the
|
|
|
|
service in the topic.
|
|
|
|
|
|
|
|
* Integrations that don't match a team's workflow can often be
|
|
|
|
uselessly spammy. Give careful thought to providing options for
|
|
|
|
triggering Zulip messages only for certain message types, certain
|
|
|
|
projects, or sending different messages to different streams/topics,
|
|
|
|
to make it easy for teams to configure the integration to support
|
|
|
|
their workflow.
|
|
|
|
|
|
|
|
* Consistently capitalize the name of the integration in the
|
|
|
|
documentation and the Client name the way the vendor does. It's OK
|
|
|
|
to use all-lower-case in the implementation.
|
|
|
|
|
|
|
|
* Sometimes it can be helpful to contact the vendor if it appears they
|
|
|
|
don't have an API or webhook we can use; sometimes the right API
|
|
|
|
is just not properly documented.
|
|
|
|
|
|
|
|
* A helpful tool for testing your integration is
|
|
|
|
[UltraHook](http://www.ultrahook.com/), which allows you to receive webhook
|
|
|
|
calls via your local Zulip development environment. This enables you to do end-to-end
|
|
|
|
testing with live data from the service you're integrating and can help you
|
|
|
|
spot why something isn't working or if the service is using custom HTTP
|
|
|
|
headers.
|