Add HelloWorld webhook integration + Walkthough.

Improves webhook integration docs and Hello World webhook.  Includes
many suggested improvements from @timabbott and @tomaszkolek.
This commit is contained in:
Christie Koehler 2016-05-20 10:38:46 -07:00 committed by Tim Abbott
parent 2a37dafcbb
commit 31efecf03d
10 changed files with 489 additions and 0 deletions

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

View File

@ -62,6 +62,14 @@ with the product at all.
don't have an API or webhook we can use -- sometimes the right API
is just not properly documented.
* A helpful tool for testing your integration is
[UltraHook](http://www.ultrahook.com/), which allows you to receive webhook
calls via your local Zulip dev environment. This enables you to do end-to-end
testing with live data from the service you're integrating and can help you
spot why something isn't working or if the service is using custom HTTP
headers.
## Writing Webhook integrations
New Zulip webhook integrations can take just a few hours to write,
@ -106,6 +114,377 @@ Here's how we recommend doing it:
* Finally, write documentation for the integration (see below)!
### Files that need to be updated
* `templates/zerver/integrations.html`: Edit to add end-user documentation and
integration icon. See [Documenting your
integration](#documenting-your-integration) for details.
* `zerver/test_hooks.py`: Edit to include tests for your webbook. See [Testing
and writing tests](testing.html) for details.
* `zproject/urls.py`: Edit to add externally available url of the webhook and
associate with the function added to `zerver/views/webhooks/mywebhook.py`
### Files that need to be created
Select a name for your webhook and use it consistently. The examples below are
for a webhook named 'MyWebHook'.
* `static/images/integrations/logos/mywebhook.png`: An image to represent
your integration in the user interface. Generally this Should be the logo of the
platform/server/product you are integrating. See [Documenting your
integration](#documenting-your-integration) for details.
* `static/images/integrations/mywebbook/001.png`: A screen capture of your
integration for use in the user interface. You can add as many images as needed
to effectively document your webhook integration. See [Documenting your
integration](#documenting-your-integration) for details.
* `zerver/fixtures/mywebhook/mywebhook_messagetype.json`: Sample json payload data
used by tests. Add one fixture file per type of message supported by your
integration. See [Testing and writing tests](testing.html) for details.
* `zerver/views/webhooks/mywebhook.py`: Includes the main webhook integration
function including any needed helper functions.
### Walkthrough of `Hello World` webhook
Below explains each part of a simple webhook integration, called **Hello
World**. This webhook sends a "hello" message to the `test` stream and includes
a link to the Wikipedia article of the day, which it formats from json data it
receives in the http request.
Use this walkthrough to learn how to write your first webhook
integration.
#### Step 0: Create fixtures
The first step in creating a webhook is to examine the data that the
service you want to integrate will be sending to Zulip.
You can use [requestb.in](http://requestb.in/) or a similar tool to capture
webook payload(s) from the service you are integrating. Examining this
data allows you to do two things:
1. Determine how you will need to structure your webook code, including what
message types your integration should support and how; and,
2. Create fixtures for your webook tests.
Fixtures enable the testing of webhook integration code without the need to
actually contact the service being integrated.
Because `Hello World` is a very simple webhook that does one thing, it requires
only one fixture, `zerver/fixtures/helloworld/helloworld_hello.json`:
```
{
"featured_title":"Marilyn Monroe",
"featured_url":"https://en.wikipedia.org/wiki/Marilyn_Monroe",
}
```
When writing your own webhook integration, you'll want to write a test function
for each distinct message condition your webhook supports. You'll also need a
corresponding fixture for each of these tests. See [Step 3: Create
tests](#step-3-create-tests) or [Testing](testing.html) for further details.
#### Step 1: Create main webhook code
The majority of the code for your webhook integration will be in a single
python file in `zerver/views/webhooks/`. The name of this file should be the
name of your webhook, all lower-case, with file extension `.py`:
`mywebhook.py`.
The Hello World integration is in `zerver/views/webhooks/helloworld.py`:
```
from __future__ import absolute_import
from django.utils.translation import ugettext as _
from zerver.lib.actions import check_send_message
from zerver.lib.response import json_success, json_error
from zerver.decorator import REQ, has_request_variables, api_key_only_webhook_view
from zerver.lib.validator import check_dict, check_string
from zerver.models import Client, UserProfile
from django.http import HttpRequest, HttpResponse
from six import text_type
from typing import Dict, Any, Iterable, Optional
@api_key_only_webhook_view('HelloWorld')
@has_request_variables
def api_helloworld_webhook(request, user_profile, client,
payload=REQ(argument_type='body'),
stream=REQ(default='test'),
topic=REQ(default='Hello World')):
# type: (HttpRequest, UserProfile, Client, Dict[str, Iterable[Dict[str, Any]]], text_type, Optional[text_type]) -> HttpResponse
# construct the body of the message
body = 'Hello! I am happy to be here! :smile:'
# try to add the Wikipedia article of the day
# return appropriate error if not successful
try:
body_template = '\nThe Wikipedia featured article for today is **[{featured_title}]({featured_url})**'
body += body_template.format(**payload)
except KeyError as e:
return json_error(_("Missing key {} in JSON").format(str(e)))
# send the message
check_send_message(user_profile, client, 'stream', [stream], topic, body)
# return json result
return json_success()
```
The above code imports the required functions and defines the main webhook
function `api_helloworld_webook`, decorating it with `api_key_only_webhook_view` and
`has_request_variables`.
You must pass the name of your webhook to the `api_key_only_webhook_view`
decorator. Here we have used `HelloWorld`. To be consistent with Zulip code
style, use the name of the product you are integrating in camel case, spelled
as the product spells its own name (except always first letter upper-case).
You should name your webhook function as such `api_webhookname_webhook` where
`webhookname` is the name of your webhook and is always lower-case.
At minimum, the webhook function must accept `request` (Django
[HttpRequest](https://docs.djangoproject.com/en/1.8/ref/request-response/#django.http.HttpRequest)
object), `user_profile` (Zulip's user object), and `client` (Zulip's analogue
of UserAgent). You may also want to define additional parameters using the
`REQ` object.
In the example above, we have defined `payload` which is populated from the
body of the http request, `stream` with a default of `test` (available by
default in Zulip dev environment), and `topic` with a default of `Hello World`.
The line that begins `# type` is a mypy type annotation. See [this
page](mypy.html) for details about how to properly annotate your webhook
functions.
In the body of the function we define the body of the message as `Hello! I am
happy to be here! :smile:`. The `:smile:` indicates an emoji. Then we append a
link to the Wikipedia article of the day as provided by the json payload. If
the json payload does not include data for `featured_title` and `featured_url`
we catch a `KeyError` and use `json_error` to return the appropriate
information: a 400 http status code with relevant details.
Then we send a public (stream) message with `check_send_message` which will
validate the message and then send it.
Finally, we return a 200 http status with a JSON format success message via
`json_success()`.
#### Step 2: Create an api endpoint for the webhook
In order for a webhook to be externally available, it must be mapped to a url.
This is done in `zproject/urls.py`. Look for the lines:
```
# Incoming webhook URLs
urls += [
# Sorted integration-specific webhook callbacks.
```
And you'll find the entry for Hello World:
```
url(r'^api/v1/external/helloworld$', 'zerver.views.webhooks.helloworld.api_helloworld_webhook'),
```
This tells the Zulip api to call the `api_helloworld_webhook` function in
`zerver/views/webhooks/helloworld.py` when it receives a request at
`/api/v1/external/helloworld`.
At this point, if you're following along and/or writing your own Hello World
webhook, you have written enough code to test your integration.
You can do so by using Zulip itself or curl on the command line.
Using `manage.py` from within Zulip Dev environment:
```
(zulip-venv)vagrant@vagrant-ubuntu-trusty-64:/srv/zulip$
./manage.py send_webhook_fixture_message \
> --fixture=zerver/fixtures/helloworld/helloworld_hello.json \
> '--url=http://localhost:9991/api/v1/external/helloworld?api_key=<api_key>'
```
After which you should see something similar to:
```
2016-07-07 15:06:59,187 INFO 127.0.0.1 POST 200 143ms (mem: 6ms/13) (md: 43ms/1) (db: 20ms/9q) (+start: 147ms) /api/v1/external/helloworld (helloworld-bot@zulip.com via ZulipHelloWorldWebhook)
```
Using curl:
```
curl -X POST -H "Content-Type: application/json" -d '{ "featured_title":"Marilyn Monroe", "featured_url":"https://en.wikipedia.org/wiki/Marilyn_Monroe" }' http://localhost:9991/api/v1/external/helloworld\?api_key\=<api_key>
```
After which you should see:
```
{"msg":"","result":"success"}
```
Using either method will create a message in Zulip:
![Image of Hello World webhook message](images/helloworld-webhook.png)
#### Step 3: Create tests
Every webhook integraton should have a corresponding test class in
`zerver/tests/test_hooks.py`.
You should name the class `<WebhookName>HookTests` and this class should accept
`WebhookTestCase`. For our HelloWorld webhook, we name the test class
`HelloWorldHookTests`:
```
class HelloWorldHookTests(WebhookTestCase):
STREAM_NAME = 'test'
URL_TEMPLATE = "/api/v1/external/helloworld?&api_key={api_key}"
FIXTURE_DIR_NAME = 'helloworld'
# Note: Include a test function per each distinct message condition your integration supports
def test_hello_message(self):
# type: () -> None
expected_subject = u"Hello World";
expected_message = u"Hello! I am happy to be here! :smile: \nThe Wikipedia featured article for today is **[Marilyn Monroe](https://en.wikipedia.org/wiki/Marilyn_Monroe)**";
# use fixture named helloworld_hello
self.send_and_test_stream_message('hello', expected_subject, expected_message,
content_type="application/x-www-form-urlencoded")
def get_body(self, fixture_name):
# type: (text_type) -> text_type
return self.fixture_data("helloworld", fixture_name, file_type="json")
```
When writing tests for your webook, you'll want to include one test function
(and corresponding fixture) per each distinct message condition that your
integration supports.
If, for example, we added support for sending a goodbye message to our `Hello
World` webook, we would add another test function to `HelloWorldHookTests`
class called something like `test_goodbye_message`:
```
def test_goodbye_message(self):
# type: () -> None
expected_subject = u"Hello World";
expected_message = u"Hello! I am happy to be here! :smile:\nThe Wikipedia featured article for today is **[Goodbye](https://en.wikipedia.org/wiki/Goodbye)**";
# use fixture named helloworld_goodbye
self.send_and_test_stream_message('goodbye', expected_subject, expected_message,
content_type="application/x-www-form-urlencoded")
```
As well as a new fixture `helloworld_goodbye.json` in
`zerver/fixtures/helloworld/`:
```
{
"featured_title":"Goodbye",
"featured_url":"https://en.wikipedia.org/wiki/Goodbye",
}
```
Once you have written some tests, you can run just these new tests from within
the Zulip dev environment with this command:
```
(zulip-venv)vagrant@vagrant-ubuntu-trusty-64:/srv/zulip$
./tools/test-backend zerver.tests.test_hooks.HelloWorldHookTests
```
(Note: You must run the tests from `/srv/zulip` directory.)
You will see some script output and if all the tests have passed, you will see:
```
Running zerver.tests.test_hooks.HelloWorldHookTests.test_hello_message
DONE!
```
#### Step 4: Create documentation
Next, we add end-user documentation for our webhook integration to
`templates/zerver/integrations.html`.
First, add a `div` that displays the logo of your integration and a link to its
documentation:
```
<div class="integration-lozenge integration-helloworld">
<a class="integration-link integration-helloworld" href="#helloworld">
<img class="integration-logo" src="/static/images/integrations/logos/helloworld.png" alt="Hello World logo" />
<span class="integration-label">Hello World</span>
</a>
</div>
```
And second, a div with the usage instructions:
```
<div id="helloworld" class="integration-instructions">
<p>Learn how Zulip integrations work with this simple Hello World example!</p>
<p>The Hello World webhook will use the <code>test<code> stream, which is
created by default in the Zulip dev environment. If you are running
Zulip in production, you should make sure this stream exists.</p>
<p>Next, on your <a href="/#settings" target="_blank">Zulip
settings page</a>, create a Hello World bot. Construct the URL for
the Hello World bot using the API key and stream name:
<code>{{ external_api_uri }}/v1/external/helloworld?api_key=abcdefgh&amp;stream=test</code>
</p>
<p>To trigger a notication using this webhook, use `send_webhook_fixture_message` from the Zulip command line:</p>
<div class="codehilite">
<pre>(zulip-venv)vagrant@vagrant-ubuntu-trusty-64:/srv/zulip$
./manage.py send_webhook_fixture_message \
> --fixture=zerver/fixtures/helloworld/helloworld_hello.json \
> '--url=http://localhost:9991/api/v1/external/helloworld?api_key=<api_key>'</pre>
</div>
<p>Or, use curl:</p>
<div class="codehilite">
<pre>curl -X POST -H "Content-Type: application/json" -d '{ "featured_title":"Marilyn Monroe", "featured_url":"https://en.wikipedia.org/wiki/Marilyn_Monroe" }' http://localhost:9991/api/v1/external/helloworld\?api_key\=<api_key></pre>
</div>
<p><b>Congratulations! You're done!</b><br /> Your messages may look like:</p>
<img class="screenshot" src="/static/images/integrations/helloworld/001.png" />
</div>
```
Both blocks should fall alphabetically so we add these two divs between the
blocks for Github and Hubot, respectively.
See [Documenting your integration](#documenting-your-integration) for further
details, including how to easily create the message screenshot.
#### Step 5: Preparing a pull request to zulip/zulip
When you have finished your webhook integration and are ready for it to be
available in the Zulip product, follow these steps to prepare your pull
request:
1. Run tests including linters and ensure you have addressed any issues they
report. See [Testing](testing.html) for details.
2. Read through [Code styles and conventions](code-style.html) and take a look
through your code to double-check that you've followed Zulip's guidelines.
3. Take a look at your git history to ensure your commits have been clear and
logical (see [Version Control](version-control.html) for tips). If not,
consider revising them with `git rebase --interactive`. For most webhooks,
you'll want to squash your changes into a single commit and include a good,
clear commit message.
4. Push code to your fork.
5. Submit a pull request to zulip/zulip.
If you would like feedback on your integration as you go, feel free to submit
pull requests as you go, prefixing them with `[WIP]`.
## Writing Python script and plugin integrations integrations
For plugin integrations, usually you will need to consult the

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.6 KiB

View File

@ -131,6 +131,12 @@
<span class="integration-label">Freshdesk</span>
</a>
</div>
<div class="integration-lozenge integration-helloworld">
<a class="integration-link integration-helloworld" href="#helloworld">
<img class="integration-logo" src="/static/images/integrations/logos/helloworld.png" alt="Hello World logo" />
<span class="integration-label">Hello World</span>
</a>
</div>
<div class="integration-lozenge integration-git">
<a class="integration-link integration-git" href="#git">
<img class="integration-logo" src="/static/images/integrations/logos/git.png" alt="Git logo" />
@ -1153,6 +1159,38 @@
</div>
<div id="helloworld" class="integration-instructions">
<p>Learn how Zulip integrations work with this simple Hello World example!</p>
<p>The Hello World webhook will use the <code>test</code> stream, which is
created by default in the Zulip dev environment. If you are running
Zulip in production, you should make sure this stream exists.</p>
<p>Next, on your <a href="/#settings" target="_blank">Zulip
settings page</a>, create a Hello World bot. Construct the URL for
the Hello World bot using the API key and stream name:
<code>{{ external_api_uri }}/v1/external/helloworld?api_key=abcdefgh&amp;stream=test</code>
</p>
<p>To trigger a notication using this webhook, use `send_webhook_fixture_message` from the Zulip command line:</p>
<div class="codehilite">
<pre>(zulip-venv)vagrant@vagrant-ubuntu-trusty-64:/srv/zulip$
./manage.py send_webhook_fixture_message \
> --fixture=zerver/fixtures/helloworld/helloworld_hello.json \
> '--url=http://localhost:9991/api/v1/external/helloworld?api_key=&lt;api_key&gt;'</pre>
</div>
<p>Or, use curl:</p>
<div class="codehilite">
<pre>curl -X POST -H "Content-Type: application/json" -d '{ "featured_title":"Marilyn Monroe", "featured_url":"https://en.wikipedia.org/wiki/Marilyn_Monroe" }' http://localhost:9991/api/v1/external/helloworld\?api_key\=&lt;api_key&gt;</pre>
</div>
<p><b>Congratulations! You're done!</b><br /> Your messages may look like:</p>
<img class="screenshot" src="/static/images/integrations/helloworld/001.png" />
</div>
<div id="hubot" class="integration-instructions">

View File

@ -0,0 +1,4 @@
{
"featured_title":"Goodbye",
"featured_url":"https://en.wikipedia.org/wiki/Goodbye",
}

View File

@ -0,0 +1,4 @@
{
"featured_title":"Marilyn Monroe",
"featured_url":"https://en.wikipedia.org/wiki/Marilyn_Monroe",
}

View File

@ -1703,3 +1703,31 @@ class TrelloHookTests(WebhookTestCase):
# type: () -> None
expected_message = u"TomaszKolek renamed the board from Welcome Board to [New name](https://trello.com/b/iqXXzYEj)."
self.send_and_test_stream_message('renaming_board', u"New name.", expected_message)
class HelloWorldHookTests(WebhookTestCase):
STREAM_NAME = 'test'
URL_TEMPLATE = "/api/v1/external/helloworld?&api_key={api_key}"
FIXTURE_DIR_NAME = 'hello'
# Note: Include a test function per each distinct message condition your integration supports
def test_hello_message(self):
# type: () -> None
expected_subject = u"Hello World";
expected_message = u"Hello! I am happy to be here! :smile:\nThe Wikipedia featured article for today is **[Marilyn Monroe](https://en.wikipedia.org/wiki/Marilyn_Monroe)**";
# use fixture named helloworld_hello
self.send_and_test_stream_message('hello', expected_subject, expected_message,
content_type="application/x-www-form-urlencoded")
def test_goodbye_message(self):
# type: () -> None
expected_subject = u"Hello World";
expected_message = u"Hello! I am happy to be here! :smile:\nThe Wikipedia featured article for today is **[Goodbye](https://en.wikipedia.org/wiki/Goodbye)**";
# use fixture named helloworld_goodbye
self.send_and_test_stream_message('goodbye', expected_subject, expected_message,
content_type="application/x-www-form-urlencoded")
def get_body(self, fixture_name):
# type: (text_type) -> text_type
return self.fixture_data("helloworld", fixture_name, file_type="json")

View File

@ -0,0 +1,35 @@
# Webhooks for external integrations.
from __future__ import absolute_import
from django.utils.translation import ugettext as _
from zerver.lib.actions import check_send_message
from zerver.lib.response import json_success, json_error
from zerver.decorator import REQ, has_request_variables, api_key_only_webhook_view
from zerver.lib.validator import check_dict, check_string
from zerver.models import Client, UserProfile
from django.http import HttpRequest, HttpResponse
from six import text_type
from typing import Dict, Any, Iterable, Optional
@api_key_only_webhook_view('HelloWorld')
@has_request_variables
def api_helloworld_webhook(request, user_profile, client,
payload=REQ(argument_type='body'), stream=REQ(default='test'),
topic=REQ(default='Hello World')):
# type: (HttpRequest, UserProfile, Client, Dict[str, Iterable[Dict[str, Any]]], text_type, Optional[text_type]) -> HttpResponse
# construct the body of the message
body = 'Hello! I am happy to be here! :smile:'
# try to add the Wikipedia article of the day
# return appropriate error if not successful
try:
body_template = '\nThe Wikipedia featured article for today is **[{featured_title}]({featured_url})**'
body += body_template.format(**payload)
except KeyError as e:
return json_error(_("Missing key {} in JSON").format(str(e)))
# send the message
check_send_message(user_profile, client, 'stream', [stream], topic, body)
return json_success()

View File

@ -262,6 +262,7 @@ urls += [
url(r'^api/v1/external/desk$', 'zerver.views.webhooks.deskdotcom.api_deskdotcom_webhook'),
url(r'^api/v1/external/freshdesk$', 'zerver.views.webhooks.freshdesk.api_freshdesk_webhook'),
url(r'^api/v1/external/github$', 'zerver.views.webhooks.github.api_github_landing'),
url(r'^api/v1/external/helloworld$', 'zerver.views.webhooks.helloworld.api_helloworld_webhook'),
url(r'^api/v1/external/ifttt$', 'zerver.views.webhooks.ifttt.api_iftt_app_webhook'),
url(r'^api/v1/external/jira$', 'zerver.views.webhooks.jira.api_jira_webhook'),
url(r'^api/v1/external/newrelic$', 'zerver.views.webhooks.newrelic.api_newrelic_webhook'),