markdown: Rewrite include plugin without markdown-include.

markdown-include is GPL licensed.

Also, rewrite it as a block processor, so that it works correctly
inside indented blocks.

Signed-off-by: Anders Kaseorg <anders@zulip.com>

docs/documentation/integrations.md

@@ -76,7 +76,7 @@ Here are a few common macros used to document Zulip's integrations:
   integration or incoming webhook. For an example rendering, see **Step 1** of
   [the docs for Zulip's GitHub integration][github-integration].
 
-- `{!create-bot-construct-url-indented.md!}` macro - Instructs users to create a bot
+- `{!create-bot-construct-url.md!}` macro - Instructs users to create a bot
   for a given integration and construct a webhook URL using the bot API key
   and stream name. The URL is generated automatically for every incoming webhook
   by using attributes in the `WebhookIntegration` class in
@@ -94,12 +94,12 @@ Here are a few common macros used to document Zulip's integrations:
 - `{!append-stream-name.md!}` macro - Recommends appending `&stream=stream_name`
   to a URL in cases where supplying a stream name in the URL is optional.
   Supplying a stream name is optional for most Zulip integrations. If you use
-  `{!create-bot-construct-url-indented.md!}`, this macro need not be used.
+  `{!create-bot-construct-url.md!}`, this macro need not be used.
 
 - `{!append-topic.md!}` macro - Recommends appending `&topic=my_topic` to a URL
   to supply a custom topic for webhook notification messages. Supplying a custom
   topic is optional for most Zulip integrations. If you use
-  `{!create-bot-construct-url-indented.md!}`, this macro need not be used.
+  `{!create-bot-construct-url.md!}`, this macro need not be used.
 
 - `{!congrats.md!}` macro - Inserts congratulatory lines signifying the
   successful setup of a given integration. This macro is usually used at
@@ -127,7 +127,7 @@ Here are a few common macros used to document Zulip's integrations:
   see the last paragraph of **Step 2** in
   [the docs for Zulip's GitHub integration][github-integration].
 
-- `{!webhook-url.md!}` - Used internally by `{!create-bot-construct-url-indented.md!}`
+- `{!webhook-url.md!}` - Used internally by `{!create-bot-construct-url.md!}`
   to generate the webhook URL.
 
 - `{!zulip-config.md!}` - Used internally by `{!change-zulip-config-file.md!}`
@@ -202,7 +202,7 @@ A typical doc will then have the following steps.
 
 ##### "Create the bot" step
 
-- Typically, use the `create-bot-construct-url-indented` macro.
+- Typically, use the `create-bot-construct-url` macro.
 - [Existing macros](#markdown-macros) should be used for this if they exist, but if the macro
   defaults don’t work, it may make sense to write something custom for the
   integration in question. This step is mandatory for all integrations.

pyproject.toml

@@ -69,7 +69,6 @@ module = [
     "gitlint.*",
     "jsonref.*",
     "ldap.*",
-    "markdown_include.*",
     "moto.*", # https://github.com/spulec/moto/issues/4944
     "onelogin.*",
     "openapi_core.*",

requirements/common.in

@@ -61,9 +61,6 @@ premailer
 # Needed for JWT-based auth
 PyJWT
 
-# Needed for including other Markdown files for user docs
-markdown-include
-
 # Needed to access RabbitMQ
 pika
 

requirements/dev.txt

@@ -902,11 +902,6 @@ lxml-stubs==0.4.0 \
 markdown==3.3.7 \
     --hash=sha256:cbb516f16218e643d8e0a95b309f77eb118cb138d39a4f27851e6a63581db874 \
     --hash=sha256:f5da449a6e1c989a4cea2631aa8ee67caa5a2ef855d551c88f9e309f4634c621
-    # via
-    #   -r requirements/common.in
-    #   markdown-include
-markdown-include==0.6.0 \
-    --hash=sha256:6f5d680e36f7780c7f0f61dca53ca581bd50d1b56137ddcd6353efafa0c3e4a2
     # via -r requirements/common.in
 markdown-it-py==2.1.0 \
     --hash=sha256:93de681e5c021a432c63147656fe21790bc01231e0cd2da73626f1aa3ac0fe27 \

requirements/prod.txt

@@ -584,11 +584,6 @@ lxml==4.6.5 \
 markdown==3.3.7 \
     --hash=sha256:cbb516f16218e643d8e0a95b309f77eb118cb138d39a4f27851e6a63581db874 \
     --hash=sha256:f5da449a6e1c989a4cea2631aa8ee67caa5a2ef855d551c88f9e309f4634c621
-    # via
-    #   -r requirements/common.in
-    #   markdown-include
-markdown-include==0.6.0 \
-    --hash=sha256:6f5d680e36f7780c7f0f61dca53ca581bd50d1b56137ddcd6353efafa0c3e4a2
     # via -r requirements/common.in
 markupsafe==2.1.1 \
     --hash=sha256:0212a68688482dc52b2d45013df70d169f542b7394fc744c02a57374a4207003 \

templates/zerver/help/include/change-zulip-config-file-indented.md

@@ -1,9 +0,0 @@
-    Open `/usr/local/share/zulip/integrations/{{ integration_name }}/zulip_{{ integration_name }}_config.py`
-    with your favorite editor, and change the following lines to specify the
-    email address and API key for your {{ integration_display_name }} bot:
-
-    ```
-    ZULIP_USER = "{{ integration_name }}-bot@example.com"
-    ZULIP_API_KEY = "0123456789abcdef0123456789abcdef"
-    ZULIP_SITE = "{{ api_url }}"
-    ```

templates/zerver/help/include/change-zulip-config-file.md

@@ -1,9 +1,9 @@
-On your {{ settings_html|safe }}, create a bot for
-{{ integration_display_name }}.
-
-Next, open
-`/usr/local/share/zulip/integrations/{{ integration_name }}/zulip_{{ integration_name }}_config.py`
+Open `/usr/local/share/zulip/integrations/{{ integration_name }}/zulip_{{ integration_name }}_config.py`
 with your favorite editor, and change the following lines to specify the
 email address and API key for your {{ integration_display_name }} bot:
 
-{!zulip-config.md!}
+```
+ZULIP_USER = "{{ integration_name }}-bot@example.com"
+ZULIP_API_KEY = "0123456789abcdef0123456789abcdef"
+ZULIP_SITE = "{{ api_url }}"
+```

templates/zerver/help/include/create-a-bot-indented.md

@@ -1,5 +0,0 @@
-    [Create a bot](/help/add-a-bot-or-integration) for
-    {{ integration_display_name }}. Make sure that you select
-    **Incoming webhook** as the **Bot type**:
-
-    ![](/static/images/integrations/bot_types.png)

templates/zerver/help/include/create-bot-construct-url-indented.md

@@ -1,15 +0,0 @@
-    {!create-a-bot-indented.md!}
-
-    Construct the URL for the {{ integration_display_name }}
-    bot using the bot's API key and the desired stream name:
-
-    {!webhook-url.md!}
-
-    Modify the parameters of the URL above, where `api_key` is the API key
-    of your Zulip bot, and `stream` is the [URL-encoded](https://www.urlencoder.org/)
-    stream name you want the notifications sent to. If you do not specify a
-    `stream`, the bot will send notifications via PMs to the creator of the bot.
-
-    If you'd like this integration to always send to a specific topic,
-    just include the (URL-encoded) topic as an additional parameter
-    (E.g. for `your topic`, append `&topic=your%20topic` to the URL).

templates/zerver/help/include/git-webhook-url-with-branches-indented.md

@@ -1,4 +0,0 @@
-    You can also limit the branches you receive notifications for by
-    specifying them in a comma-separated list at the end of the URL,
-    like so:
-    `{{ api_url }}{{ integration_url }}?api_key=abcdefgh&stream={{ recommended_stream_name }}&branches=main,development`

templates/zerver/help/include/git-webhook-url-with-branches.md

@@ -1,5 +1,4 @@
 You can also limit the branches you receive notifications for by
 specifying them in a comma-separated list at the end of the URL,
 like so:
-
 `{{ api_url }}{{ integration_url }}?api_key=abcdefgh&stream={{ recommended_stream_name }}&branches=main,development`

templates/zerver/help/include/webhook-url-with-bot-email-indented.md

@@ -1,9 +0,0 @@
-    Construct the URL for the {{ integration_display_name }}
-    bot using the bot's API key and email address:
-
-    `{{ external_uri_scheme }}bot_email:bot_api_key@{{ api_url_scheme_relative }}{{ integration_url }}`
-
-    Modify the parameters of the URL above, where `bot_email` is
-    the bot's URL-encoded email address and `bot_api_key` is the
-    bot's API key.  To URL-encode the email address, you just need
-    to replace `@` in the bot's email address with `%40`.

templates/zerver/help/include/webhook-url-with-bot-email.md

@@ -1,7 +1,9 @@
-In the URL field, enter:
+Construct the URL for the {{ integration_display_name }}
+bot using the bot's API key and email address:
 
 `{{ external_uri_scheme }}bot_email:bot_api_key@{{ api_url_scheme_relative }}{{ integration_url }}`
 
-Replace `bot_email` and `bot_api_key` above with the URL-encoded email
-address and API key of the bot.  To URL-encode the email address, you
-just need to replace `@` in the bot's email address with `%40`.
+Modify the parameters of the URL above, where `bot_email` is
+the bot's URL-encoded email address and `bot_api_key` is the
+bot's API key.  To URL-encode the email address, you just need
+to replace `@` in the bot's email address with `%40`.

templates/zerver/integrations/capistrano.md

@@ -1,6 +1,6 @@
 Get Zulip notifications for your Capistrano deploys!
 
-1.  {!create-a-bot-indented.md!}
+1.  {!create-a-bot.md!}
 
 1.  {!download-python-bindings.md!}
 

templates/zerver/integrations/codebase.md

@@ -5,11 +5,11 @@ Get Codebase notifications in Zulip!
    updates. We recommend naming the streams `codebase` and `tickets`, respectively.
    After creating these streams, make sure to subscribe all interested parties.
 
-1. {!create-a-bot-indented.md!}
+1. {!create-a-bot.md!}
 
 1. {!download-python-bindings.md!}
 
-1. {!change-zulip-config-file-indented.md!}
+1. {!change-zulip-config-file.md!}
 
     You may also need to update the value of `ZULIP_TICKETS_STREAM_NAME` and
     `ZULIP_COMMITS_STREAM_NAME`.

templates/zerver/integrations/git.md

@@ -1,12 +1,12 @@
 Get Zulip notifications for your Git repositories!
 
-1. {!create-a-bot-indented.md!}
+1. {!create-a-bot.md!}
 
 1. {!download-python-bindings.md!}
 
 1. {!create-stream.md!}
 
-1. {!change-zulip-config-file-indented.md!}
+1. {!change-zulip-config-file.md!}
 
     You may also need to change the value of `STREAM_NAME`.
 

templates/zerver/integrations/jenkins.md

@@ -1,6 +1,6 @@
 1. {!create-stream.md!}
 
-1. {!create-a-bot-indented.md!}
+1. {!create-a-bot.md!}
 
 1. Install the "Zulip" plugin by going to
    **Manage Jenkins > Manage Plugins > Available**,

templates/zerver/integrations/openshift.md

@@ -16,7 +16,7 @@ in an OpenShift instance.
 
     ![Connecting to application](/static/images/integrations/openshift/002.png)
 
-1.  {!change-zulip-config-file-indented.md!}
+1.  {!change-zulip-config-file.md!}
 
 1.  You can also specify which pushes will result in notifications and to
     what stream the notifications will be sent by modifying the

templates/zerver/integrations/perforce.md

@@ -8,7 +8,7 @@ that fires once a changelist is submitted and committed.
 1.  The Perforce trigger will be installed to a location like
     `/usr/local/share/zulip/integrations/perforce`.
 
-1.  {!change-zulip-config-file-indented.md!}
+1.  {!change-zulip-config-file.md!}
 
 1.  If you have a P4Web viewer set up, you may change `P4_WEB`
     to point at the base URL of the server. If this is configured,

templates/zerver/integrations/svn.md

@@ -11,7 +11,7 @@ post-commit hook. To do this:
 
    [1]: http://pysvn.tigris.org/project_downloads.html
 
-1. {!change-zulip-config-file-indented.md!}
+1. {!change-zulip-config-file.md!}
 
 1. Copy `integrations/svn/zulip_svn_config.py` and
    `integrations/svn/post-commit` from the API bindings directory

templates/zerver/integrations/trac.md

@@ -2,7 +2,7 @@
 
 1.  {!download-python-bindings.md!}
 
-1.  {!change-zulip-config-file-indented.md!}
+1.  {!change-zulip-config-file.md!}
 
 1.  Also, change the following lines:
 

version.py

@@ -48,4 +48,4 @@ API_FEATURE_LEVEL = 132
 #   historical commits sharing the same major version, in which case a
 #   minor version bump suffices.
 
-PROVISION_VERSION = "191.1"
+PROVISION_VERSION = "192.0"

zerver/lib/markdown/__init__.py

@@ -2191,6 +2191,7 @@ class Markdown(markdown.Markdown):
         parser.blockprocessors.register(OListProcessor(parser), "olist", 65)
         parser.blockprocessors.register(UListProcessor(parser), "ulist", 60)
         parser.blockprocessors.register(BlockQuoteProcessor(parser), "quote", 55)
+        # We get priority 51 from our 'include' extension
         parser.blockprocessors.register(
             markdown.blockprocessors.ParagraphProcessor(parser), "paragraph", 50
         )

zerver/lib/markdown/include.py

@@ -1,70 +1,54 @@
 import os
 import re
-from typing import Any, List
+from typing import List, Match
+from xml.etree.ElementTree import Element
 
-import markdown
-from markdown_include.include import IncludePreprocessor, MarkdownInclude
+from markdown import Extension, Markdown
+from markdown.blockparser import BlockParser
+from markdown.blockprocessors import BlockProcessor
 
 from zerver.lib.exceptions import InvalidMarkdownIncludeStatement
-from zerver.lib.markdown.priorities import PREPROCESSOR_PRIORITES
-
-INC_SYNTAX = re.compile(r"\{!\s*(.+?)\s*!\}")
+from zerver.lib.markdown.priorities import BLOCK_PROCESSOR_PRIORITIES
 
 
-class MarkdownIncludeCustom(MarkdownInclude):
-    def extendMarkdown(self, md: markdown.Markdown) -> None:
-        md.preprocessors.register(
-            IncludeCustomPreprocessor(md, self.getConfigs()),
-            "include_wrapper",
-            PREPROCESSOR_PRIORITES["include"],
+class IncludeExtension(Extension):
+    def __init__(self, base_path: str) -> None:
+        super().__init__()
+        self.base_path = base_path
+
+    def extendMarkdown(self, md: Markdown) -> None:
+        md.parser.blockprocessors.register(
+            IncludeBlockProcessor(md.parser, self.base_path),
+            "include",
+            BLOCK_PROCESSOR_PRIORITIES["include"],
         )
 
 
-class IncludeCustomPreprocessor(IncludePreprocessor):
-    """
-    This is a custom implementation of the markdown_include
-    extension that checks for include statements and if the included
-    macro file does not exist or can't be opened, raises a custom
-    JsonableError exception. The rest of the functionality is identical
-    to the original markdown_include extension.
-    """
+class IncludeBlockProcessor(BlockProcessor):
+    RE = re.compile(r"^ {,3}\{!([^!]+)!\} *$", re.M)
 
-    def run(self, lines: List[str]) -> List[str]:
-        done = False
-        while not done:
-            for line in lines:
-                loc = lines.index(line)
-                m = INC_SYNTAX.search(line)
+    def __init__(self, parser: BlockParser, base_path: str) -> None:
+        super().__init__(parser)
+        self.base_path = base_path
 
-                if m:
-                    filename = m.group(1)
-                    filename = os.path.expanduser(filename)
-                    if not os.path.isabs(filename):
-                        filename = os.path.normpath(
-                            os.path.join(self.base_path, filename),
-                        )
-                    try:
-                        with open(filename, encoding=self.encoding) as r:
-                            text = r.readlines()
-                    except Exception as e:
-                        print(f"Warning: could not find file {filename}. Error: {e}")
-                        lines[loc] = INC_SYNTAX.sub("", line)
-                        raise InvalidMarkdownIncludeStatement(m.group(0).strip())
+    def test(self, parent: Element, block: str) -> bool:  # type: ignore[override] # https://github.com/python/typeshed/pull/8166
+        return bool(self.RE.search(block))
 
-                    line_split = INC_SYNTAX.split(line)
-                    if len(text) == 0:
-                        text.append("")
-                    for i in range(len(text)):
-                        text[i] = text[i].rstrip("\r\n")
-                    text[0] = line_split[0] + text[0]
-                    text[-1] = text[-1] + line_split[2]
-                    lines = lines[:loc] + text + lines[loc + 1 :]
-                    break
-            else:
-                done = True
+    def expand_include(self, m: Match[str]) -> str:
+        try:
+            with open(os.path.normpath(os.path.join(self.base_path, m[1]))) as f:
+                lines = f.read().splitlines()
+        except OSError as e:
+            raise InvalidMarkdownIncludeStatement(m[0].strip()) from e
 
-        return lines
+        for prep in self.parser.md.preprocessors:
+            lines = prep.run(lines)
+
+        return "\n".join(lines)
+
+    def run(self, parent: Element, blocks: List[str]) -> None:
+        blocks[:1] = self.RE.sub(self.expand_include, blocks[0]).split("\n\n")
 
 
-def makeExtension(*args: Any, **kwargs: str) -> MarkdownIncludeCustom:
-    return MarkdownIncludeCustom(kwargs)
+def makeExtension(base_path: str) -> IncludeExtension:
+    return IncludeExtension(base_path=base_path)

zerver/lib/markdown/priorities.py

@@ -10,8 +10,6 @@ PREPROCESSOR_PRIORITES = {
     "generate_code_example": 525,
     "generate_return_values": 510,
     "generate_api_arguments": 505,
-    "include": 500,
-    # "include_wrapper": 500,
     "help_relative_links": 475,
     "setting": 450,
     # "normalize_whitespace": 30,
@@ -21,3 +19,7 @@ PREPROCESSOR_PRIORITES = {
     "nested_code_blocks": -500,
     "emoticon_translations": -505,
 }
+
+BLOCK_PROCESSOR_PRIORITIES = {
+    "include": 51,
+}

zerver/tests/test_templates.py

@@ -1,5 +1,3 @@
-from unittest.mock import MagicMock, call, patch
-
 from django.template.loader import get_template
 
 from zerver.lib.exceptions import InvalidMarkdownIncludeStatement
@@ -118,8 +116,7 @@ footer
         )
         self.assertEqual(content_sans_whitespace, expected)
 
-    @patch("builtins.print")
-    def test_custom_markdown_include_extension(self, mock_print: MagicMock) -> None:
+    def test_custom_markdown_include_extension(self) -> None:
         template = get_template("tests/test_markdown.html")
         context = {
             "markdown_test_file": "zerver/tests/markdown/test_custom_include_extension.md",
@@ -129,14 +126,6 @@ footer
             InvalidMarkdownIncludeStatement, "Invalid Markdown include statement"
         ):
             template.render(context)
-        self.assertEqual(
-            mock_print.mock_calls,
-            [
-                call(
-                    "Warning: could not find file templates/zerver/help/include/nonexistent-macro.md. Error: [Errno 2] No such file or directory: 'templates/zerver/help/include/nonexistent-macro.md'"
-                )
-            ],
-        )
 
     def test_custom_markdown_include_extension_empty_macro(self) -> None:
         template = get_template("tests/test_markdown.html")

zerver/webhooks/airbrake/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications for your Airbrake bug tracker!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your project's settings on the Airbrake site. Click on the
    **Integration** section, and select **Webhook**.

zerver/webhooks/alertmanager/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications from Alertmanager!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
     Additionally, you may specify URL parameters named `name` and `desc` to specify which labels
     or annotations will be used to construct the alert message. This allows you to use arbitrary labels

zerver/webhooks/ansibletower/doc.md

@@ -2,7 +2,7 @@ Get Ansible Tower notifications in Zulip!
 
  1. {!create-stream.md!}
 
- 1. {!create-bot-construct-url-indented.md!}
+ 1. {!create-bot-construct-url.md!}
 
  1. Go to your Ansible Tower or AWX Admin Portal. Click **Notifications** on
     the left sidebar, and click **Add**.

zerver/webhooks/appfollow/doc.md

@@ -3,7 +3,7 @@ using the Zulip AppFollow integration!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. In AppFollow, click on **Integrations** on the left sidebar, and click **Add new integration**.
 Click on your app under **Tracked apps**.

zerver/webhooks/appveyor/doc.md

@@ -2,7 +2,7 @@ Receive AppVeyor notifications in Zulip!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your AppVeyor Project Settings, and click **Notifications**.
    Click **Add notification** and select **Webhook**.

zerver/webhooks/basecamp/doc.md

@@ -3,7 +3,7 @@ events in Basecamp.
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your project on Basecamp. From the **Settings** menu in
    the top right corner, choose **Set up webhooks**. Click on

zerver/webhooks/beanstalk/doc.md

@@ -2,9 +2,9 @@ Zulip supports both SVN and Git notifications from Beanstalk.
 
 1. {!create-stream.md!}
 
-1. {!create-a-bot-indented.md!}
+1. {!create-a-bot.md!}
 
-   {!webhook-url-with-bot-email-indented.md!}
+   {!webhook-url-with-bot-email.md!}
    {!git-append-branches.md!}
 
 1. On your repository's webpage, click on the **Settings**

zerver/webhooks/beeminder/doc.md

@@ -3,7 +3,7 @@ to derail from your goal!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your Beeminder **Reminders** settings.
 

zerver/webhooks/bitbucket2/doc.md

@@ -8,8 +8,8 @@ Bitbucket Server click [here](./bitbucket3).
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
-   {!git-webhook-url-with-branches-indented.md!}
+1. {!create-bot-construct-url.md!}
+   {!git-webhook-url-with-branches.md!}
 
 1. On your repository's web page, click on **Settings**. Select
    **Webhooks**, and click **Add webhook**.

zerver/webhooks/bitbucket3/doc.md

@@ -8,8 +8,8 @@ Bitbucket Cloud (SAAS service) click [here](./bitbucket2).
 
 1. {!create-stream.md!}
 
-2. {!create-bot-construct-url-indented.md!}
-   {!git-webhook-url-with-branches-indented.md!}
+2. {!create-bot-construct-url.md!}
+   {!git-webhook-url-with-branches.md!}
 
 3. On your repository's web page, click on **Settings**. Select
    **Webhooks**, and click **Add webhook**.

zerver/webhooks/buildbot/doc.md

@@ -6,7 +6,7 @@ Get Zulip notifications for your Buildbot builds!
 
 1. {!create-stream.md!}
 
-1. {!create-a-bot-indented.md!}
+1. {!create-a-bot.md!}
 
 1. Edit the Buildbot configuration file to add a new Zulip reporter
  ([or follow the steps listed here][1]):

zerver/webhooks/canarytoken/doc.md

@@ -4,7 +4,7 @@ See your Thinkst Canarytoken alerts in Zulip! This integration works with Canary
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to [canarytokens.org][canarytokens], and select the type of
    webhook that you want to create and add a reminder note.

zerver/webhooks/circleci/doc.md

@@ -3,7 +3,7 @@ your build statuses.
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Add the following to the bottom of your `circle.yml` file:
 

zerver/webhooks/clubhouse/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications for your Clubhouse Stories and Epics!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your Clubhouse Dashboard, and click on the settings icon in
    the top-right corner. Click on **Integrations**, and select **Webhooks**.

zerver/webhooks/codeship/doc.md

@@ -3,7 +3,7 @@ your build statuses.
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. On your project's webpage, click on **Project Settings** in
    the top right corner. Click on the **Notifications** tab, and click on

zerver/webhooks/crashlytics/doc.md

@@ -3,7 +3,7 @@ about Crashlytics issues.
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Select the app you'd like to be notified about in your
    [Crashlytics settings panel](https://fabric.io/settings/apps).

zerver/webhooks/delighted/doc.md

@@ -3,7 +3,7 @@ about updates in feedback responses organized by Delighted.
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. On your Delighted dashboard, click on **Settings** in the
    top-right corner. Click on **Integrations**. Scroll down

zerver/webhooks/dropbox/doc.md

@@ -2,7 +2,7 @@ Get Dropbox notifications in Zulip!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your [Dropbox apps page](https://www.dropbox.com/developers/apps).
    Click on **Create app** near the top-right corner, and follow the on-screen

zerver/webhooks/errbit/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications for the Errbit error tracker!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your project's settings on the Errbit site. Click on the
    **Edit** button for your Errbit app, and select **Webhook**.

zerver/webhooks/flock/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications from your Flock channels.
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Click on **Apps** in the bottom-right corner.
    Click on **Admin Panel**, and click on **Webhooks**.

zerver/webhooks/freshdesk/doc.md

@@ -3,7 +3,7 @@ integration!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 ### Add notifications for new Freshdesk tickets
 

zerver/webhooks/freshping/doc.md

@@ -2,7 +2,7 @@ Receive Freshping notifications in Zulip!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your **Freshping** dashboard and click **Settings**.
    Click **Integrations** and then click **Create Integration**

zerver/webhooks/freshstatus/doc.md

@@ -2,7 +2,7 @@ Receive Freshstatus notifications in Zulip!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. On your Freshstatus dashboard, click **Settings** and click **Integrations**.
    Go to **Webhooks** and click **Manage**. Click **New Webhook**

zerver/webhooks/front/doc.md

@@ -4,7 +4,7 @@ to receive Front notifications without leaving Zulip!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to the **Settings** page of your Front organization. Click on the
 **Integrations** tab, and enable the **Webhooks** integration. Click on

zerver/webhooks/gci/doc.md

@@ -4,7 +4,7 @@ you can now get Task notifications in Zulip!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Send an email to `gci-support@google.com` asking them to enable webhooks
    for your organization. They'll need the name of your organization, and

zerver/webhooks/gitea/doc.md

@@ -2,9 +2,9 @@ Receive Gitea notifications in Zulip!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
-   {!git-webhook-url-with-branches-indented.md!}
+   {!git-webhook-url-with-branches.md!}
 
 1. Go to your repository on Gitea and click on **Settings**. Select
    **Webhooks** on the left sidebar, and click **Add Webhook**.

zerver/webhooks/github/doc.md

@@ -2,9 +2,9 @@ Get GitHub notifications in Zulip!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
-   {!git-webhook-url-with-branches-indented.md!}
+   {!git-webhook-url-with-branches.md!}
 
 1. Go to your repository on GitHub and click on the **Settings** tab.
    Select **Webhooks**. Click on **Add webhook**. GitHub may prompt

zerver/webhooks/gitlab/doc.md

@@ -2,13 +2,13 @@ Receive GitLab notifications in Zulip!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
     By default, the Zulip topics for merge requests will contain the
     title of the GitLab merge request.  You can change the topic format to
     just contain the merge request ID by adding
     `&use_merge_request_title=false` at the end of the URL.
-    {!git-webhook-url-with-branches-indented.md!}
+    {!git-webhook-url-with-branches.md!}
 
 1. Go to your repository on GitLab and click **Settings** on the left
    sidebar.  Click on **Integrations**.

zerver/webhooks/gocd/doc.md

@@ -3,7 +3,7 @@ your build statuses.
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Add the following to your `Config.XML` file.
 

zerver/webhooks/gogs/doc.md

@@ -2,9 +2,9 @@ Receive Gogs notifications in Zulip!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
-   {!git-webhook-url-with-branches-indented.md!}
+   {!git-webhook-url-with-branches.md!}
 
 1. Go to your repository on Gogs and click on **Settings**. Select
    **Webhooks** on the left sidebar, and click **Add Webhook**.

zerver/webhooks/gosquared/doc.md

@@ -2,7 +2,7 @@ Receive GoSquared notifications in Zulip!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your project's **Settings** and click on **Services**.
    Scroll down and next to **Webhook**, click on **Connect**. Click

zerver/webhooks/grafana/doc.md

@@ -2,7 +2,7 @@ See your Grafana dashboard alerts in Zulip!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. In Grafana, go to **Alerting**. Click on **Notification channels**.
    Configure **Edit Notification Channel** as appropriate for your

zerver/webhooks/greenhouse/doc.md

@@ -2,7 +2,7 @@ Receive Greenhouse notifications in Zulip!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. On your Greenhouse **Dashboard**, click on the
    **gear** (<i class="fa fa-cog"></i>) icon in the upper right

zerver/webhooks/groove/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications for your Groove events!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. On your Groove dashboard, click on **Settings**. Under **Company**,
    click on **API**. Open the **Add Webhook** dropdown and select an

zerver/webhooks/harbor/doc.md

@@ -4,7 +4,7 @@ Harbor's webhooks feature is available in version 1.9 and later.
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your Harbor **Projects** page. Open a project and click on the **Webhooks** tab.
 

zerver/webhooks/hellosign/doc.md

@@ -2,7 +2,7 @@ Configuring the HelloSign integration is easy!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your HelloSign account **Settings**. Select **Integrations** and
    click on **API**.

zerver/webhooks/heroku/doc.md

@@ -3,7 +3,7 @@ is pushed to Heroku using the Zulip Heroku plugin!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your project on Heroku
    and click the **Resources** tab. Add the **Deploy Hooks** add-on.

zerver/webhooks/ifttt/doc.md

@@ -5,7 +5,7 @@ in Zulip.
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Create an [IFTTT account](https://ifttt.com/join). Select the service you'd like
    to receive notifications from as `this`. Select **Webhooks** as

zerver/webhooks/insping/doc.md

@@ -4,7 +4,7 @@ a website or service is up and running or down.
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. On your Insping **Dashboard**, click on **Integrations**. Click
    on **Create Webhook**.

zerver/webhooks/intercom/doc.md

@@ -2,7 +2,7 @@ Get Intercom notifications in Zulip!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your Intercom account's **Settings** page and click on **Developers**
    on the left sidebar. Click on **Developer Hub** and click **New app**.

zerver/webhooks/jira/doc.md

@@ -5,7 +5,7 @@ These instructions apply to Atlassian Cloud's hosted Jira, and Jira Server versi
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your Jira **Site administration** page. Click **Jira** on the left.
    On the left sidebar, scroll down, and under **Advanced**, click **WebHooks**.

zerver/webhooks/jotform/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications for your Jotform responses!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
    By default, the integration will use the form's title as the topic.
 
 1. In Jotform, go to **Form Builder**, and click on **Settings** tab.

zerver/webhooks/json/doc.md

@@ -5,7 +5,7 @@ integration](/api/incoming-webhooks-overview).
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1.  Configure your application to send the webhook
     payload to the **URL** generated above.

zerver/webhooks/librato/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications for your Librato/AppOptics alerts!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
    By default, the integration will use the name of the alert.
 
 1. Go to your AppOptics/Librato homepage, and click on **Settings**

zerver/webhooks/lidarr/doc.md

@@ -2,7 +2,7 @@ Receive Lidarr notifications in Zulip!
 
 1.  {!create-stream.md!}
 
-1.  {!create-bot-construct-url-indented.md!}
+1.  {!create-bot-construct-url.md!}
 
 1.  Go to your Lidarr dashboard. Click **Settings** and
     click **Connect**. Click the **+** icon.

zerver/webhooks/mention/doc.md

@@ -2,7 +2,7 @@ Get Mention notifications within Zulip via Zapier!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your Mention feed, and click on your profile in the top-right
    corner. Select **Settings**. Click on **Integrations**. Select the

zerver/webhooks/netlify/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications for your Netlify deployments!
 
 1. {!create-stream.md!}
 
-2. {!create-bot-construct-url-indented.md!}
+2. {!create-bot-construct-url.md!}
 
 3. Go to your Netlify project, and click **Settings**. Click **Build & deploy**, and select **Deploy notifications**.
    Click **Add Notification**, and select **Outgoing webhook**.

zerver/webhooks/newrelic/doc.md

@@ -2,7 +2,7 @@ New Relic can send messages to a Zulip stream for incidents.
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. On [New Relic](https://one.newrelic.com),
   select **Alerts & AI**.

zerver/webhooks/opbeat/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications for your Opbeat events!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your Opbeat **Organization Settings**.
 

zerver/webhooks/opencollective/doc.md

@@ -3,7 +3,7 @@ when a new member signs-up on an **Open Collective** page.
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to [Open Collective Website](https://opencollective.com/), find
 your desired collective page, then go to *Settings* -> *Webhooks*, paste the

zerver/webhooks/opsgenie/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications for your Opsgenie events!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to <https://app.opsgenie.com/integration>. Click on
    **Add New Integrations**, and select **Webhook**.

zerver/webhooks/pagerduty/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications for your PagerDuty services!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Open the **Integrations** tab, and click **Generic Webhooks (v3)**.
 

zerver/webhooks/papertrail/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications for your Papertrail logs!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. On your Papertrail dashboard, search for the logs you'd like
    to set up alerts for, and click on **Save Search**. Provide a

zerver/webhooks/pingdom/doc.md

@@ -3,7 +3,7 @@ uptime status changes from your Pingdom dashboard.
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1.  Next, go to the following URL:
 

zerver/webhooks/pivotal/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications for the stories in your Pivotal Tracker project!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your Pivotal project's **Settings** page, and click on **Webhooks**.
 

zerver/webhooks/radarr/doc.md

@@ -2,7 +2,7 @@ Receive Radarr notifications in Zulip!
 
 1.  {!create-stream.md!}
 
-1.  {!create-bot-construct-url-indented.md!}
+1.  {!create-bot-construct-url.md!}
 
 1.  Go to your Radarr dashboard. Click **Settings** and click **Connect**.
     Click the **+** icon.

zerver/webhooks/raygun/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications for your Raygun events!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your Raygun dashboard, and click on **Integrations**.
    Click on **Webhook**, and click on the **Setup** tab.

zerver/webhooks/reviewboard/doc.md

@@ -2,7 +2,7 @@ Get Review Board notifications in Zulip!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. On your Review Board **Dashboard**, click your team's name in the top-right
    corner, and click **Team administration**. Select **WebHooks** on the

zerver/webhooks/semaphore/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications for your Semaphore builds!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. In Semaphore 2.0, under **Configuration** select **Notifications**. Click on
    **Create New Notification**. Add the the URL constructed above to the Webhook

zerver/webhooks/sentry/doc.md

@@ -6,7 +6,7 @@ us](/help/contact-support) if a platform you care about is missing.
 
 1. {!create-stream.md!}
 
-2. {!create-bot-construct-url-indented.md!}
+2. {!create-bot-construct-url.md!}
 
     The default topic, if not set in the URL, will be the title of the
     issue or event.

zerver/webhooks/slack/doc.md

@@ -5,7 +5,7 @@ See also the [Slack-compatible webhook](/integrations/doc/slack_incoming).
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
     If you'd like to map Slack channels to different topics within the same
     stream, add `&channels_map_to_topics=1` to the end of the URL. The `topic`

zerver/webhooks/slack_incoming/doc.md

@@ -4,7 +4,7 @@ API](https://api.slack.com/messaging/webhooks).
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Use your new webhook URL any place that you would use a Slack webhook.
 

zerver/webhooks/solano/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications for your Solano CI builds!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your Solano CI **Organizations** page, and click on
    **Organization Settings**. Click on **Web Hooks**.

zerver/webhooks/sonarqube/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications for your Sonarqube code analysis!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. To configure webhooks for a specific SonarQube project, go to the project and select **Administration**. Select
  **Webhooks** and click **Create**. **Note**: you can also configure webhooks globally by going to **Configurations** ->

zerver/webhooks/sonarr/doc.md

@@ -2,7 +2,7 @@ Receive Sonarr notifications in Zulip!
 
 1.  {!create-stream.md!}
 
-1.  {!create-bot-construct-url-indented.md!}
+1.  {!create-bot-construct-url.md!}
 
 1.  Go to your Sonarr dashboard. Click **Settings** and click **Connect**.
     Click the **+** icon.

zerver/webhooks/splunk/doc.md

@@ -2,7 +2,7 @@ See your Splunk Search alerts in Zulip!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
     If you do not specify a topic, the name of the search is used
     (truncated to fit if needed).

zerver/webhooks/statuspage/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications for your Statuspage.io subscriptions!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your Statuspage Dashboard, and click on **Notifications**
    near the bottom-left corner. Select the **Webhook** tab. If webhook

zerver/webhooks/stripe/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications for Stripe events!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. On your Stripe Dashboard, click on **Developers** on the left
    sidebar. Click on **Webhooks**, and click on **+ Add endpoint**.

zerver/webhooks/taiga/doc.md

@@ -2,7 +2,7 @@ Receive Zulip notifications for your Taiga projects!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
     Make sure to specify the topic in the URL above. Otherwise, the
     default topic `General` will be used.

zerver/webhooks/teamcity/doc.md

@@ -2,7 +2,7 @@ Get Zulip notifications for your TeamCity builds!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Install the [tcWebHooks plugin](https://github.com/tcplugins/tcWebHooks/releases)
    onto your TeamCity server. Follow the plugin instructions in your

zerver/webhooks/thinkst/doc.md

@@ -4,7 +4,7 @@ Canarytokens from Thinkst's paid product, not [canarytokens.org][canarytokens] -
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. Go to your Thinkst Canary settings, and click on **Webhooks** on
    the left sidebar. Select the **Generic** tab. Press

zerver/webhooks/transifex/doc.md

@@ -2,7 +2,7 @@ Get Transifex notifications in Zulip!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. On your Transifex Dashboard, select your project, and click on
    **Settings**. Click on the **Webhooks** tab, and click

zerver/webhooks/travis/doc.md

@@ -2,7 +2,7 @@ See your Travis CI build notifications in Zulip!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
    By default, pull request events are ignored since most people
    don't want notifications for new pushes to pull requests.  To
    enable notifications for pull request builds, just

zerver/webhooks/trello/doc.md

@@ -9,7 +9,7 @@ Get Zulip notifications from your Trello boards!
 
 1. {!create-stream.md!}
 
-1. {!create-bot-construct-url-indented.md!}
+1. {!create-bot-construct-url.md!}
 
 1. **Log in to Trello**, and collect the following three items: