zulip/docs/documentation/api.md

# Documenting REST API endpoints

This document explains the system for documenting [Zulip's REST
API](https://zulip.com/api/rest).

Zulip's API documentation is an essential resource both for users and
for the developers of Zulip's mobile and terminal apps. Our vision is
for the documentation to be sufficiently good that developers of
Zulip's apps should never need to look at the server's implementation
to answer questions about the API's semantics.

To achieve these goals, Zulip leverages the popular OpenAPI format as
the data source to ensure that Zulip's API documentation is correct
and remains so as Zulip's API evolves.

In particular, the top goal for this system is that all mistakes in
verifiable content (i.e. not the English explanations) should cause
the Zulip test suite to fail. This is incredibly important, because
once you notice one error in API documentation, you no longer trust it
to be correct, which ends up wasting the time of its users.

Since it's very difficult to not make little mistakes when writing any
untested code, the only good solution to this is a way to test
the documentation. We found dozens of errors in the process of adding
the validation Zulip has today.

Our API documentation is defined by a few sets of files:

- The primary source of our API documentation is the Zulip server's
  [OpenAPI description](../documentation/openapi.md) at
  `zerver/openapi/zulip.yaml`.
- The documentation is written the same Markdown framework that powers
  our [user docs](../documentation/user.md), with some special
  extensions for rendering nice code blocks and example
  responses. Most API endpoints share a common template,
  `templates/zerver/api/api-doc-template.md`, which renders the
  OpenAPI description of the API endpoint. A handful of endpoints that
  require special content, as well as pages that document general API
  details rather than specific endpoints, live at
  `templates/zerver/api/*.md`.
- We have an extensive set of tests designed to validate that the data
  in the OpenAPI file matching the implementation. Specifically,
  `zerver/tests/test_openapi.py` compares every endpoint's accepted
  parameters in `views` code with those declared in `zulip.yaml`. And
  the [backend test suite](../testing/testing-with-django.md) checks
  that every API response served during our extensive backend test
  suite matches one the declared OpenAPI schema for that endpoint.
- The text for the Python examples comes from a test suite for the
  Python API documentation (`zerver/openapi/python_examples.py`; run via
  `tools/test-api`). The `generate_code_example` macro will magically
  read content from that test suite and render it as the code example.
  This structure ensures that Zulip's API documentation is robust to a
  wide range of possible typos and other bugs in the API
  documentation.
- The JavaScript examples are similarly generated and tested using
  `zerver/openapi/javascript_examples.js`.
- The cURL examples are generated and tested using
  `zerver/openapi/curl_param_value_generators.py`.
- The REST API index
  (`templates/zerver/help/include/rest-endpoints.md`) in the broader
  /api left sidebar (`templates/zerver/api/sidebar_index.md`).

This first section is focused on explaining how the API documentation
system is put together; when actually documenting an endpoint, you'll
want to also read the [Step by step guide](#step-by-step-guide).

## How it works

To understand how this documentation system works, start by reading an
existing doc file (`templates/zerver/api/render-message.md` is a good
example; accessible live
[here](https://zulip.com/api/render-message) or in the development
environment at `http://localhost:9991/api/render-message`).

We highly recommend looking at those resources while reading this page.

If you look at the documentation for existing endpoints, you'll notice
that a typical endpoint's documentation is divided into four sections:

- The top-level **Title and description**
- **Usage examples**
- **Arguments**
- **Responses**

The rest of this guide describes how each of these sections works.

### Title and description

Displayed at the top of any REST endpoint documentation page, the
title comes from the `summary` parameter in OpenAPI data. The
description should explain what the endpoint does in clear
English. Include details on how to use it correctly or what it's good
or bad for, with links to any alternative endpoints the user might
want to consider.

These sections should often contain a link to the documentation of the
relevant feature in `/help/`.

### Usage examples

We display usage examples in three languages: Python, JavaScript and
`curl`; we may add more in the future. Every endpoint should have
Python and `curl` documentation; `JavaScript` is optional as we don't
consider that API library to be fully supported. The examples are
defined using a special Markdown extension
(`zerver/openapi/markdown_extension.py`). To use this extension, one
writes a Markdown file block that looks something like this:

```md
{start_tabs}
{tab|python}

{generate_code_example(python)|/messages/render:post|example}

{tab|js}
...

{tab|curl}

{generate_code_example(curl)|/messages/render:post|example}

{end_tabs}
```

In some cases, one wants to configure specific parameters to be
included or excluded from the example `curl` requests for readability
reasons. One can do that using the `x-curl-examples-parameters`
parameter.

#### Writing Python examples

For the Python examples, you'll write the example in
`zerver/openapi/python_examples.py`, and it'll be run and verified
automatically in Zulip's automated test suite. The code there will
look something like this:

``` python
@openapi_test_function('/messages/render:post')
def render_message(client: Client) -> None:
    # {code_example|start}
    # Render a message
    request = {
        'content': '**foo**'
    }
    result = client.render_message(request)
    # {code_example|end}

    validate_against_openapi_schema(result, '/messages/render', 'post', '200')
```

This is an actual Python function which will be run as part of the
`tools/test-api` test suite. The `validate_against_opanapi_schema`
function will verify that the result of that request is as defined in
the examples in `zerver/openapi/zulip.yaml`.

To run as part of the test suite, the `render_message` function needs
to be called from `test_messages` (or one of the other functions at
the bottom of the file). The final function, `test_the_api`, is what
actually runs the tests. Tests with the `openapi_test_function`
decorator that are not called will fail tests, as will new endpoints
that are not covered by an `openapi_test_function`-decorated test.

You will still want to manually test the example using Zulip's Python
API client by copy-pasting from the website; it's easy to make typos
and other mistakes where variables are defined outside the tested
block, and the tests are not foolproof.

The code that renders `/api` pages will extract the block between the
`# {code_example|start}` and `# {code_example|end}` comments, and
substitute it in place of
`{generate_code_example(python)|/messages/render:post|example}`
wherever that string appears in the API documentation.

- Additional Python imports can be added using the custom
  `x-python-examples-extra-imports` field in the OpenAPI definition.
- Endpoints that only administrators can use should be tagged with the
  custom `x-requires-administrator` field in the OpenAPI definition.

### Parameters

We have a separate Markdown extension to document the parameters that
an API endpoint supports. You'll see this in files like
`templates/zerver/api/render-message.md` via the following Markdown
directive (implemented in
`zerver/lib/markdown/api_arguments_table_generator.py`):

```md
{generate_api_arguments_table|zulip.yaml|/messages/render:post}
```

Just as in the usage examples, the `/messages/render` key must match a
URL definition in `zerver/openapi/zulip.yaml`, and that URL definition
must have a `post` HTTP method defined.

Additional content that you'd like to appear in the parameter
description area can be declared using the custom
`x-parameter-description` field in the OpenAPI definition.

### Displaying example payloads/responses

If you've already followed the steps in the [Usage examples](#usage-examples)
section, this part should be fairly trivial.

You can use the following Markdown directive to render all the fixtures
defined in the OpenAPI `zulip.yaml` for a given endpoint

```md
{generate_code_example|/messages/render:post|fixture}
```

Additional content that you'd like to appear in the responses part of
the page can be added using the custom `x-response-description` field
in the OpenAPI definition.

## Step by step guide

This section offers a step-by-step process for adding documentation
for a new API endpoint. It assumes you've read and understood the
above.

1. Start by adding [OpenAPI format](../documentation/openapi.md)
   data to `zerver/openapi/zulip.yaml` for the endpoint. If you
   copy-paste (which is helpful to get the indentation structure
   right), be sure to update all the content that you copied to
   correctly describe your endpoint!

   In order to do this, you need to figure out how the endpoint in
   question works by reading the code! To understand how arguments
   are specified in Zulip backend endpoints, read our [REST API
   tutorial][rest-api-tutorial], paying special attention to the
   details of `REQ` and `has_request_variables`.

   Once you understand that, the best way to determine the supported
   arguments for an API endpoint is to find the corresponding URL
   pattern in `zprojects/urls.py`, look up the backend function for
   that endpoint in `zerver/views/`, and inspect its arguments
   declared using `REQ`.

   You can check your formatting using these helpful tools.
   - `tools/check-openapi` will verify the syntax of `zerver/openapi/zulip.yaml`.
   - `tools/test-backend zerver/tests/test_openapi.py`; this test compares
      your documentation against the code and can find many common
      mistakes in how arguments are declared.
   - `test-backend`: The full Zulip backend test suite will fail if
     any actual API responses generated by the tests don't match your
     defined OpenAPI schema. Use `test-backend --rerun` for a fast
     edit/refresh cycle when debugging.

   [rest-api-tutorial]: ../tutorials/writing-views.html#writing-api-rest-endpoints

1. Add a function for the endpoint you'd like to document to
   `zerver/openapi/python_examples.py`, decorated with
   `@openapi_test_function`. `render_message` is a good example to
   follow. There are generally two key pieces to your test: (1) doing
   an API query and (2) verifying its result has the expected format
   using `validate_against_openapi_schema`.

1. Make the desired API call inside the function. If our Python
   bindings don't have a dedicated method for a specific API call,
   you may either use `client.call_endpoint` or add a dedicated
   function to the [zulip PyPI
   package](https://github.com/zulip/python-zulip-api/tree/main/zulip).
   Ultimately, the goal is for every endpoint to be documented the
   latter way, but it's useful to be able to write working
   documentation for an endpoint that isn't supported by
   `python-zulip-api` yet.

1. Add the function to one of the `test_*` functions at the end of
   `zerver/openapi/python_examples.py`; this will ensure your
   function will be called when running `test-api`.

1. Capture the JSON response returned by the API call (the test
   "fixture"). The easiest way to do this is add an appropriate print
   statement (usually `json.dumps(result, indent=4, sort_keys=True)`),
   and then run `tools/test-api`. You can also use
   <https://jsonformatter.curiousconcept.com/> to format the JSON
   fixtures. Add the fixture to the `example` subsection of the
   `responses` section for the endpoint in
   `zerver/openapi/zulip.yaml`.

1. Run `./tools/test-api` to make sure your new test function is being
   run and the tests pass.

1. Now, inside the function, isolate the lines of code that call the API and could
   be displayed as a code example. Wrap the relevant lines in
   `# {code_example|start} ... relevant lines go here ... # {code_example|end}`
   comments. The lines inside these comments are what will be displayed as the
   code example on our `/api` page.

1. Finally, if the API docs page of the endpoint doesn't follow the
   common API docs template in
   `templates/zerver/api/api-docs-template.md`, then add its custom
   Markdown file under `templates/zerver/api/`. However, it is a goal
   to minimize the number of files that diverse from the common
   template, so only do this if there's a good reason.

1. Add the endpoint to the index in
   `templates/zerver/help/include/rest-endpoints.md`. The URL should
   match the `operationId` for the endpoint, and the link text should
   match the title of the endpoint from the OpenAPI `summary` field.

1. Test your endpoint, pretending to be a new user in a hurry, by
   visiting it via the links on `http://localhost:9991/api` (the API
   docs are rendered from the Markdown source files on page load, so
   just reload to see an updated version as you edit). You should
   make sure that copy-pasting the code in your examples works, and
   post an example of the output in the pull request.

1. Document the new API in `templates/zerver/api/changelog.md` and
   bump the `API_FEATURE_LEVEL` in `version.py`. Also, make sure to
   add a `**Changes**` entry in the description of the new API/event
   in `zerver/openapi/zulip.yaml`, which mentions the API feature level
   at which they were added.

[javascript-examples]: https://github.com/zulip/zulip-js/tree/main/examples

## Why a custom system?

Given that our documentation is written in large part using the
OpenAPI format, why maintain a custom Markdown system for displaying
it? There's several major benefits to this system:

- It is extremely common for API documentation to become out of date
  as an API evolves; this automated testing system helps make it
  possible for Zulip to maintain accurate documentation without a lot
  of manual management.
- Every Zulip server can host correct API documentation for its
  version, with the key variables (like the Zulip server URL) already
  pre-substituted for the user.
- We're able to share implementation language and visual styling with
  our Help Center, which is especially useful for the extensive
  non-REST API documentation pages (e.g. our bot framework).

Using the standard OpenAPI format gives us flexibility, though; if we
later choose to migrate to third-party tools, we don't need to redo
the actual documentation work in order to migrate tools.