Commit Graph

86 Commits

Author SHA1 Message Date
Anders Kaseorg 029e765e20 openapi: Validate real requests and responses, not fictional mocks.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-02-05 19:57:21 -05:00
Anders Kaseorg 5cac872e4b openapi: Get parameters from requestBody too.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-02-05 19:57:21 -05:00
Anders Kaseorg 131b230e2b openapi: Represent OpenAPI parameters with a Parameter class.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-02-05 19:57:21 -05:00
Anders Kaseorg 0cee3bee00 openapi: Remove confusing check for 200 responses.
This error message didn’t make sense for the check as written, and our
OpenAPI document already provides the expected format for our 200
responses.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-02-05 19:57:21 -05:00
Anders Kaseorg 0514f92bdb openapi: Loosen anchor parameter specification to string.
Real requests would not validate against the previous version.  There
seems to be no consistent way to determine whether a string parameter
should be coerced to an integer for validation against an allOf
schema (which works at the level of JSON objects, not strings).

See also https://github.com/python-openapi/openapi-core/issues/698.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-02-05 19:57:21 -05:00
Anders Kaseorg d9f2f23c6a requirements: Upgrade to openapi-core pre-release.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-02-05 19:57:21 -05:00
Anders Kaseorg c8784634e1 openapi: Remove unnecessary cast.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-11-15 17:10:48 -08:00
Lauryn Menard f755fd1a3a api-docs: Check for deprecated at top level of endpoint docs.
Previously, we had checked that deprecated parameters and return
values had been marked as `deprecated: true` in the OpenAPI
documentation and had a description with a deprecated note.

Here we extend that check at the top level to deprecated endpoints.

The backend test that catches a failed assertion for this check
is `test_api_doc_endpoints` in zerver/tests/test_docs.py as that
test checks for a success response all pages linked the sidebar of
the API documentation.
2023-10-25 12:21:53 -07:00
Anders Kaseorg d2faae36c6 openapi: Remove fix_events.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-23 09:07:07 -07:00
Anders Kaseorg 9ba102f3ea openapi: Remove unused display_brief_error parameter.
It’s unused since commit a881918a05
(#24979).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-20 10:05:29 -07:00
Anders Kaseorg f99cce91bf middleware: Send got_request_exception signal for JSON 500 errors.
This is ordinarily emitted by Django at
  https://github.com/django/django/blob/4.2.6/django/core/handlers/exception.py#L139
and received by Sentry at
  https://github.com/getsentry/sentry-python/blob/1.31.0/sentry_sdk/integrations/django/__init__.py#L166

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-04 17:25:20 -07:00
Lauryn Menard 31daef7f79 response: Remove "result: partially_completed" for success responses.
In feature levels 153 and 154, a new value of "partially_completed"
for `result` in a success (HTTP status code 200) was added for two
endpoints that process messages in batches: /api/delete-topic and
/api/mark-all-as-read.

Prior to these changes, `result` was either "success" or "error" for
all responses, which was a useful API invariant to have for clients.

So, here we remove "partially_completed" as a potential value for
"result" in a response. And instead, for the two endpoints noted
above, we return a boolean field "complete" to indicate if the
response successfully deleted/marked as read all the targeted
messages (complete: true) or if only some of the targeted messages
were processed (complete: false).

The "code" field for an error string that was also returned as part
of a partially completed response is removed in these changes as
well.

The web app does not currently use the /api/mark-all-as-read
endpoint, but it does use the /api/delete-topic endpoint, so these
changes update that to check the `complete` boolean instead of the
string value for `result`.
2023-09-18 13:18:24 -07:00
Anders Kaseorg 74d6d76046 ruff: Fix SIM118 Use `k not in d` instead of `k not in d.keys()`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-24 10:39:28 -07:00
Anders Kaseorg 9bb3d15a79 openapi: Switch to new openapi_core validation API.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-19 16:14:59 -07:00
Lalit a686c0cc02 docs: Add documentation for delete emoji endpoint.
This endpoint was previously marked as `intentionally_undocumented`
but that was mistake.

Removed `intentionally_undocumented` and added proper documentation
with valid `python_example` for this Endpoint.

Fixes: #24084
2023-06-28 10:23:47 -07:00
Anders Kaseorg 85681546ce openapi: Convert deprecated Spec.create to Spec.from_dict.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-06-11 13:51:17 -07:00
Anders Kaseorg b5e5728112 coverage: Clean up coverage configuration.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-31 13:53:04 -07:00
Anders Kaseorg 2db9c0bc21 openapi: Remove unused prune_schema_by_type function.
It’s unused since commit a881918a05
(#24979).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-31 13:53:04 -07:00
Anders Kaseorg 0eff862459 openapi: Remove unused get_event_type function.
It’s unused since commit 8108acbdfd
(#16000).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-31 13:53:04 -07:00
Anders Kaseorg 9797de52a0 ruff: Fix RUF010 Use conversion in f-string.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-26 22:09:18 -07:00
Anders Kaseorg 03b3c8522d requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-25 21:20:33 -07:00
Anders Kaseorg a881918a05 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-03 22:39:21 -07:00
Anders Kaseorg df001db1a9 black: Reformat with Black 23.
Black 23 enforces some slightly more specific rules about empty line
counts and redundant parenthesis removal, but the result is still
compatible with Black 22.

(This does not actually upgrade our Python environment to Black 23
yet.)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-02 10:40:13 -08:00
Lauryn Menard c2bcfb52aa api-tests: Reduce error output for `/register` openapi validation.
For descriptive endpoints, such as `/register`, that might raise
Schema Validation errors via `validate_against_openapi_schema`,
omits the OpenAPI schema definition in the error output.

Also omits the error instance definition in the error output
when it is a jsonschema object with over 100 properties. This
means that the test instance for objects, like user settings,
will be printed in the error output, but the test instance for
the entire endpoint will not be printed to the console.

The omitted output can be thousands of lines long making it
difficult to find the initial console output that actually helps
the contributor with debugging.

Adds a section in "Documenting REST API endpoints" about
debugging and understanding these errors that is linked to
in the error console output.
2023-01-17 14:50:42 -08:00
Anders Kaseorg 17300f196c ruff: Fix ISC003 Explicitly concatenated string.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-04 16:25:07 -08:00
Mateusz Mandera a410f6b241 do_mark_all_as_read: Split up the work into batches.
Fixes #15403.
2022-10-27 16:59:54 -07:00
Anders Kaseorg 133c8b16ed requirements: Upgrade openapi-core.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-07 10:03:51 -07:00
Anders Kaseorg 47c5deeccd python: Mark dict parameters with defaults as read-only.
Found by semgrep 0.115 more accurately applying the rule added in
commit 0d6c771baf (#15349).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-06 13:48:28 -07:00
Anders Kaseorg b0ce4f1bce docs: Fix many spelling mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-07 18:51:06 -08:00
Anders Kaseorg 031f4596ab openapi: Use openapi_core ResponseValidator to validate responses.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:34:11 -08:00
Anders Kaseorg 4cd5e0e578 openapi: Fix display_brief_error not to rely on naively_merge_allOf.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:34:11 -08:00
Anders Kaseorg dc18aadeb2 test_classes: Type kwargs for client_get and friends.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-12-17 08:03:52 -08:00
Anders Kaseorg 89277e84df openapi: Remove unused to_python_type function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:53:47 -07:00
Suyash Vardhan Mathur 6fbe7ad61e apidocs: Remove automatically added : at end of response. 2021-08-17 06:25:02 -07:00
Anders Kaseorg 3665deb93a python: Remove unnecessary intermediate lists.
Generated automatically by pyupgrade.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-02 15:53:52 -07:00
Tim Abbott 256091dc15 settings: Merge settings API endpoints.
This API change removes unnecessary complexity from a client that
wants to change a user's personal settings, and also saves developers
from needing to make decisions about what sort of setting something is
at the API level.

We preserve the old settings endpoints as mapping to the same function
as the new one for backwards-compatibility. We delete the
documentation for the old endpoints, though the documentation for the
merged /settings endpoint mentions how to use the old endpoints when
needed.

We migrate all backend tests to the new endpoints, except for
individual tests for each legacy endpoint to verify they still work.

Co-authored-by: sahil839 <sahilbatra839@gmail.com>
2021-07-19 21:14:32 -07:00
Suyash Vardhan Mathur 981e4f8946
openapi: Render all responses of an operation.
Previously, one needed to specifying all the HTTP status
codes that we want to render along with the operation,
but the primary use case just needs the responses of
all the status codes, and not just one.

This commit modifies the Markdown extension to render
all the responses of all status codes of a specified
operation in a loop.
2021-07-13 08:33:43 -07:00
Suyash Vardhan Mathur 7bbafbc881 python_examples: Add auto-rendering of extra imports.
Currently, there is no provision of rendering
additional imports automatically, and those examples
were hardcoded in the templates.

This commit adds an openapi parameter
to store the additional imports required for the endpoint.
Further, it changes code to replace `import zulip`
with the modified imports.
2021-07-06 10:14:46 -07:00
Anders Kaseorg 24c239d991 openapi: Add missing __init__.py; fix type errors hidden by its absence.
The absence of __init__.py was preventing mypy from following any of
the zerver.openapi imports.  These errors were being silenced by
ignore_missing_imports.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-05 12:20:39 -07:00
Suyash Vardhan Mathur 35ba5b8d6f openapi: Add markdown extension for parameter descriptions.
As a goal of a common template, there
is a need for a tool to auto-generate
general description for all parameters
directly from OpenAPI data.

This description is to be stored in
x-response-description field, and this
commit adds a markdown extesion to process the same.
2021-06-22 10:58:43 -07:00
Suyash Vardhan Mathur dbcffebf2c openapi: Add markdown extension for response descriptions.
As a goal of a common template, there
is a need for a tool to auto-generate
general description for all responses
directly from OpenAPI data.

This description is to be stored in
x-response-description field, and this
commit adds a markdown extesion to process the same.
2021-06-22 10:58:43 -07:00
Suyash Vardhan Mathur 246262fb57 openapi: Modify curl example generation logic.
This commit adds support for using the
x-curl-examples-parameters parameter in OpenAPI
data to fetch curl examples configuration. This
also contains any descriptions necessary for each
example, and directly generates all possible
curl examples directly.

A follow-up commit is needed to modify the templates
accordingly.
2021-06-21 11:06:05 -07:00
Suyash Vardhan Mathur 1acee9f516 apidocs: Migrate admin_config out of templates.
Currently, the `admin_config` configuration was
hardcoded in the templates, but as a goal of creating
a common template, we need to move all configurations
outside.

Moved the checking for function and language which need
admin_config out of templates into the code and added a
boolean `x-admin-config` to store whehter the operation
requires admin config.

Also, added the banner for admin access to auto-generate
if admin_config is present, and fixed the admin_config
for endpoints that were earlier missing it in the templates.
2021-06-16 09:52:43 -07:00
Suyash Vardhan Mathur 6692250858 openapi: Enhance markdown extension for responses.
Currently, the descriptions for API responses were
hardcoded in the templates. However, they now exist
in YAML data as well, and so can be fetched from there.

Also, as a part of moving towards a common template, it is
beneficial to show return response description along with
the code example directly. Also, for the same reason, the
need for mentioning subschemas for the response has been
removed, and ALL responses for that response code should
be returned in the proper format.

This also includes returning empty string if that response
code doesn't exist. This allows us to directly mention both
200 and 400 responses in all templates, and they are parsed
automatically.

A followup is necessary to remove the descriptions in the template
as they are duplicated just by this commit
2021-06-09 12:43:12 -07:00
Suyash Vardhan Mathur 52b9c96c5d apidocs: Add support for using OpenAPI title for /api pages.
The current logic of geneerating HTML titles requires the title to be
present as a heading in the first line of .md file. However, this will
shortly be no longer true for /api pages where these are
auto-generated from OpenAPI data.  Modified the code to fetch the
title from OpenAPI data in case of such pages.
2021-05-26 17:38:01 -07:00
Suyash Vardhan Mathur 58a8009786 apidocs: Add markdown processor for title.
Currently, the title of each endpoint are hardcoded in .md
files, but these are also added in summary parameter of
openapi. Added a markdown processor to insert the title
of a given endpoint.
2021-05-26 17:38:01 -07:00
Suyash Vardhan Mathur 96bfeeb9e6 api docs: Expand checking for deprecated fields.
Added assertion to check that if a deprecated flag is in a field's
schema, then it should have deprecated mentioned in description
as well, and moved these checks to a separate function.
Fixes part of #15967.
2021-02-16 15:34:52 -08:00
Anders Kaseorg 6e4c3e41dc python: Normalize quotes with Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 11741543da python: Reformat with Black, except quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Harsh Srivastava 9b31df009b openapi: Fix excessively large test_events failure output.
Because of the very large `oneOf` clause of the formats of events
possible in Zulip's `GET /events` system, we had issues with
`test-backend` failures for missing documentation for a new event
format being like 1000 lines of output, which was very much unhelpful.

Fix this by limiting the output use only the oneOf variants that are
broadly similar to the actual payload received.

Fixes #16023.
2020-10-23 17:00:17 -07:00