Commit Graph

259 Commits

Author SHA1 Message Date
orientor c7a2c48bb7 openapi: Add tests for checking presence of various attributes.
The swagger validator is a basic tool to check whether our
openapi specification file follows the basic syntax. But to ensure
that our zulip.yaml file is not only syntactically compatible but
also describes our API well, we need to add custom tests. This
commit currently checks whether each endpoint has an `operationId`
and a valid tag. It also makes it easier to check for custom rules
in the future.
2020-05-20 13:10:58 -07:00
Rohitt Vashishtha 1e4ad0bdfc js-api: Migrate and test send-message example. 2020-05-20 10:18:29 -07:00
Rohitt Vashishtha 203b722624 js-api: Migrate and test create-user example. 2020-05-20 10:18:29 -07:00
Kartik Srivastava 5f3849b517 openapi/markdown_extension: Avoid shelling out to node for JS snippets.
This refactors render_javascript_code_example to avoid shelling out to
node and parse the javascript file with python instead, to get example
code snippets.
2020-05-20 10:18:29 -07:00
Rohitt Vashishtha 6dd4030e67 js-api: Add hooks to run tests and render docs of JS API examples.
This commit adds python code to call javascript_examples.js in its
two supported modes. tools/test-api asserts that the example output
is as expected, whereas the API markdown extension is used to render
these examples in the docs.
2020-05-20 10:18:29 -07:00
Kartik Srivastava 642d1a20d0 openapi/markdown_extension: Refactor extract_code_example.
This refactors `extract_code_example` to return a nested list
of code snippets between '{code_example|start/end}' instead of
returing a list of all the lines between '{code_example|start/end}'
markers in the code examples.

Appropriate changes have been made to render_python_code_example.
2020-05-20 10:18:29 -07:00
Kartik Srivastava 302906211d js-api: Refactor ExamplesHandler to avoid running examples in a loop.
This refactors `ExamplesHandler` to avoid running examples in a loop
and add result objects to `response_data` array one by one with
`generate_validation_data`.
2020-05-20 10:18:29 -07:00
Rohitt Vashishtha 31d04fb370 js-api: Add module to create and run JS API examples.
This file will act as the container for all JS API examples to use
in our documentation, similar to our python and curl API testing
and examples generation code.

This module has two modes of operation:

- node javascript_examples.js generate-responses

   This mode runs all the examples against a server and prints the JSON
   output of all the examples we ran.

- node javascript_examples.js generate-example <endpoint>

   This mode prints example code for endpoints like: /users:post. We then
   want to render this full example code in our docs.
2020-05-20 10:18:29 -07:00
orientor 915d801327 openapi: Add 'tags' attribute to endpoints.
'tags' attribute is helpful in differentiating and grouping the
endpoints on basis of their usage. For example tags like 'messages'
help in grouping all endpoints related to messages and thus make the
api specification more user-friendly. So give tags to the endpoints
on the basis of what heading they are under in the API docs.
2020-05-19 23:17:31 -07:00
orientor 3467d2fe68 openapi: Add 'operationId' attribute to endpoints.
'operationId' helps code generators in naming functions and other purposes.
So name operationId of endpoints as their function names in python-zulip-api
if it exists else use most appropriate function name.

Part of #14100 .
2020-05-19 23:13:39 -07:00
Kartik Srivastava 59ac3ca95f openapi/python_examples: Add reactivate_user example. 2020-05-19 23:10:50 -07:00
Kartik Srivastava 4af7ad70e7 openapi: Add OpenAPI data for /users/{user_id}/reactivate. 2020-05-19 23:10:50 -07:00
sahil839 9b78a73e36 populate_db: Add new admin user as 'Desdemona'.
This commit adds a second admin user named 'Desdemona' to dev and
test database.
2020-05-19 11:42:27 -07:00
Kartik Srivastava 112f539034 openapi/python_examples: Update update_user example.
This updates the `update_user` example to use `update_user_by_id`.
2020-05-16 14:45:47 -07:00
Kartik Srivastava d2ab5673fd openapi/python_examples: Update deactivate_user example.
This updates the `deactivate_user` example to use
`deactivate_user_by_id`.
2020-05-16 14:45:47 -07:00
orientor 9170931da3 openapi: Add test for validating examples.
Zulip's openapi specification in zulip.yaml has various examples
for various schemas. Validate the example with their respective
schemas to ensure that all the examples are schematically correct.

Part of #14100.
2020-05-12 23:03:06 -07:00
orientor fab2ec9e63 openapi: Rectify wrong examples and schemas.
Some examples mentioned in zulip.yaml did not match their schema.
Change either the schema or the example so that all examples are
valid with respect to their schemas.
2020-05-12 23:03:06 -07:00
orientor 3ffc9466c9 openapi: Combine two similar openapi markdown extensions.
Previously api_description and api_code_examples were two independent
markdown extensions for displaying OpenAPI content used in the same
places.  We combine them into a single markdown extension (with two
processors) and move them to the openapi folder to make the codebase
more readable and better group the openapi code in the same place.
2020-05-05 21:42:31 -07:00
Tim Abbott 5ea942caff api docs: Improve a few common OpenAPI descriptions.
This fixes a few regressions of documentation introduced earlier in
this branch.
2020-05-05 17:11:14 -07:00
shubhamgupta2956 57c38f1c79 api_docs: Add "EmojiCode" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "emoji_code" in components.
2020-05-05 17:02:56 -07:00
shubhamgupta2956 08314e3bf4 api_docs: Add "ReactionType" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "reaction_type" in components.
2020-05-05 17:02:56 -07:00
shubhamgupta2956 db3d95fd16 api_docs: Add "Principals" component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "principals" in components.
2020-05-05 17:02:56 -07:00
shubhamgupta2956 c4ebe98bba api_docs: Add IncludeCustomProfileFields component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "include_custom_profile_fields" in
components.
2020-05-05 17:02:53 -07:00
shubhamgupta2956 c28ccacb73 api_docs: Add "EmojiName" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "emoji_name" in components.
2020-05-05 17:02:53 -07:00
shubhamgupta2956 64ef886eb9 api_docs: Add "IncludeSubscribers" component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "IncludeSubscribers" in components.
2020-05-05 17:02:53 -07:00
shubhamgupta2956 597269c43e api_docs: Add HistoryPublicToSubscribers component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "history_public_to_subscribers" in
components.
2020-05-05 17:02:51 -07:00
shubhamgupta2956 c5bef400bb api_docs: Add "StreamPostPolicy" component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "stream_post_policy" in components.
2020-05-05 16:48:33 -07:00
shubhamgupta2956 b0c7daf7e5 api_docs: Add "UserId" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "user_id" in components.
2020-05-05 16:48:33 -07:00
Hashir Sarwar 55a8e7dff2 settings: Offer hiding presence info from other users.
For privacy-minded folks who don't want to leak the
information of whether they're online, this adds an
option to disable sending presence updates to other
users.

The new settings lies in the "Other notification
settings" section of the "Notification settings"
page, under a "Presence" subheading.

Closes #14798.
2020-05-02 15:09:05 -07:00
Steve Howell 2c63130195 api: Always return zulip_version/zulip_feature_level.
We no longer make these conditional to simplify writing clients;
there's no cost to including them, and a real cost to not doing so.
2020-04-29 11:58:22 -07:00
Tim Abbott e8c57f2466 api docs: Document API level 2 change in custom emoji. 2020-04-28 22:23:25 -07:00
Tim Abbott cbe0723199 api docs: Add an API changelog page.
I imagine this can be improved in various ways, but I've initialized
this with all the **Changes** entries recorded in either zulip.yaml or
the rest of the API documentation, and I expect we'll be able to
iterate on this effectively.

It'll also be useful as a record of changes that we should remember to
document the API documentation as we document more endpoints that
currently don't discuss these issues.

While working on this, I fixed various issues where feature levels
could be mentioned or endpoints didn't properly document changes.
2020-04-28 22:23:25 -07:00
Tim Abbott 1be4cc5f17 openapi: Fix servers section to work in online editor. 2020-04-28 14:27:43 -07:00
orientor 4d8c988ef2 openapi: Use description markdown for rendering endpoint descriptions.
Firstly, change endpoint descriptions in zulip.yaml so that they
match their counterpart in the api docs. Then edit the api docs
so that they use api description markdown extension for displaying
endpoint description.
2020-04-28 12:57:19 -07:00
orientor 64c6bab276 openapi: Create markdown extension for rendering endpoint descriptions.
Add function in openapi.py to access endpoint descriptions written
in zulip.yaml. Use this function for creating a markdown extension
for rendering endpoint descriptions written in zulip.yaml.

We use this extension for a single endpoint to get test coverage.
2020-04-28 12:57:19 -07:00
Eeshan Garg 75b2264a3f pypi: Upgrade zulip/zulip-bots dependencies to version 0.7.0.
Includes this change:
* openapi/python_examples: Update get_single_user.

This updates get_single_user to pass keyword arguments to
get_user_by_id instead of passing a dictionary.

Which is required for CI to pass, as we indeed fixed the API of that
function (which had only been present with the wrong API for one release).
2020-04-23 17:41:47 -07:00
Anders Kaseorg fead14951c python: Convert assignment type annotations to Python 3.6 style.
This commit was split by tabbott; this piece covers the vast majority
of files in Zulip, but excludes scripts/, tools/, and puppet/ to help
ensure we at least show the right error messages for Xenial systems.

We can likely further refine the remaining pieces with some testing.

Generated by com2ann, with whitespace fixes and various manual fixes
for runtime issues:

-    invoiced_through: Optional[LicenseLedger] = models.ForeignKey(
+    invoiced_through: Optional["LicenseLedger"] = models.ForeignKey(

-_apns_client: Optional[APNsClient] = None
+_apns_client: Optional["APNsClient"] = None

-    notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
-    signup_notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
+    notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
+    signup_notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)

-    author: Optional[UserProfile] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE)
+    author: Optional["UserProfile"] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE)

-    bot_owner: Optional[UserProfile] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL)
+    bot_owner: Optional["UserProfile"] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL)

-    default_sending_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
-    default_events_register_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
+    default_sending_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
+    default_events_register_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)

-descriptors_by_handler_id: Dict[int, ClientDescriptor] = {}
+descriptors_by_handler_id: Dict[int, "ClientDescriptor"] = {}

-worker_classes: Dict[str, Type[QueueProcessingWorker]] = {}
-queues: Dict[str, Dict[str, Type[QueueProcessingWorker]]] = {}
+worker_classes: Dict[str, Type["QueueProcessingWorker"]] = {}
+queues: Dict[str, Dict[str, Type["QueueProcessingWorker"]]] = {}

-AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional[LDAPSearch] = None
+AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional["LDAPSearch"] = None

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-22 11:02:32 -07:00
Hashir Sarwar e3b90a5ec8 api: Add a monotonic integer "feature level" for non-webapp clients.
The purpose is to provide a way for (non-webapp) clients,
like the mobile and terminal apps, to tell whether the
server it's talking to is new enough to support a given
API feature -- in particular a way that

* is finer-grained than release numbers, so that for
features developed after e.g. 2.1.0 we can use them
immediately on servers deployed from master (like
chat.zulip.org and zulipchat.com) without waiting the
months until a 2.2 release;

* is reliable, unlike e.g. looking at the number of
commits since a release;

* doesn't lead to a growing bag of named feature flags
which the server has to go on sending forever.

Tweaked by tabbott to extend the documentation.

Closes #14618.
2020-04-21 13:37:57 -07:00
Steve Howell 105c168b19 test-api: Avoid flakes with curl tests. 2020-04-21 11:30:29 -04:00
shubhamgupta2956 7c3169a1d9 api_docs: Add "MessageId" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "message_id" in components.
2020-04-20 17:52:49 -07:00
shubhamgupta2956 05bb1e7075 api_docs: Add "Content" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "content" in components.
2020-04-20 17:52:49 -07:00
shubhamgupta2956 0425f53e88 api_docs: Add "ClientGravatar" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "client_gravatar" in components.
2020-04-20 17:52:49 -07:00
shubhamgupta2956 c814afd4a7 api_docs: Add "StreamIdInPath" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "stream_id" (in path) in components.
2020-04-20 17:52:48 -07:00
shubhamgupta2956 942fd39065 api_docs: Add "StreamIdInQuery" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "stream_id" (in query) in components.
2020-04-20 17:52:30 -07:00
shubhamgupta2956 d5f96c3c12 api_docs: Add "Stream" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "stream" in components.
2020-04-20 17:49:53 -07:00
shubhamgupta2956 22c273eacd api_docs: Add "QueueId" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "queue_id" in components.
2020-04-20 17:49:53 -07:00
shubhamgupta2956 86eed5d9bb api_docs: Add "Topic" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "topic" in components.
2020-04-20 17:49:52 -07:00
shubhamgupta2956 05b1ddeb84 api_docs: Add "GroupId" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "group_id" in components.
2020-04-20 17:47:13 -07:00
shubhamgupta2956 8e617d7f26 api_docs: Add "Narrow" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "narrow" in components.
2020-04-20 17:47:13 -07:00
Anders Kaseorg 5901e7ba7e python: Convert function type annotations to Python 3 style.
Generated by com2ann (slightly patched to avoid also converting
assignment type annotations, which require Python 3.6), followed by
some manual whitespace adjustment, and six fixes for runtime issues:

-    def __init__(self, token: Token, parent: Optional[Node]) -> None:
+    def __init__(self, token: Token, parent: "Optional[Node]") -> None:

-def main(options: argparse.Namespace) -> NoReturn:
+def main(options: argparse.Namespace) -> "NoReturn":

-def fetch_request(url: str, callback: Any, **kwargs: Any) -> Generator[Callable[..., Any], Any, None]:
+def fetch_request(url: str, callback: Any, **kwargs: Any) -> "Generator[Callable[..., Any], Any, None]":

-def assert_server_running(server: subprocess.Popen[bytes], log_file: Optional[str]) -> None:
+def assert_server_running(server: "subprocess.Popen[bytes]", log_file: Optional[str]) -> None:

-def server_is_up(server: subprocess.Popen[bytes], log_file: Optional[str]) -> bool:
+def server_is_up(server: "subprocess.Popen[bytes]", log_file: Optional[str]) -> bool:

-    method_kwarg_pairs: List[FuncKwargPair],
+    method_kwarg_pairs: "List[FuncKwargPair]",

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-18 20:42:48 -07:00