Commit Graph

675 Commits

Author SHA1 Message Date
Dinesh 27e4f5da92 typing: Support sending stream/topic typing status.
This extends the /json/typing endpoint to also accept
stream_id and topic. With this change, the requests
sent to /json/typing should have these:
* `to`: a list set to
    - recipients for a PM
    - stream_id for a stream message
* `topic`, in case of stream message
along with `op`(start or stop).

On receiving a request with stream_id and topic, we send
typing events to clients with stream_typing_notifications set
to True for all users subscribed to that stream.
2021-04-27 20:52:21 -07:00
Dinesh 734d935d4a client_capabilities: Add stream_typing_notifications. 2021-04-27 20:52:21 -07:00
Tim Abbott 2bdbbbd234 api docs: Add design details for muted users.
And also link to those details from the documentation.

We may later (partially?) move this content to Help Center
documentation on the feature.
2021-04-27 18:10:39 -07:00
Anders Kaseorg e3f2ffa681 docs: Capitalize “Markdown” consistently.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-26 09:31:08 -07:00
Anders Kaseorg 178736c8eb docs: Fix spelling errors caught by codespell.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-26 09:31:08 -07:00
Mateusz Mandera 977a2f7fa0 emails: Rename "missed message email" to "message notification email". 2021-04-21 10:10:54 -07:00
akshatdalton 6509c4f8f4 linkifiers: Add an API to support the editing of linkifier.
This commit adds an API to `zproject/urls.py` to edit/update
the realm linkifier. Its helper function to update the
database is added in `zerver/lib/actions.py`.

`zulip.yaml` is documented accordingly as well, clearly
stating that this API updates one linkifier at a time.

The tests are added for the API and helper function which
updates the realm linkifier.

Fixes #10830.
2021-04-19 18:01:45 -07:00
akshatdalton 5bbfa24d74 Refactor: Rename `Filter` to `Linkifier`.
Linkifier error message: `Filter not found` is
updated to `Linkifier not found.`.

Similarly, `filter_id` description is updated to:
`The ID of the linkifier that you want to remove.`,
renamed the term `filter` with `linkifier`, in `zulip.yaml`.
2021-04-19 18:01:45 -07:00
Tim Abbott 3810eaaa25 api docs: Document that servers ignore unexpected event types.
Clients like the mobile and terminal apps will want to rely on this
detail of our API, so we should write it down.
2021-04-19 14:14:14 -07:00
Sumanth V Rao f9b79999ed playgrounds: Avoid json encoding each field in POST /realm/playgrounds.
This prevents us from having to json encode every field in the POST
request to /realm/playgrounds, and keeps the client logic simpler
when adding a playground.
2021-04-19 11:23:09 -07:00
sahil839 2dc99aa90f settings: Add new setting for controlling who can move msgs to stream.
This commit adds a new setting 'move_messages_between_streams_policy`
for controlling who can move messages from one stream to other.
2021-04-16 15:10:39 -07:00
Aman Agrawal 802c450b3f realm: Add setting to configure GIPHY rating.
Organization admins can use this setting to restrict the maximum
rating of GIFs that will be retrieved from GIPHY. Also, there
is option to disable GIPHY too.
2021-04-14 10:29:39 -07:00
Tim Abbott c30a4c5466 settings: Standardize on "video call provider" (not "video chat").
We had a mix of the two names, and "video call provider" both feels
more professional and more clear about precisely what it does.

We don't change the API fields, since it doesn't seem worth an API
migration.
2021-04-14 08:48:21 -07:00
akshatdalton 53f123df72 linkifiers: Extract shared parameter in `zulip.yaml`.
In zulip.yaml, `pattern` and `url_format_string` are extracted
as a shared parameters.

This is done as a preparatory commit for `Add settings UI to edit
linkifiers`.

Related issue: #10830.
2021-04-14 01:37:00 -07:00
100RABHpy 2ead0fa824 openapi: Refactor the way we find uncalled curl test functions.
In "test_curl_examples.py" we find the functions that registered but
never called. To improve readablity, now we have the full
implementation in curl_param_value_generators, rather than inspecting
its fields from another module.
2021-04-13 18:21:38 -07:00
Abhijeet Prasad Bodas 3947b0c80a linkifiers: Update API to send data using dictionaries.
* This introduces a new event type `realm_linkifiers` and
a new key for the initial data fetch of the same name.
Newer clients will be expected to use these.

* Backwards compatibility is ensured by changing neither
the current event nor the /register key. The data which
these hold is the same as before, but internally, it is
generated by processing the `realm_linkifiers` data.
We send both the old and the new event types to clients
whenever the linkifiers are changed.
Older clients will simply ignore the new event type, and
vice versa.

* The `realm/filters:GET` endpoint (which returns tuples)
is currently used by none of the official Zulip clients.
This commit replaces it with `realm/linkifiers:GET` which
returns data in the new dictionary format.
TODO: Update the `get_realm_filters` method in the API
bindings, to hit this new URL instead of the old one.

* This also updates the webapp frontend to use the newer
events and keys.
2021-04-13 12:16:07 -07:00
Zeeshan Equbal 2da4443cc5
api: Add max_message_length field to API data.
Commit 4a3ad0d introduced some extra stream-level parameters
to the `realm` object. This commit extends that to add a
max_message_length paramter too in the same server_level.
2021-04-12 16:03:31 -07:00
Tim Abbott 4a3ad0da06 api: Improve encoding of stream/topic max field lengths.
Previously, you had to request the `stream` event type in order to get
the stream-level parameters; this was a bad design in part because the
`subscription` event type has similar data and is preferred by most
clients.

So we move these to the `realm` object.  We also add the maximum topic
length, as an adjacent parameter.

While changing this, we also fix these to better match the names of
similar API parameters.
2021-04-10 10:07:57 -07:00
Tim Abbott 33a732708e api docs: Record surprising required encoding for emojiset.
This fixes the API documentation tests following recent changes, but
isn't the right solution -- we probably want to change the API itself
to not require this strange JSON-encoding-of-a-string.

But this is necessary to have CI pass.
2021-04-07 12:37:51 -07:00
m-e-l-u-h-a-n 24800fca12 openapi: Add links to display settings in register documentation.
This may let us spend a bit more effort documenting these in one
place, like adding links.
2021-04-07 12:10:35 -07:00
m-e-l-u-h-a-n 079f016a35 api docs: Document display settings endpoint.
With substantial changes by tabbott to fix bugs, clarify the
documentation, and add links.

Fixes #16048.
2021-04-07 12:10:25 -07:00
Tim Abbott c6f27440c6 openapi: Fix display for boolean example values.
The comments explain in some detail, but basically we were displaying
the types for booleans incorrectly, and the types for strings in a
somewhat confusing fashion.  Fix this with comments explaining the logic.

Using JSON dumping also results in our showing strings inside
quotation marks in our examples, which seems net helpful.

Thanks to ArunSankarKs for finding where we needed to change the
codebase.

Fixes #18021.
2021-04-07 09:52:42 -07:00
Sumanth V Rao 75e539b7a7 openapi: Add feature level information for playground APIs/events.
We missed this when adding the new API endpoints/events for
configuring realm "playground" options in the series starting
with 251b415987.
2021-04-07 09:27:36 -07:00
sahil839 86f7b6af60 events: Pass can_invite_others_to_realm to client using page_params.
This commit adds backend code for passing can_invite_others_to_realm
field to clients using the fetch_initial_state_data in the page_params
object.

Though this field is not used by webapp as of now, but will be used
to fix a bug of incorreclty showing the invite users option in
settings overlay in the next commit.
2021-04-07 09:05:16 -07:00
sahil839 554ce5bf0d realm: Add moderators and full members option in invite_to_realm_policy.
We add moderators and full members option to invite_to_realm_policy
by using COMMON_POLICY_TYPES and use can_invite_others_to_realm helper
added in previous commit. This commit only does the backend work,
frontend work will be done in separate commit.
2021-04-07 09:05:16 -07:00
sahil839 4c8339fa8c settings: Replace invite_by_admins_policy with invite_to_realm_policy.
This commit replaces invite_by_admins_policy, which was a bool field,
with a new enum field invite_by_realm_policy.

Though the final goal is to add moderators and full members option
using COMMON_POLICY_TYPES, but this will be done in a separate
commit to make this easy for review.
2021-04-07 09:02:33 -07:00
Tim Abbott 54999f629c openapi: Fix missing example for typing notifications parameter. 2021-04-07 01:14:10 -07:00
Dinesh cd62b40f20 typing: Refactor view to support message_type.
For extending typing notifications support for streams.
2021-04-07 00:17:30 -07:00
Sumanth V Rao 1ac8fe7538 events/tests/api: Send realm_playground events to clients.
We send the whole data set as a part of the event rather than
doing an add/remove operation for couple of reasons:
    * This would make the client logic simpler.
    * The playground data is small enough for us to not worry
      about performance.

Tweaked both `fetch_initial_state_data` and `apply_events` to
handle the new playground event.

Tests added to validate the event matches the expected schema.

Documented realm_playgrounds sections inside /events and
/register to support our openapi validation system in test_events.

Tweaked other tests like test_event_system.py and test_home.py
to account for the new event being generated.

Lastly, documented the changes to the API endpoints in
api/changelog.md and bumped API_FEATURE_LEVEL.

Tweaked by tabbott to add an `id` field in RealmPlayground objects
sent to clients, which is essential to sending the API request to
remove one.
2021-04-06 20:56:58 -07:00
Sumanth V Rao d2e5b62dce realm/playground: Add API endpoint for deleting a playground entry.
Similar to the previous commit, we have added a `do_*` function
which does the deletion from the DB. The next commit handles sending
the events when both adding and deleting a playground entry.

Added the openAPI format data to zulip.yaml for DELETE
/realm/playgrounds/{playground_id}. Also added python and curl
examples to remove-playground.md.

Tests added.
2021-04-07 08:20:53 +05:30
Sumanth V Rao 251b415987 realm/playground: Add API endpoint for creating playground entry.
This endpoint will allow clients to create a playground entry
containing the name, pygments language and url_prefix for the
playground of their choice.

Introduced the `do_*` function in-charge of creating the entry in
the model. Handling the process of sending events which will be
done in a follow up commit.

Added the openAPI format data to zulip.yaml for POST
/realm/playgrounds. Also added python and curl examples for using
the endpoint in its markdown documented (add-playground.md).

Tests added.
2021-04-07 08:20:53 +05:30
Abhijeet Prasad Bodas 3bfcaa3968 mute user: Add backend infrastructure code.
Adds backend code for the mute users feature.
This is just infrastructure work (database
interactions, helpers, tests, events, API docs
etc) and does not involve any behavioral/semantic
aspects of muted users.

Adds POST and DELETE endpoints, to keep the
URL scheme mostly consistent in terms of `users/me`.

TODOs:
1. Add tests for exporting `zulip_muteduser` database table.
2. Add dedicated methods to python-zulip-api to be used
   in place of the current `client.call_endpoint` implementation.
2021-04-06 18:44:08 -07:00
PIG208 c6dfe7bf40 api: Add bot name to outgoing webhook payload.
Fixes: #12282
2021-04-06 17:37:50 -07:00
LoopThrough-i-j bdcbd53db7 docs: Allow passing additional params to `call_on_each_event`.
`call_on_each_event` now supports additional params other than
`event_type` and `narrow`; Ex: `all_public_streams` to fetch events
of all public streams.

Also add a bit of explanation of how this parameter works.

Fixes zulip/python-zulip-api#647
2021-04-06 08:40:32 -07:00
Abhijeet Prasad Bodas 68fe912c63 refactor: Rename most of "filter" to "linkifier".
After this only the database table, events,
and API endpoints remain.
2021-04-05 18:14:07 -07:00
Aman Agrawal 5e83965e80 giphy: Use GIPHY web SDK to allow inserting GIFs in compose box.
We use GIPHY web SDK to create popover containing GIFs in a
grid format. Simply clicking on the GIFs will insert the GIF in the compose
box.

We add GIPHY logo to compose box action icons which opens the GIPHY
picker popover containing GIFs with "Powered by GIPHY"
attribution.
2021-04-05 15:04:49 -07:00
Mateusz Mandera 82b43a8cfe streams: Use "archive stream" phrasing instead of "delete stream".
The previous phrasing was misleading in relation to what the action
actually does.
2021-04-02 22:06:48 -07:00
Dinesh 2d40224bb6 api docs: Fix incorrect comment in delete_stream documentation. 2021-04-01 07:49:42 -07:00
m-e-l-u-h-a-n aea31eb31f api: Add REALM_DEACTIVATED error code.
In `validate_account_and_subdomain` we check
if user's realm is not deactivated. In case
of failure of this check, we raise our standard
JsonableError. While this works well in most
cases but it creates difficulties in handling
of users with deactivated realms for non-browser
clients.

So we register a new REALM_DEACTIVATED error
code so that clients can distinguish if error
is because of deactivated account. Following
these changes `validate_account_and_subdomain`
raises RealmDeactivatedError if user's realm
is deactivated.

This error is also documented in
`/api/rest-error-handling`.

Testing: I have mostly relied on automated
backend tests to test this.

Fixes #17763.
2021-03-31 08:46:13 -07:00
m-e-l-u-h-a-n 2eeb82edba api: Add USER_DEACTIVATED error code.
In validate_account_and_subdomain we check if
user's account is not deactivated. In case of
failure of this check we raise our standard
JsonableError. While this works well in most
cases but it creates difficulties in handling
of deactivated accounts for non-browser clients.

So we register a new USER_DEACTIVATED error
code so that clients can distinguish if error
is because of deactivated account. Following
these changes `validate_account_and_subdomain`
raises UserDeactivatedError if user's account
is deactivated.

This error is also documented in
`/api/rest-error-handling`.

Testing: I have mostly relied on automated
backend tests to test this.

Partially addresses issue #17763.
2021-03-31 08:46:13 -07:00
Tim Abbott 36e320cf18 api docs: Fix link to old Help Center linkifiers URL. 2021-03-30 16:51:42 -07:00
Sumanth V Rao e12f682e2e markdown: Include text & url in `topic_links` parameter of our API.
The linkifier code now includes both the shortened text and the expanded
URL, sorted by the order of the occurrence in a topic. This list is passed
back in the `topic_links` parameter of the /messages and the /events APIs.

topic_links earlier vs now:

earlier: ['https://www.google.com', 'https://github.com/zulip/zulip/32']

now: [{'url': 'https://www.google.com', 'text': 'https://www.google/com},
      {'url': 'https://github.com/zulip/zulip/32', 'text': '#32'}]

Similarly, the topic_links local echo logic in the frontend now returns
back an object.

Fixes: #17109.
2021-03-30 15:53:07 -07:00
sahil839 54be0dd1a4 streams: Add moderators option in stream_post_policy.
This commit adds a new option of STREAM_POST_POLICY_MODERATORS
in stream_post_policy which will allow only realm admins and
moderators to post in that stream.
2021-03-30 09:06:20 -07:00
vagrant 92f4d206a6 documentation: Update upload_file python api documentation.
The current API documentation uses call_endpoint to upload a file;
since we've added a custom helper in python-zulip-api, we should
document that cleaner approach.
2021-03-30 08:52:07 -07:00
sahil839 6aa45c8e20 openapi: Fix typo in stream_post_policy sections of openapi docs.
The STREAM_POST_POLICY_RESTRICT_NEW_MEMBERS option of stream_post_policy
was explained as "Only new members can post" in the api docs. It should
instead be "Only full members can post" and this commit fixes it.
2021-03-29 17:37:34 -07:00
Suyash Vardhan Mathur 27ddb554fb openapi: Fix non-checking of /events and /register schema.
The /events and /register endpoints were excluded from schema validations,
because they were earlier not completely documented. However, they can
now be added for proper checking. Removed them from excluded endpoints list
and fixed the documentation for /register and /events after the checking.
Fixes #17796.
2021-03-28 22:05:44 -07:00
Anders Kaseorg d55dc6f8f1 requirements: Upgrade python-zulip-api from Git.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-26 16:31:03 -07:00
shanukun bc2d58ad4a custom_profile_fields: Remove op field for the event.
* `op` (operation) field, added in f6fb88549f, was never intended for
`custom_profile_fields` event. This commit removes the `op` as it doesn't
have any use in the code.

* As a part of cleanup, this also eliminates the schema check warnings
for `custom_profile_fields` event, mentioned in #17568.
2021-03-26 16:28:33 -07:00
Suyash Vardhan Mathur d3a3c6898c api docs: Add documentation of deactivate-own-user endpoint.
Currently, there was no markdown page for deactivate-own-user API
endpoint. Created deactivate-own-user.md for the API page and
created a new owner client in test-api to reactivate the client
deactivated during testing.

Also changed endpoint name from deactivate-my-account to
deactivate-own-user, for better consistency with other endpoints.

Fixes #16163.
2021-03-25 01:05:44 -07:00
Suyash Vardhan Mathur 9840803c00 openapi: Fix schema for unread_msgs in /register.
The schema for unread_msgs was missing additionalProperties: false
that was causing tests to pass even with undocumented parameters
which were validated as an additional property. Set
additionalProperties to false and added documentation for missing
count variable.

Fixes #17728.
2021-03-24 13:03:36 -07:00
shanukun cfe0fa3788 event_schema: Add schema check for realm/deactivated event.
This add the schema checker, openapi schema, and also a test for
realm/deactivated event.

With several block comments by tabbott explaining the logic behind our
behavior here.

Part of #17568.
2021-03-23 12:16:16 -07:00
shanukun a4cb264885 openapi: Remove add operation from op list of stream event.
This removes the `add` from op list of stream event, as we do not
actually generate the stream/add event in the API, and when a stream
is created we identify it using the `create` operation.

(This was likely just a mistake introduced as a result of the fact
that `create` does not fit the normal naming scheme; probably
long-term we should actually migrate this to "add", but more important
for now is to document what's accurate).

Part of #17568.
2021-03-22 23:44:56 -07:00
Tim Abbott f121e40848 message: Record whether unread_msgs data is truncated.
This is preparatory work for investigating reports of missing unread
messages.

It's a little surprising that not test failed after adding the code
without API documentation.

Co-Author-By: Tushar Upadhyay (tushar912).
2021-03-21 19:48:13 -07:00
Abhijeet Prasad Bodas 9223dced3b refactor: Rename filter to linkifier in frontend code and docs.
This only leaves `page_params.realm_filters`, which
will be changed in further commits along with the
API change.
2021-03-15 11:19:59 -07:00
Tushar912 55de66f944 api: Support user_id in get_user_presence_endpoint.
This is part of our general process of replacing emails, which are not
static with time, with user_ids when referring to users in the API.

We still keep the `email` reference option, since it can be useful for
linking third-party applications to Zulip on an intranet that might
have a user's corporate email handy and not want to do the extra round
trip to lookup the user.

The name of the parameter, user_id_or_email, was chosen to to make it
clear that the default/preferred option is user_id.

Fixes #14304.
2021-03-11 20:14:04 -08:00
Aman Agrawal e587c029f6 display_settings: Allow user to set default_view.
TextField is used to allow users to set long stream + topic narrow
names in the urls.

We currently restrict users to only set "all_messages" and
"recent_topics" as narrows.

This commit achieves 3 things:
* Removes recent topics as the default view which loads when
  hash is empty.
* Loads default_view when hash is empty.
* Loads default_view on pressing escape key when it is unhandled by
  other present UI elements.

NOTE: After this commit loading zulip with an empty hash will
automatically set hash to default_view.  Ideally, we'd just display
the default view without a hash, but that involves extra complexity.

One exception is when user is trying to load an overlay directly,
i.e. zulip is loaded with an overlay hash. In this case,
we render recent topics is background irrespective of default_view.

We consider this last detail to be a bug not important enough to block
adding this setting.
2021-03-11 18:09:08 -08:00
sahil839 9a432b0c3b events: Remove name field from update subscription events.
This commit removes name field from update subscription
events, as it is not used by any of the clients, and use
stream_id in the events code instead.
2021-03-07 22:03:24 -08:00
Tim Abbott 99001f5040 api docs: Fix claim that subject won't appear for PMs.
This was inaccurate after testing the implementation, and there's an
argument that we shouldn't move it as it will simplify migrating to a
world where (some) private message threads can have topics.
2021-03-01 20:35:13 -08:00
sahil839 b53c773987 events: Remove email field from update subscription events.
This commit removes email field from update subscription
events, as email field is of no use in this case.
2021-03-01 14:52:06 -08:00
shanukun 4b67946605 refactor: Make acting_user a mandatory kwarg for do_create_user. 2021-02-25 17:58:00 -08:00
Suyash Vardhan Mathur 0d01e7070f api docs: Add missing ID fields for event responses.
Some event responses were missing the id field. Added the ID at appropriate places.
2021-02-25 17:55:09 -08:00
Tim Abbott 101bccc867 api docs: Clarify when custom emoji events are sent. 2021-02-25 17:51:44 -08:00
neiljp (Neil Pilgrim) 3cad47bf4c api docs: Clarify realm_emoji update event returns all custom emoji. 2021-02-25 17:49:33 -08:00
Suyash Vardhan Mathur f1eda37836 api docs: Fix documentation of update-subscription-settings.
The description of request parameter of update-subscription-settings was
wrongly pasted in yaml and wasn't completely removed from the md file.
Made appropriate fixes in yaml and md file.
2021-02-24 13:28:58 -08:00
Suyash Vardhan Mathur dd8964a31f api docs: Fix id and type fields of events and display them.
Currently, the ID and Type fields didn't have a description,
and weren't being displayed. Added a schema component to add
descriptions, and display on the api page. Fixes part of #15967.
2021-02-23 15:22:53 -08:00
sahil839 6b5cf231a1 users: Add new user 'shiva' as realm moderator.
Note that at this point, it's not possible to create moderator users;
this just will make it easier to write tests for logic involving them
as we develop the feature.
2021-02-23 15:00:49 -08:00
LoopThrough-i-j d1ee2d31c7 examples-python-api: Fix `message_id` type.
The `message_id` was made an `str` object because
the request expected `Dict[str, str]`. The request is now
casted to `Dict[str, Any]` to fix the issue and removed
typecast of `message_id` to str.

python-zulip-api reference:
https://github.com/zulip/python-zulip-api/pull/653
2021-02-23 08:58:00 -08:00
Suyash Vardhan Mathur 96bfeeb9e6 api docs: Expand checking for deprecated fields.
Added assertion to check that if a deprecated flag is in a field's
schema, then it should have deprecated mentioned in description
as well, and moved these checks to a separate function.
Fixes part of #15967.
2021-02-16 15:34:52 -08:00
Tim Abbott a3e1b9161c openapi: Document deprecation for max_message_id field. 2021-02-16 08:08:40 -08:00
Tushar912 dfafdda9b3 api: Add REST API endpoint for looking up a user by email address.
Add new rest api endpoint GET users/{email} for looking up a user by
email, which is useful especially for corporate API applications that
might already have a user's email address.

Fixes #14302.
2021-02-15 17:38:33 -08:00
Shanu 7f196967ad event_queue: Remove internal fields being leaked to the API.
A few internal fields used for tracking which types of notifications
have already been sent for a given message, like `hander_id` and the
`push_notified` bundle of fields were being incorrectly included in
message events delivered to clients clients.

One could argue these fields might be useful hints to clients, but
because notifications can be triggered later on via
`missedmessage_hook`, they have no useful purpose in the API.

This commit move these extended event field on a `internal_data`
object within the event object, and delete this field in `contents()`
for call points that would serve data to clients.

Tweaked by tabbott to provide a cleaner interface.

We're not bumping API_FEATURE_LEVEL because these fields have always
been documented as being present only due to a bug, so no clients
should be expecting or relying on them.

Fixes: #15947.
2021-02-14 21:42:19 -08:00
Anders Kaseorg 6e4c3e41dc python: Normalize quotes with Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 11741543da python: Reformat with Black, except quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Suyash Vardhan Mathur 38dc1131b9 api docs: Minor fixes in documentation.
Added some missing response names and type in additionalProperties.
2021-02-05 10:41:42 -08:00
Suyash Vardhan Mathur 63c13a8f13 api docs: Fix documentation of update-subscription-settings.
The responses for the API weren't being rendered from yaml, and were
incorrectly formatted in yaml. The parameters also weren't completely
included in yaml and needed to be moved. Made appropriate fixes in
yaml and markdown file.
2021-02-05 10:41:42 -08:00
Anders Kaseorg a873a68cc7 eslint: Fix unicorn/no-new-array.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-25 14:53:19 -08:00
Steve Howell d9740045a5 refactor: Eliminate checks in build_stream_dict_for_sub.
We eliminate some redundant checks.

We also consistently provide a `subscribers` field
in our stream data with `[]`, even if our users
can't access subscribers.  We therefore bump
the API version and tweak the docs.  (See further
down for a detailed justification of the change.)

Even though it is sometimes fine to have redundant code
that is defensive in nature, some upcoming changes are gonna
move subscriber-related logic out of build_stream_dict_for_sub
for certain codepaths as part of our effort to streamline
the payload for subscribers within page_params.

So we can't rely on the code that I removed here
inside of build_stream_dict_for_sub.

Anyway, it makes more sense to do these checks explicitly
in the validate function.

The code in build_stream_dict_for_sub was almost effectively
a noop, since the validation function was already preventing
us from getting subscriber info.  The only difference it
made was sometimes converting `[]` to `None`, and then
subsequently omitting the subscribers field.

Neither ZT nor the webapp make any distinction between
`[]` or <missing key> for the `subscribers` data in
`page_params`.

The webapp has had this code for a long time (and now
equivalent code elsewhere in this PR):

    if (!Object.prototype.hasOwnProperty.call(sub, "subscribers")) {
        sub.subscribers = new LazySet([]);
    }

The webapp calculates access based on booleans, anyway:

    sub.can_access_subscribers =
        page_params.is_admin || sub.subscribed ||
        (!page_params.is_guest && !sub.invite_only);

And ZT would choke if `subscribers` were missing, except that
it never gets to the relevant code due to other checks:

    def get_other_subscribers_in_stream(<snip>):
        assert stream_id is not None or stream_name is not None

        if stream_id:
            assert self.is_user_subscribed_to_stream(stream_id)

            return [sub
                    for sub in self.stream_dict[stream_id]['subscribers']
                    if sub != self.user_id]
        else:
            return [sub
                    for _, stream in self.stream_dict.items()
                    for sub in stream['subscribers']
                    if stream['name'] == stream_name
                    if sub != self.user_id]

You could make a semantic argument that we should prefer
<missing key> to `[]` when subscribers aren't even available, but
we have precedent from the way that `bulk_get_subscriber_user_ids`
has traditionally populated its result:

    result: Dict[int, List[int]] =
        {stream["id"]: [] for stream in stream_dicts}

If we changed `stream_dicts` to `target_stream_dicts` we
would faciliate a move toward `None`, but it would just cause
headaches for other server code as well as the frontends
(which, to reiterate, already prefer the empty array
for convenience).
2021-01-21 15:04:07 -08:00
Mateusz Mandera 160cc5120a api: Require can_create_users permission to create users via API.
Allowing any admins to create arbitrary users is not ideal because it
can lead to abuse issues.  We should require something stronger that
requires the server operator's approval and thus we add a new
can_create_users permission.
2020-12-21 13:20:21 -08:00
Anders Kaseorg 2ab0b3d4fc validator: Reject ISO 8601 dates missing leading zeros.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-15 16:36:50 -08:00
Anders Kaseorg 415897f491 api docs: Use normal async/await code in JavaScript examples.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-15 11:32:18 -08:00
Anders Kaseorg 2c5e9f65f8 eslint: Fix new-cap errors.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-10 19:52:22 -08:00
Anders Kaseorg 4398eecd2b markdown: Use immutable type for extension config.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-11-10 15:54:27 -08:00
Anders Kaseorg 13c11ec5f3 openapi: Fix escaping in curl command generation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-11-05 09:36:31 -08:00
Tim Abbott eae14baa05 api: URL-quote password when testing authentication API.
The passwords generated for our development environment / test suite
include the `+` character, which needs to be quoted when encoded as an
HTTP POST parameter.

This is hopefully sufficient to fix the CI failures we've seen with
the tests for POST /api/v1/fetch_api_key; I haven't reproduced the
failure so am not completely sure.
2020-11-03 15:55:30 -08:00
shanukun be39672026 api_docs: Document the /fetch-api-key endpoint.
With tweaks by tabbott to document addition details.

Fixes: #16408.
2020-11-02 16:45:42 -08:00
shanukun da9d586254 openapi: Add parameter examples for fetch api key endpoints. 2020-11-02 16:45:42 -08:00
Anders Kaseorg aaa7b766d8 python: Use universal_newlines to get str from subprocess.
We can replace ‘universal_newlines’ with ‘text’ when we bump our
minimum Python version to 3.7.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-30 11:36:38 -07:00
Chris Bobbe 500f92169e openapi: Fix the "the the" instances. 2020-10-26 12:50:01 -07:00
Chris Bobbe dd43d4feb8 openapi: Note that a user's `avatar_url` may be missing.
Like it already says in the detail about
`user_avatar_url_field_optional`, but on the field itself [1].

[1] https://github.com/zulip/zulip-mobile/pull/4230#discussion_r493109645
2020-10-26 12:50:01 -07:00
Anders Kaseorg 31d0141a30 python: Close opened files.
Fixes various instances of ‘ResourceWarning: unclosed file’ with
python -Wd.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-26 12:31:30 -07:00
Harsh Srivastava 9b31df009b openapi: Fix excessively large test_events failure output.
Because of the very large `oneOf` clause of the formats of events
possible in Zulip's `GET /events` system, we had issues with
`test-backend` failures for missing documentation for a new event
format being like 1000 lines of output, which was very much unhelpful.

Fix this by limiting the output use only the oneOf variants that are
broadly similar to the actual payload received.

Fixes #16023.
2020-10-23 17:00:17 -07:00
Anders Kaseorg 72d6ff3c3b docs: Fix more capitalization issues.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-23 11:46:55 -07:00
Steve Howell 7ff3859136 subscriber events: Change schema for peer_add/peer_remove.
We now can send an implied matrix of user/stream tuples
for peer_add and peer_remove events.

The client code basically does this:

    for stream_id in event['stream_ids']:
        for user_id in event['user_ids']:
            update_sub(stream_id, user_id)

We used to send individual events, which gets real
expensive when you are creating new streams. For
the case of copy-to-stream case, we should see
events go from U to 1, where U is the number of users
added.

Note that we don't yet fully optimize the potential
of this schema.  For adding a new user with lots
of default streams, we still send S peer_add events.

And if you subscribe a bunch of users to a bunch of
private streams, we only go from U * S to S; we can't
optimize it down to one event easily.
2020-10-22 11:19:53 -07:00
Anders Kaseorg 254b904965 markdown: Migrate off deprecated extension registration interface.
Fixes #15205.

https://python-markdown.github.io/change_log/release-3.0/#homegrown-ordereddict-has-been-replaced-with-a-purpose-built-registry
https://python-markdown.github.io/change_log/release-3.0/#md_globals-keyword-deprecated-from-extension-api

The priority numbers are arbitrarily chosen to preserve the existing
order.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-19 18:31:12 -07:00
Steve Howell a9356508ca events: Stop sending occupy/vacate events.
We used to send occupy/vacate events when
either the first person entered a stream
or the last person exited.

It appears that our two main apps have never
looked at these events.  Instead, it's
generally the case that clients handle
events related to stream creation/deactivation
and subscribe/unsubscribe.

Note that we removed the apply_events code
related to these events.  This doesn't affect
the webapp, because the webapp doesn't care
about the "streams" field in do_events_register.

There is a theoretical situation where a
third party client could be the victim of
a race where the "streams" data includes
a stream where the last subscriber has left.
I suspect in most of those situations it
will be harmless, or possibly even helpful
to the extent that they'll learn about
streams that are in a "quasi" state where
they're activated but not occupied.

We could try to patch apply_event to
detect when subscriptions get added
or removed. Or we could just make the
"streams" piece of do_events_register
not care about occupy/vacate semantics.
I favor the latter, since it might
actually be what users what, and it will
also simplify the code and improve
performance.
2020-10-14 10:53:10 -07:00
Anders Kaseorg c7cbdba257 eslint: Fix unicorn/{new-for-builtins,throw-new-error,error-message}.
https://github.com/sindresorhus/eslint-plugin-unicorn/blob/master/docs/rules/new-for-builtins.md
https://github.com/sindresorhus/eslint-plugin-unicorn/blob/master/docs/rules/throw-new-error.md
https://github.com/sindresorhus/eslint-plugin-unicorn/blob/master/docs/rules/error-message.md

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:00:33 -07:00
sahil839 78b98d8067 realm: Add wildcard_mention_policy setting.
We add a new wildcard_mention_policy setting to handle wildcard
mentions in large streams, with a wide range of policies available to
organizations.

We set the default to the safe option for preventing accidental spam:
only stream administrators being able to use wildcard mentions in
large streams.
2020-10-01 12:18:03 -07:00
Anders Kaseorg cf5ededa35 openapi: Use at most one inline subschema in allOf.
This fixes some of the warnings from openapi-generator.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-30 19:45:28 -07:00
Anders Kaseorg cfd93096b5 openapi: Remove yamole.
As explained in the previous commit, yamole preprocessed allOf with an
algorithm that is not standards compliant.  We replicate that
algorithm, but importantly, we only use it for our own code and not
for building the openapi_core RequestValidator.

This improves the time taken by OpenAPISpec().check_reload() from
1.69s to 0.53s, nearly all of which is inside
openapi_core.create_spec.

Closes #10484.  Significantly improves #16068.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-29 16:47:10 -07:00
Anders Kaseorg fb2d7c6741 openapi: Fix allOf usage to conform to the OpenAPI specification.
yamole preprocesses our schema by naïvely merging all the objects in
an allOf array together, but this fails to capture the meaning of
allOf according to the OpenAPI specification.  allOf is supposed to be
a strict logical intersection of each subschema interpreted
independently.  It does not combine their properties maps before
interpreting additionalProperties.  So according to the old definition
of JsonSuccess, every response is invalid:

allOf:
  - additionalProperties: false
    properties:
      result:
        type: string
  - required:
      - result
      - msg
    properties:
      msg:
        type: string

because the first subschema disallowed msg and the second subschema
required msg.

To fix this, whenever we use allOf for schema “inheritence”, the base
schema must not specify additionalProperties, and the child schema
must explicitly list all properties recursively inherited from the
base schema in any subschema that uses additionalProperties.

Fixes #16109.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-29 16:47:10 -07:00
Anders Kaseorg c5765c9da6 openapi: Remove some keys redundant with a $ref.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-29 16:47:10 -07:00
Anders Kaseorg f3ff082107 openapi: Remove trivial allOf wrapping.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-29 16:47:10 -07:00
Harsh Srivastava ba36624442 api docs: Removing order dependency of deactivate user test.
We create a User using `do_create_user`, before running the
deactivation test.  This lets us removing the ordering logic
introduced in 7c17bdb9c5.
2020-09-14 22:31:40 -07:00
sahil839 9c3341ad95 openapi: Rearrange users/{user_id}/subscriptions/{stream_id} docs.
This commit moves docs for users/{user_id}/subscriptions/{stream_id}
enndpoint to be after users/me/subscriptions/muted_topics docs.

We are rearranging the docs because after adding the new patch
endpoint for users/{user_id}/subscriptions/{stream_id}, openapi_core
validator tries to match 'users/me/subscriptions/muted_topics'
with 'users/{user_id}/subscriptions/{stream_id}' path in zulip.yaml
and thus gives error while running tests.

This is a bug in 'openapi_core' as it does not follows OpenAPI specs
to match concrete paths before their templated counterparts. Thus,
this commit rearranges the docs such that openapi_core validator
tries to match muted_topics endpoint with the correct path in
zulip.yaml docs.
2020-09-14 22:04:45 -07:00
Tim Abbott f751acbec5 openapi: Add comments on top of various OpenAPI doc files.
This is part of our standard approach for trying to make it easy for
folks to find relevant documentation on a system they're trying to
understand.
2020-09-14 15:25:46 -07:00
Anders Kaseorg 7c17bdb9c5 openapi: Remove ‘example’ $ref siblings.
$ref siblings are ignored according to the OpenAPI specification, and
the referenced definitions already have examples.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-12 11:57:13 -07:00
Anders Kaseorg caa08d76b5 openapi: Inline parameter references to avoid ‘required’ $ref siblings.
$ref siblings are ignored according to the OpenAPI specification.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-12 11:57:13 -07:00
Tim Abbott f0c2c640b6 api docs: Add changes for community_topic_editing_limit_seconds.
The previous commit fixed this in the changelog, but it should have
also been documented directly as well.
2020-09-11 15:50:06 -07:00
Anders Kaseorg a276eefcfe python: Rewrite dict() as {}.
Suggested by the flake8-comprehensions plugin.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:15:41 -07:00
Anders Kaseorg 1ded51aa9d python: Replace list literal concatenation with * unpacking.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:15:41 -07:00
sahil839 fbae1685d6 openapi: Fix examples in response for deactivating user endpoints.
This commit fixes examples in "400" response for deactivating user
endpoints to have msg as "Cannot deactivate the last organization
owner" instead of "Cannot deactivate the last organization
administrator".

We had already removed the restriction on deactivating last admin
and added it for last owner, while adding owner role.
2020-08-30 17:12:57 -07:00
Tim Abbott c81b9cb516 api docs: Expand details on typing notifications API.
This provides proper documentation on exactly what is expected of
clients doing typing notifications.
2020-08-30 16:43:44 -07:00
orientor 148c375e5b events: Add documentation and tests for `typing: stop` event.
The `typing: stop` event did not have any tests in test_events
hence its documentation wasn't added. So add tests and relevant
documentation for the typing stop event. Also edit the documentation
of `typing: start` to include the fact that servers should use
their own timeout incase `stop` event event isn't received.

Fixes #16122.
2020-08-30 16:43:44 -07:00
orientor 12efa41ed6 api_docs: Add response details to outgoing webhooks documentation.
Improve OpenAPI documentation of /zulip-outgoing-webhook by moving
data and making appropriate additions from its couterpart in the
/outgoing-webhook docs. Then remove the redundant documentation
from the doc and add command to render OpenAPI documetation. Also
add a test to outgoing_webhooks_interface.py to ensure that OpenAPI
documentation is correct.

Fixes #16203.
2020-08-28 16:47:10 -07:00
Kartik Srivastava 0b77525814 api docs: Document POST /user_groups/{group_id}/members endpoint. 2020-08-26 15:40:19 -07:00
Kartik Srivastava 7677ba2d2b api docs: Rename 'group_id' to 'user_group_id'.
This renames 'group_id' to 'user_group_id' in the api docs to remove
the naming mismatch between the url config and the docs and eventually
remove the 'user_groups' endpoints from 'pending_endpoints' in
test_openapi.py.
2020-08-26 15:40:19 -07:00
Tim Abbott 2095ed6ae7 api docs: Edit custom profile field descriptions. 2020-08-26 15:35:10 -07:00
Kartik Srivastava 190701a062 api docs: Document POST /realm/profile_fields api endpoint. 2020-08-26 12:48:16 -07:00
Kartik Srivastava b74bf64c9d api docs: Document PATCH /realm/profile_fields endpoint. 2020-08-26 12:48:16 -07:00
Kartik Srivastava f8d6b9755a api docs: Document GET /realm/profile_fields endpoint. 2020-08-26 12:48:16 -07:00
Kartik Srivastava 94a29f5870 openapi: Add OpenAPI data for /realm/profile_fields api endpoint. 2020-08-26 12:48:16 -07:00
Hemanth V. Alluri 31a5e0b441 openapi: Fix DELETE /users/me status 200 response description.
If you look at line number 1121 (new) of commit 14c0a387cf,
I seem to have accidently set the description for a status
200 response to "Bad Request" instead of "Success" which
is what it really is. It's basically an ugly typo (maybe
due to hastily copy-pasting the template).

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-08-24 16:30:38 -07:00
orientor 372e010dbb events: Add `op` field to `update_message_flags` events.
`update_message_flags` events used `operation` instead of `op`, the
latter being the standard field used in other events. So add `op`
field to `update_message_flags` and mark `operation` as deprecated,
so that it can be removed later.
2020-08-24 12:42:03 -07:00
Steve Howell f33a314a4d openapi: Add enum for stream role. 2020-08-16 08:35:18 -04:00
sahil839 f046c9c58a streams: Add role field to Subscription objects passed to clients.
This commit adds "role" field to the Subscription objects passed to
clients.  This is important preparation for being able to work on the
frontend for this feature.
2020-08-14 16:33:11 -07:00
Steve Howell 9b6da158ad openapi: Add enum values to role node in specs.
We will eventually want to improve descriptions here
too.
2020-08-14 10:40:29 -04:00
Steve Howell 5931ebffd2 openapi: Use enum for message_type. 2020-08-14 10:40:29 -04:00
Anders Kaseorg a9539972f4 openapi: Fix validate_schema recursion structure.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-12 16:11:29 -07:00
Anders Kaseorg f1a9c87897 openapi: Add missing object types.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-12 16:11:29 -07:00
Anders Kaseorg ff46de305a openapi: Use reasonable variable names.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-12 16:11:29 -07:00
Anders Kaseorg 1d1149903b openapi: Remove unused document_events member.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-12 16:11:29 -07:00
Anders Kaseorg 4990e6d479 openapi: Deduplicate last modified check.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-12 16:11:29 -07:00
Kartik Srivastava 63173d5554 api: Return 'user_id' in 'POST /users' response.
This adds 'user_id' to the simple success response for 'POST /users'
api endpoint, to make it convenient for API clients to get details
about users they just created.  Appropriate changes have been made in
the docs and test_users.py.

Fixes #16072.
2020-08-11 16:40:12 -07:00
Anders Kaseorg 2061bd95f1 zulip.yaml: Don’t redundantly escape slashes.
These escapes are valid YAML 1.2 (for JSON compatibility) but not
valid YAML 1.1, which means they don’t work with the faster
yaml.CSafeLoader that we’d like to transition to.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 15:20:34 -07:00
Anders Kaseorg fb2e56e3c9 docs: Fix capitalization of keyboard keys.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:25:53 -07:00
Anders Kaseorg 768f9f93cd docs: Capitalize Markdown consistently.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:23:06 -07:00
Anders Kaseorg 60a25b2721 docs: Fix spelling errors caught by codespell.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:23:06 -07:00
orientor fae3f1ca53 openapi: Improve Bots documentation by dividing bots.
Firstly divide the Bot schema into Bot and BasicBot for ease
of reusability. Also separate bot remove and bot delete into
two separate events.
2020-08-06 12:29:43 -07:00
orientor 1d88c9e12e openapi: Document /register and add tests for it.
We'll want to do more iteration on the details here, but this is a
huge milestone.

Fixes #14188.
2020-08-05 17:57:24 -07:00
orientor 63af93ebe7 openapi: Add documentation for bot `services`. 2020-08-04 17:15:27 -07:00
orientor 1a6aeb710e openapi: Move frequently used data to schemas section without any edits.
To increase code reusability and reduce code redundancy, we move data
structures which occur multiple times in the OpenAPI documentation to
the `schemas` section. Note that this a pure data movement commit
without any changes to the data beyond removing over-specific
descriptions (E.g. that suggest the user group was just created).

(Future commits will use these)
2020-08-04 17:15:27 -07:00
orientor 8108acbdfd openapi_py: Make `/events` checking strict.
Previously there was a documented_events set which provided for partial
OpenAPI documentation while documentation was still going on. But since
the documentation is complete now, remove it.
2020-08-03 18:07:35 -07:00
orientor e7c9c55664 api: Complete get_events response format documetation.
This giant commit completes basic OpenAPI documentation for all events
in Zulip's real-time events API.

Further work will be required in the near future to make
/api/get_events usable.

With many edits by tabbott for wording and correctness (especially
around which clients receive events, and their purpose).
2020-08-03 18:07:09 -07:00
Anders Kaseorg 6ec808b8df js: Add "use strict" directive to CommonJS files.
ES and TypeScript modules are strict by default and don’t need this
directive.  ESLint will remind us to add it to new CommonJS files and
remove it from ES and TypeScript modules.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-31 22:09:46 -07:00
orientor 314c8ce52c events_documentation: Add more documentation. 2020-07-30 16:39:28 -07:00
Clara Dantas a9af80d7a2 streams: Make /streams endpoint return also web-public streams.
This commit modifies the /streams endpoint so that the web-public
streams are included in the default list of streams that users
have access to.

This is part of PR #14638 that aims to allow guest users to
browse and subscribe themselves to web public streams.
2020-07-29 17:52:36 -07:00
orientor f266b52469 events_documentation: Document all events of type stream and some others.
Document all events of `type`=stream i.e all `op`s. Also document some other
events.

Tweaked by tabbott to clarify some documentation details (especially
around who receives events).
2020-07-28 16:00:12 -07:00
Tim Abbott dbde901684 events: Document unintentionally exposed API fields.
It's a bug that these are sent in event payloads; I'll open an issue
for resolving that issue.  For now we document them mainly to make our
tests pass.
2020-07-27 18:08:41 -07:00
orientor 227c90a4ae openapi: Document `message` event. 2020-07-27 18:08:41 -07:00
orientor ec40a5dda4 openapi_py: Add validation for `/events`.
Edit the function `validate_against_openapi_schema` and add some
helper functions to allow for validation of documented events.
Also add OpenAPI response validation in `verify_action` as it is
called in a large number of `/events` tests.
2020-07-27 18:08:41 -07:00
orientor ed8d1925fb openapi: Edit `/events` OpenAPI specification for validation.
Some events in `/events` were incorrectly or incompletely documented.
Rectify them.
2020-07-27 18:08:41 -07:00
orientor cdacd3223b openapi: Document various events. 2020-07-27 18:08:41 -07:00
Tim Abbott 3252dfaa72 api: Document API change for Stream.date_created. 2020-07-27 17:10:59 -07:00
Sumanth V Rao 9b6de63afe stream/docs: Add date_created to Stream.API_FIELDS.
The parameter Stream.date_created is now sent down to the clients
for both:

    - client.get_streams()
    - client.list_subscriptions()

API docs updated for stream and subscriptions.

Fixes #15410
2020-07-27 16:33:36 -07:00
orientor c91c106cfb openapi_py: Change condition for invalid requests.
Change the condition for allowing failed validation to the condition
that `if the test fails, response status code begins with 4`. Also
add `intentionally_undocumented` argument in `validate_request` for
allowing passing of tests which return `200` responses but fail
validation due to some intentionally undocumented feature in
OpenAPI specification.
2020-07-26 16:26:56 -07:00
orientor c082fe301d openapi: Extract Emoji Reaction schema from `Messages`.
The Emoji Reaction schema is used in multiple place and hence
it is better to have it extracted as another schema in `components`.
2020-07-24 09:54:03 -07:00
orientor 881d57806b openapi: Move `stream` schema to components section.
The stream schema is used in two locations so move it to the
components section. Also the `is_default` key returned by `/streams`
is not returned by `/events`. So handle it separately.
2020-07-24 09:51:31 -07:00
orientor 2b879ecbdf openapi: Edit `Messages` component to match the `message` event object.
The `Messages` schema present in `#/components/schemas` was a
combination of all keys possible in any message object used in Zulip.
Edit it so that the original `Messages` contains just the keys present
in the `message` event. Also make another schema  `GetMessages` which
adds a few other keys which are received when using `GET /messages`.
The message object in the `/zulip-outgoing-webhook` has also been
modified and corrected.
2020-07-24 09:51:31 -07:00
orientor bdf3ecea42 openapi: Move `subscriptions` schema to the components section.
The `subscriptions` has use in multiple endpoints and hence instead
of redefining it at every point move it to the the components section
for easier reuse.
2020-07-24 09:51:31 -07:00
Anders Kaseorg 96dcc0ce6e js: Use ES6 object literal shorthand syntax.
Generated by ESLint.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-21 12:42:22 -07:00
Anders Kaseorg b65d2e063d js: Reformat with Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-17 14:31:25 -07:00
Anders Kaseorg 883e2fd325 js: Remove inner spacing from object literals.
We’re configuring Prettier with bracketSpacing: false.  Generated by
ESLint.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-17 14:31:25 -07:00
Anders Kaseorg f3726db89a js: Normalize strings to double quotes.
Prettier would do this anyway, but it’s separated out for a more
reviewable diff.  Generated by ESLint.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-17 14:31:24 -07:00
Anders Kaseorg 06691e1e45 prettier: Disable bracketSpacing.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-17 14:30:52 -07:00
Steve Howell c60f4236a9 api: Do not require short_name to create user.
When you post to /json/users, we no longer
require or look at the short_name parameter,
since we don't use it in any meaningful way.

An upcoming commit will eliminate it from the
database.
2020-07-17 11:15:15 -07:00
Steve Howell b375581f58 api: Remove (sender_)short_name from message APIs. 2020-07-17 11:15:14 -07:00
Rohitt Vashishtha 544c1cd3fe test-api: Use dynamic message ids in JS examples.
We replace hardcoded message ids which was prone to failure. This
new approach makes sure that the messages we're using for tests
actually exist.
2020-07-15 12:33:08 -07:00
Anders Kaseorg 2794bc1ef4 lint: Reformat YAML files with Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-14 16:25:31 -07:00
orientor 40f76bb030 openapi_py: Add function for validating requests.
Add a function for validating requests made during tests in `openapi.py`.
2020-07-14 14:23:58 -07:00
orientor bff8e43729 openapi_py: Add openapi-core validator object to OpenAPISpec class.
OpenAPISpec is our main class for accessing OpenAPI objects and
has the capability of creating the OpenAPI objects only once, thus
saving time. Since the openapi-core request validator object is going
to be accessed for considerable time during testing, hence add it to
the class for faster testing.
2020-07-14 13:57:50 -07:00
orientor ade5dae564 openapi: Fix `/settings/notifications` OpenAPI documentation.
We pass these values encoded as JSON and parsed with
validator=check_int, not as raw integers.
2020-07-14 13:57:50 -07:00
orientor ee39522eef openapi: Improve `/messages/{message_id}` PATCH documentation.
We extract OptionalContent and RequiredContent since some endpoints
require it and other don't.

In an ideal world, we'd have a better way to express these small
variants.
2020-07-14 13:57:50 -07:00
orientor 4ca4d43b0c openapi: Improve `/message` GET OpenAPI spec.
The `num_after` and `num_before` parameters should be 0 or greater.
2020-07-14 13:57:50 -07:00
orientor bb405ebc3d openapi: Move `/users/{user_id}` below endpoints with similar regex.
openapi-core, the request validator has a bug due to which data type
of path parameters is not checked. Hence `/users/{user_id}` can match
with `users/me`. So change the position of`/user/{user_id}` after all
such possible matches to avoid errors.

See https://github.com/p1c2u/openapi-core/issues/226 for details.
2020-07-14 13:57:50 -07:00
orientor 26bcd7e4d1 openapi: Move `/messages/{message_id}` below endpoints with similar regex.
openapi-core, the request validator has a bug due to which data type
of path parameters is not checked. Hence `/messages/{message_id}`
can match with `messages/matches_narrow`. So change the position of
of `message/{message_id}` after all such possible matches to avoid
errors.

See https://github.com/p1c2u/openapi-core/issues/226 for details.
2020-07-14 13:57:50 -07:00
Anders Kaseorg aa838ec4bc openapi: Disable arrow-body-style for javascript_examples.js.
Tweaked by tabbott to add a comment explaining why.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-03 16:55:43 -07:00
Kartik Srivastava b32e851458 js_examples: Migrate and test get_streams example.
Fixes #15496.
2020-07-01 17:23:30 -07:00
Kartik Srivastava 87787efa81 js_examples: Migrate and test get_events example.
This also adds "queue_id" to the response parameters and in the example
for GET /events in zulip.yaml.
2020-07-01 17:23:30 -07:00
orientor 529da34513 openapi: Use third-party validator for schema validation.
Our previous OpenAPI schema validator that we implemented ourselves
was useful training wheels for our understanding OpenAPI properly, and
was mostly correct.  But given that we've finally reached the point
where our OpenAPI file accurately describes the API, it makes sense to
switch to use an official OpenAPI validator.  We lose some ability to
do exclude rules for particular elements, but those were primarily
important for us when we had a lot of them.

As part of this change, we need to add `additionalProperties: false`
for all of our dictonaries/objects where we've documented every
parameter; otherwise the OpenAPI schema checker won't know that we
expect every parameter to be documented.
2020-07-01 11:21:41 -07:00
Anders Kaseorg e582bbea4a decorator: Strengthen types of signature-preserving decorators.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-30 18:58:23 -07:00
Anders Kaseorg a7bac82f2e curl_param_value_generators: Strengthen types.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-30 18:58:23 -07:00
orientor 166314de78 openapi: Correctly encode object and array parameters.
The current description of object and array parameters in
zulip.yaml is wrong and renders incorrect requests on using OpenAPI
tools such as SwaggerIO, etc. Fix it by encoding it correctly and
changing tests accordingly.

Fixes #14380.
2020-06-28 14:04:30 -07:00
Steve Howell 69be97e365 pointer: Remove pointer from API and page_params.
There is still some miscellaneous cleanup that
has to happen for things like analytics queries
and dead code in node tests, but this should
remove the main use of pointers in the backend.

(We will also still need to drop the DB field.)
2020-06-27 16:44:38 -07:00
orientor 5629dcc8a6 openapi_docs: Display deprecated parameters with a `deprecated` tag.
In zulip.yaml, add `deprecated` tags to all parameters/keys with
`Deprecated` in the description. Then add tests to ensure that deprecated
parameters/keys will always have the `deprecated` key. Also, in
the API docs, sort the parameters according to presence of `deprecated`
key, presenting the `deprecated` keys at the end and add a `deprecated`
tag next to them.
2020-06-26 16:05:41 -07:00
orientor f188708b20 attachments: Change data type and make variable names more accurate.
Change variable `name` to `date_sent` as `name` actually stores
the date sent. Also change the data types of `name` and `create_time`
to integer. As they actually have empty decimal value.
2020-06-26 14:39:18 -07:00
Tim Abbott def6189d53 docs: Document local echo paramters for sending messages. 2020-06-25 14:44:16 -07:00
Tim Abbott 6412ea6413 api docs: Document changes in API topic encoding. 2020-06-25 14:44:04 -07:00
Tim Abbott 0ecdc663b9 api docs: Correct errors in the stream creation documentation.
* Reordered the settings relevant without stream creation to the top.
* Removed useless/misleading defaults for optional parameters.
* Clarified description of the announce and authorization_errors_fatal settings.
* Clarified that `invite_only` only applies for stream creation.
  (It's annoying to do so for its friends because they are including
   common description content and OpenAPI doesn't have a way to have
   extra content in a place you included something)

Fixes #14705.
2020-06-25 14:34:10 -07:00
Tim Abbott e46bbf18eb docs: Change next planned major release to 3.0.
After some discussion, everyone seems to agree that 3.0 is the more
appropriate version number for our next major release.  This updates
our documentation to reflect that we'll be using 3.0 as our next major
release.
2020-06-24 16:27:27 -07:00
orientor 8ab6182683 fetch_api_key: Return `email` in json_response.
`/api/v1/fetch_api_key`'s response had a key `email` with the user's
delivery email. But its JSON counterpart `/json/fetch_api_key`, which
has a completely different implementation, did not return `email` in
its success response.

So to avoid confusion, the non-API endpoint, `/json/fetch_api_key`
response has been made identical with it's `/api` counterpart by
adding the `email` key. Also it is safe to send as the calling user
will only see their own email.
2020-06-24 15:13:31 -07:00
Kartik Srivastava 3b8ee2a30f js_examples: Migrate and test update_message example. 2020-06-23 14:42:03 -07:00
Kartik Srivastava 8a00b1f1aa js_examples: Migrate and test update_message_flags example. 2020-06-23 14:42:03 -07:00
Kartik Srivastava cc4086e98a js_examples: Migrate and test remove_subscriptions example. 2020-06-23 14:35:08 -07:00
Kartik Srivastava db303ccaef js_examples: Migrate and test add_subscriptions example. 2020-06-23 14:35:08 -07:00
Kartik Srivastava cf6e1de0de js_examples: Migrate and test set_typing_status example. 2020-06-23 14:35:08 -07:00
Kartik Srivastava a1f6540cec js_examples: Migrate and test render_message example. 2020-06-23 14:35:07 -07:00
Kartik Srivastava 5a4d31825c js_examples: Migrate and test register_queue example. 2020-06-23 14:35:07 -07:00
Kartik Srivastava 4aec4cbfae js_examples: Migrate and test get_users example. 2020-06-23 14:35:07 -07:00
Kartik Srivastava 69a7d7fd30 js_examples: Migrate and test get_subscriptions example.
Also, `send_message` example is altered to send a message to the
stream 'social' to avoid getting a "first_message_id: null"
in the response for `get_subscriptions` example, that caused
`validate_against_openapi_schema` to throw an error.
2020-06-23 14:35:07 -07:00
Kartik Srivastava cc7671e602 js_examples: Migrate and test get_stream_topics example. 2020-06-23 14:35:07 -07:00