AUTHENTICATION LINE variable needs to be set after each
line executed, but in the current code, it wasn't being
set in endpoints whose files were removed in favour of
the pages being generated directly from OpenAPI data.
Moved the block to set AUTHENTICATION LINE in the loop
which executes each command, which fixes the bug.
As a goal of a common template, there
is a need for a tool to auto-generate
general description for all parameters
directly from OpenAPI data.
This description is to be stored in
x-response-description field, and this
commit adds a markdown extesion to process the same.
As a goal of a common template, there
is a need for a tool to auto-generate
general description for all responses
directly from OpenAPI data.
This description is to be stored in
x-response-description field, and this
commit adds a markdown extesion to process the same.
We incorrectly include many realm settings in the data section of
'realm/update_dict' schema. It should only contain the settings
related to message edit, realm icon, realm logo and authentication
methods and not other settings, becausea all the other settings send
'realm/update' event and not 'realm/update_dict' event.
This commit only removes 'default_twenty_four_hour_time' and
'default_language', others will be removed separately.
Now, the markdown extension of curl_examples generates
all examples of all possible configurations with
their descriptions, and so we need to separate
executable curl commands from the rest of the raw
HTML.
This commit simply changes the indentation of the
block and replaces the command being tested
with each element of the commands array. This
was split for an easier review.
Now, the markdown extension of curl_examples generates
all examples of all possible configurations with
their descriptions, and so we need to separate
executable curl commands from the rest of the raw
HTML.
This commit adds a commands variable to store
all the curl commands in HTML using regex.
Now, include and exclude configuration
are fetched from openapi data, and only
one type can be encoded for every example.
This removes the need for the assertion to
test if both include and exclude are present
since at a time, only one can be present.
This commit adds support for using the
x-curl-examples-parameters parameter in OpenAPI
data to fetch curl examples configuration. This
also contains any descriptions necessary for each
example, and directly generates all possible
curl examples directly.
A follow-up commit is needed to modify the templates
accordingly.
As a goal of moving towards a common template,
we need to fetch curl examples' configurations
directly from openapi data instead of having them
hardcoded in templates. This commit introduces
x-curl-examples-parameters to store the configs
for the same.
This commit replaces the allow_community_topic_editing boolean with
integer field edit_topic_policy and includes both frontend and
backend changes.
We also update settings_ui.disable_sub_settings_onchange to not
change the color of label as we did previously when the setting
was a checkbox. But now as the setting is dropdown we keep the
label as it is and we don't do anything with label when disabling
dropdowns. Also, this function was used only here so we can safely
change this.
Currently, the `admin_config` configuration was
hardcoded in the templates, but as a goal of creating
a common template, we need to move all configurations
outside.
Moved the checking for function and language which need
admin_config out of templates into the code and added a
boolean `x-admin-config` to store whehter the operation
requires admin config.
Also, added the banner for admin access to auto-generate
if admin_config is present, and fixed the admin_config
for endpoints that were earlier missing it in the templates.
The javascript tab in .md templates can be
generated along with the line that adds js
example.
Further, as a part of the effort of moving
towards a single template, the markdown extension
for javascrit examples is modified to return empty
string if javascript example doesn't exist for that
endpoint. This would make it possible to cover more
endpoints with a single template.
The js example tabs are now automatically generated
during generation of javascript code, and so need to
be removed. Also, the markdown function to render js
examples can be added in all templates, since it is
parses and returns an empty string if the examples
don't exist, and allows us to move towards a common
template.
The pages have been verified to be correct
by using diff between old and new pages' raw HTML.
Currently, the descriptions for API responses were
hardcoded in the templates. However, they now exist
in YAML data as well, and so can be fetched from there.
Also, as a part of moving towards a common template, it is
beneficial to show return response description along with
the code example directly. Also, for the same reason, the
need for mentioning subschemas for the response has been
removed, and ALL responses for that response code should
be returned in the proper format.
This also includes returning empty string if that response
code doesn't exist. This allows us to directly mention both
200 and 400 responses in all templates, and they are parsed
automatically.
A followup is necessary to remove the descriptions in the template
as they are duplicated just by this commit
Currently, the descriptions of API responses were
hardcoded in the template. However, as a part of the
goal of moving towards a common template, these should
be auto-generated.
This commit copies these descriptions into the `description`
parameter in the OpenAPI data for that effect.
Rename poll_timeout to event_queue_longpoll_timeout_seconds
and change its value from 90000 ms to 90 sec. Expose its
value in register api response when realm data is fetched.
Bump API_FEATURE_LEVEL to 74.
Expose the boolean value server_needs_upgrade in the
responses for register api so that it can be used
by mobile and terminal clients as well.
Highlighted in api changelog as part of
feature level 74 in commit fb93c96
(next commit).
This commit adds support for testing of
those endpoints whose .md files would
be deleted in favour of their pages
to be generated automatically by the template.
curl examples for such endpoints must exist
in accordance to the pattern of template, which
can be used to run the tests for them.
The list curl_commands_to_test gets filled
already and so, the code to test each command
in the list can be moved out of the block that
opens the file. The only change in this commit
is reducing an indentation for the entire block.
This has been done to reuse the whole block
in case the file does not exist.
This commit refactors the code to find the
lines to be first, and then test all the
lines that contained commands.
This was done to avoid duplication of
the code for the other case, where the
.md file won't exist, as those are soon
to be deleted in favour of a common template.
Soon, the .md files of each endpoint would
be removed and be auto-generated from OpenAPI
data.
So, instead of using the files directly,
we should check from the list of endpoints and
open the files from there.
A follow-up to change logic when the .md files
get deleted should be done.
Since caa08d76b5, we no longer have
a common component for stream IDs in zulip.yaml, so we might as
well change the description to be specific and clear.
The current logic of geneerating HTML titles requires the title to be
present as a heading in the first line of .md file. However, this will
shortly be no longer true for /api pages where these are
auto-generated from OpenAPI data. Modified the code to fetch the
title from OpenAPI data in case of such pages.
Currently, the title of each endpoint are hardcoded in .md
files, but these are also added in summary parameter of
openapi. Added a markdown processor to insert the title
of a given endpoint.
As a part of effort of removing .md files for /api pages, the
title of each page can be added into the summary OpenAPI
parameter. This also adds a nice summary visible in Swagger
and enhances the documentation. Added the parameter for all endpoints.
Since this is currently only useful to interpret presence data, we
send this only if presence is requested.
I'm not sure that server_timestamp is the right name for this field,
but ultimately it should match the main presence API format.
Also link to it from the API documentation page,
other help pages, and the confirmation dialog for
muting a user.
With substantial edits by tabbott and alya.
* Remove unnecessary json_validator for string parameters.
* Update frontend to pass right parameter.
Bump api feature level and highlight the fix for `emojiset`
parameter of `settings/display` endpoint in zulip.yaml file.
Fixes part of #18035.
* Remove unnecessary json validator for string validator.
* Update frontend to pass right validator.
* Update zulip.yaml to pass right parameter for curl request
in openapi.
* Update python_examples to pass right paramater.
Fixes part of #18035.
* Move the extended documentation of code blocks to a separate page.
* Merge "code playgrounds" documentation to be a section of that page.
* Document copy widget on code blocks.
* This commit changes how we refer to "```python" type syntax for code
blocks. Instead of being called a syntax highlighting label, this is
now referred to as a "language tag", since it serves both syntax
highlighting and playgrounds.
* Remap all the links.
* Advertise this new page in various places that previously did not have a link.
Linked the Help Center document in places like
- zulip.yaml (/events, /register/, realm/playgrounds,
/realm/playgrounds/{playground_id})
- /help/format-your-message-using-markdown (Linked to make
users reading the markdown code block style, aware of this
feature)
- /templates/settings/playground_settings_admin.hbs (Linked
as a reference to read more about playgrounds before
configuring one)
Also showcase the feature on /features and /for/open-source.
This was accidentally removed in 27e4f5da92
as the initial idea was to have another parameter `stream_id`
for stream type messages instead of sending it in this `to` parameter.
Also fixed a description of `to` for `stream` type.
This commit modifies the user objects returned by 'GET /users',
'GET /users/me', 'GET /users/{user_id}' and 'GET /users/{email}'
endpoints to include role field.
We also include role field in the page_params['realm_users'] dict
and in the person object sent in (type="realm_user", op="add")
event.
This help mobile and terminal clients understand whether a server
restart changed API feature levels or not, which in turn determines
whether they will need to resynchronize their data.
Also add tests and documentation for this previously undocumented
event type.
Fixes: #18205.
These details clarify the roles of `message_id` and `message_ids`, and
should help a great deal in helping clients correctly implement this
critical API endpoint.
The order of the fields of the update_message event were previous
seemingly-arbitrary. This tries to more coherently order them:
1. Metadata
2. Stream
3. Topic
4. Content
Fixes#17795
In PR #17014, we added support for deactivate-own-user.
And while doing so, we first deactivated the client and
then reactivated it. But this implementation is a bit
hacky.
So, to fix this, we're now deactivating a test_user so that
we don't have to reactivate it. We did so by changing the value
of authentication_line.
As we want to keep endpoint code out of the
“test_curl_examples”, we changed the value of
authentication_line in `curl_param_value_generators.py`.
To work this out, we create a new global variable named
AUTHENTICATION_LINE in “curl_param_value_generators.py”
and change its value in function “deactivate_own_user” and
to use this change in “test_curl_examples,” we import
AUTHENTICATION_LINE.
AUTHENTICATION_LINE is of list data type because we want a
pointer to original mutable object so that changes made during
run time show across the module. Another way to do this is to change
the way we import variable, but that will be inconsistent to
the way we had in all other files.
To remove confusion between AUTHENTICATION_LINE and
authentication_line we renamed authentication_line
to default_authentication_line.
This extends the /json/typing endpoint to also accept
stream_id and topic. With this change, the requests
sent to /json/typing should have these:
* `to`: a list set to
- recipients for a PM
- stream_id for a stream message
* `topic`, in case of stream message
along with `op`(start or stop).
On receiving a request with stream_id and topic, we send
typing events to clients with stream_typing_notifications set
to True for all users subscribed to that stream.
This commit adds an API to `zproject/urls.py` to edit/update
the realm linkifier. Its helper function to update the
database is added in `zerver/lib/actions.py`.
`zulip.yaml` is documented accordingly as well, clearly
stating that this API updates one linkifier at a time.
The tests are added for the API and helper function which
updates the realm linkifier.
Fixes#10830.
Linkifier error message: `Filter not found` is
updated to `Linkifier not found.`.
Similarly, `filter_id` description is updated to:
`The ID of the linkifier that you want to remove.`,
renamed the term `filter` with `linkifier`, in `zulip.yaml`.
This prevents us from having to json encode every field in the POST
request to /realm/playgrounds, and keeps the client logic simpler
when adding a playground.
Organization admins can use this setting to restrict the maximum
rating of GIFs that will be retrieved from GIPHY. Also, there
is option to disable GIPHY too.
We had a mix of the two names, and "video call provider" both feels
more professional and more clear about precisely what it does.
We don't change the API fields, since it doesn't seem worth an API
migration.
In zulip.yaml, `pattern` and `url_format_string` are extracted
as a shared parameters.
This is done as a preparatory commit for `Add settings UI to edit
linkifiers`.
Related issue: #10830.
In "test_curl_examples.py" we find the functions that registered but
never called. To improve readablity, now we have the full
implementation in curl_param_value_generators, rather than inspecting
its fields from another module.
* This introduces a new event type `realm_linkifiers` and
a new key for the initial data fetch of the same name.
Newer clients will be expected to use these.
* Backwards compatibility is ensured by changing neither
the current event nor the /register key. The data which
these hold is the same as before, but internally, it is
generated by processing the `realm_linkifiers` data.
We send both the old and the new event types to clients
whenever the linkifiers are changed.
Older clients will simply ignore the new event type, and
vice versa.
* The `realm/filters:GET` endpoint (which returns tuples)
is currently used by none of the official Zulip clients.
This commit replaces it with `realm/linkifiers:GET` which
returns data in the new dictionary format.
TODO: Update the `get_realm_filters` method in the API
bindings, to hit this new URL instead of the old one.
* This also updates the webapp frontend to use the newer
events and keys.
Commit 4a3ad0d introduced some extra stream-level parameters
to the `realm` object. This commit extends that to add a
max_message_length paramter too in the same server_level.
Previously, you had to request the `stream` event type in order to get
the stream-level parameters; this was a bad design in part because the
`subscription` event type has similar data and is preferred by most
clients.
So we move these to the `realm` object. We also add the maximum topic
length, as an adjacent parameter.
While changing this, we also fix these to better match the names of
similar API parameters.
This fixes the API documentation tests following recent changes, but
isn't the right solution -- we probably want to change the API itself
to not require this strange JSON-encoding-of-a-string.
But this is necessary to have CI pass.
The comments explain in some detail, but basically we were displaying
the types for booleans incorrectly, and the types for strings in a
somewhat confusing fashion. Fix this with comments explaining the logic.
Using JSON dumping also results in our showing strings inside
quotation marks in our examples, which seems net helpful.
Thanks to ArunSankarKs for finding where we needed to change the
codebase.
Fixes#18021.
This commit adds backend code for passing can_invite_others_to_realm
field to clients using the fetch_initial_state_data in the page_params
object.
Though this field is not used by webapp as of now, but will be used
to fix a bug of incorreclty showing the invite users option in
settings overlay in the next commit.
We add moderators and full members option to invite_to_realm_policy
by using COMMON_POLICY_TYPES and use can_invite_others_to_realm helper
added in previous commit. This commit only does the backend work,
frontend work will be done in separate commit.
This commit replaces invite_by_admins_policy, which was a bool field,
with a new enum field invite_by_realm_policy.
Though the final goal is to add moderators and full members option
using COMMON_POLICY_TYPES, but this will be done in a separate
commit to make this easy for review.
We send the whole data set as a part of the event rather than
doing an add/remove operation for couple of reasons:
* This would make the client logic simpler.
* The playground data is small enough for us to not worry
about performance.
Tweaked both `fetch_initial_state_data` and `apply_events` to
handle the new playground event.
Tests added to validate the event matches the expected schema.
Documented realm_playgrounds sections inside /events and
/register to support our openapi validation system in test_events.
Tweaked other tests like test_event_system.py and test_home.py
to account for the new event being generated.
Lastly, documented the changes to the API endpoints in
api/changelog.md and bumped API_FEATURE_LEVEL.
Tweaked by tabbott to add an `id` field in RealmPlayground objects
sent to clients, which is essential to sending the API request to
remove one.
Similar to the previous commit, we have added a `do_*` function
which does the deletion from the DB. The next commit handles sending
the events when both adding and deleting a playground entry.
Added the openAPI format data to zulip.yaml for DELETE
/realm/playgrounds/{playground_id}. Also added python and curl
examples to remove-playground.md.
Tests added.
This endpoint will allow clients to create a playground entry
containing the name, pygments language and url_prefix for the
playground of their choice.
Introduced the `do_*` function in-charge of creating the entry in
the model. Handling the process of sending events which will be
done in a follow up commit.
Added the openAPI format data to zulip.yaml for POST
/realm/playgrounds. Also added python and curl examples for using
the endpoint in its markdown documented (add-playground.md).
Tests added.
Adds backend code for the mute users feature.
This is just infrastructure work (database
interactions, helpers, tests, events, API docs
etc) and does not involve any behavioral/semantic
aspects of muted users.
Adds POST and DELETE endpoints, to keep the
URL scheme mostly consistent in terms of `users/me`.
TODOs:
1. Add tests for exporting `zulip_muteduser` database table.
2. Add dedicated methods to python-zulip-api to be used
in place of the current `client.call_endpoint` implementation.
`call_on_each_event` now supports additional params other than
`event_type` and `narrow`; Ex: `all_public_streams` to fetch events
of all public streams.
Also add a bit of explanation of how this parameter works.
Fixeszulip/python-zulip-api#647
We use GIPHY web SDK to create popover containing GIFs in a
grid format. Simply clicking on the GIFs will insert the GIF in the compose
box.
We add GIPHY logo to compose box action icons which opens the GIPHY
picker popover containing GIFs with "Powered by GIPHY"
attribution.
In `validate_account_and_subdomain` we check
if user's realm is not deactivated. In case
of failure of this check, we raise our standard
JsonableError. While this works well in most
cases but it creates difficulties in handling
of users with deactivated realms for non-browser
clients.
So we register a new REALM_DEACTIVATED error
code so that clients can distinguish if error
is because of deactivated account. Following
these changes `validate_account_and_subdomain`
raises RealmDeactivatedError if user's realm
is deactivated.
This error is also documented in
`/api/rest-error-handling`.
Testing: I have mostly relied on automated
backend tests to test this.
Fixes#17763.
In validate_account_and_subdomain we check if
user's account is not deactivated. In case of
failure of this check we raise our standard
JsonableError. While this works well in most
cases but it creates difficulties in handling
of deactivated accounts for non-browser clients.
So we register a new USER_DEACTIVATED error
code so that clients can distinguish if error
is because of deactivated account. Following
these changes `validate_account_and_subdomain`
raises UserDeactivatedError if user's account
is deactivated.
This error is also documented in
`/api/rest-error-handling`.
Testing: I have mostly relied on automated
backend tests to test this.
Partially addresses issue #17763.
This commit adds a new option of STREAM_POST_POLICY_MODERATORS
in stream_post_policy which will allow only realm admins and
moderators to post in that stream.
The current API documentation uses call_endpoint to upload a file;
since we've added a custom helper in python-zulip-api, we should
document that cleaner approach.
The STREAM_POST_POLICY_RESTRICT_NEW_MEMBERS option of stream_post_policy
was explained as "Only new members can post" in the api docs. It should
instead be "Only full members can post" and this commit fixes it.
The /events and /register endpoints were excluded from schema validations,
because they were earlier not completely documented. However, they can
now be added for proper checking. Removed them from excluded endpoints list
and fixed the documentation for /register and /events after the checking.
Fixes#17796.
* `op` (operation) field, added in f6fb88549f, was never intended for
`custom_profile_fields` event. This commit removes the `op` as it doesn't
have any use in the code.
* As a part of cleanup, this also eliminates the schema check warnings
for `custom_profile_fields` event, mentioned in #17568.
Currently, there was no markdown page for deactivate-own-user API
endpoint. Created deactivate-own-user.md for the API page and
created a new owner client in test-api to reactivate the client
deactivated during testing.
Also changed endpoint name from deactivate-my-account to
deactivate-own-user, for better consistency with other endpoints.
Fixes#16163.
The schema for unread_msgs was missing additionalProperties: false
that was causing tests to pass even with undocumented parameters
which were validated as an additional property. Set
additionalProperties to false and added documentation for missing
count variable.
Fixes#17728.
This add the schema checker, openapi schema, and also a test for
realm/deactivated event.
With several block comments by tabbott explaining the logic behind our
behavior here.
Part of #17568.
This removes the `add` from op list of stream event, as we do not
actually generate the stream/add event in the API, and when a stream
is created we identify it using the `create` operation.
(This was likely just a mistake introduced as a result of the fact
that `create` does not fit the normal naming scheme; probably
long-term we should actually migrate this to "add", but more important
for now is to document what's accurate).
Part of #17568.
This is preparatory work for investigating reports of missing unread
messages.
It's a little surprising that not test failed after adding the code
without API documentation.
Co-Author-By: Tushar Upadhyay (tushar912).
This is part of our general process of replacing emails, which are not
static with time, with user_ids when referring to users in the API.
We still keep the `email` reference option, since it can be useful for
linking third-party applications to Zulip on an intranet that might
have a user's corporate email handy and not want to do the extra round
trip to lookup the user.
The name of the parameter, user_id_or_email, was chosen to to make it
clear that the default/preferred option is user_id.
Fixes#14304.
TextField is used to allow users to set long stream + topic narrow
names in the urls.
We currently restrict users to only set "all_messages" and
"recent_topics" as narrows.
This commit achieves 3 things:
* Removes recent topics as the default view which loads when
hash is empty.
* Loads default_view when hash is empty.
* Loads default_view on pressing escape key when it is unhandled by
other present UI elements.
NOTE: After this commit loading zulip with an empty hash will
automatically set hash to default_view. Ideally, we'd just display
the default view without a hash, but that involves extra complexity.
One exception is when user is trying to load an overlay directly,
i.e. zulip is loaded with an overlay hash. In this case,
we render recent topics is background irrespective of default_view.
We consider this last detail to be a bug not important enough to block
adding this setting.
This was inaccurate after testing the implementation, and there's an
argument that we shouldn't move it as it will simplify migrating to a
world where (some) private message threads can have topics.
The description of request parameter of update-subscription-settings was
wrongly pasted in yaml and wasn't completely removed from the md file.
Made appropriate fixes in yaml and md file.
Currently, the ID and Type fields didn't have a description,
and weren't being displayed. Added a schema component to add
descriptions, and display on the api page. Fixes part of #15967.
Note that at this point, it's not possible to create moderator users;
this just will make it easier to write tests for logic involving them
as we develop the feature.
The `message_id` was made an `str` object because
the request expected `Dict[str, str]`. The request is now
casted to `Dict[str, Any]` to fix the issue and removed
typecast of `message_id` to str.
python-zulip-api reference:
https://github.com/zulip/python-zulip-api/pull/653
Added assertion to check that if a deprecated flag is in a field's
schema, then it should have deprecated mentioned in description
as well, and moved these checks to a separate function.
Fixes part of #15967.
Add new rest api endpoint GET users/{email} for looking up a user by
email, which is useful especially for corporate API applications that
might already have a user's email address.
Fixes#14302.
A few internal fields used for tracking which types of notifications
have already been sent for a given message, like `hander_id` and the
`push_notified` bundle of fields were being incorrectly included in
message events delivered to clients clients.
One could argue these fields might be useful hints to clients, but
because notifications can be triggered later on via
`missedmessage_hook`, they have no useful purpose in the API.
This commit move these extended event field on a `internal_data`
object within the event object, and delete this field in `contents()`
for call points that would serve data to clients.
Tweaked by tabbott to provide a cleaner interface.
We're not bumping API_FEATURE_LEVEL because these fields have always
been documented as being present only due to a bug, so no clients
should be expecting or relying on them.
Fixes: #15947.
The responses for the API weren't being rendered from yaml, and were
incorrectly formatted in yaml. The parameters also weren't completely
included in yaml and needed to be moved. Made appropriate fixes in
yaml and markdown file.
We eliminate some redundant checks.
We also consistently provide a `subscribers` field
in our stream data with `[]`, even if our users
can't access subscribers. We therefore bump
the API version and tweak the docs. (See further
down for a detailed justification of the change.)
Even though it is sometimes fine to have redundant code
that is defensive in nature, some upcoming changes are gonna
move subscriber-related logic out of build_stream_dict_for_sub
for certain codepaths as part of our effort to streamline
the payload for subscribers within page_params.
So we can't rely on the code that I removed here
inside of build_stream_dict_for_sub.
Anyway, it makes more sense to do these checks explicitly
in the validate function.
The code in build_stream_dict_for_sub was almost effectively
a noop, since the validation function was already preventing
us from getting subscriber info. The only difference it
made was sometimes converting `[]` to `None`, and then
subsequently omitting the subscribers field.
Neither ZT nor the webapp make any distinction between
`[]` or <missing key> for the `subscribers` data in
`page_params`.
The webapp has had this code for a long time (and now
equivalent code elsewhere in this PR):
if (!Object.prototype.hasOwnProperty.call(sub, "subscribers")) {
sub.subscribers = new LazySet([]);
}
The webapp calculates access based on booleans, anyway:
sub.can_access_subscribers =
page_params.is_admin || sub.subscribed ||
(!page_params.is_guest && !sub.invite_only);
And ZT would choke if `subscribers` were missing, except that
it never gets to the relevant code due to other checks:
def get_other_subscribers_in_stream(<snip>):
assert stream_id is not None or stream_name is not None
if stream_id:
assert self.is_user_subscribed_to_stream(stream_id)
return [sub
for sub in self.stream_dict[stream_id]['subscribers']
if sub != self.user_id]
else:
return [sub
for _, stream in self.stream_dict.items()
for sub in stream['subscribers']
if stream['name'] == stream_name
if sub != self.user_id]
You could make a semantic argument that we should prefer
<missing key> to `[]` when subscribers aren't even available, but
we have precedent from the way that `bulk_get_subscriber_user_ids`
has traditionally populated its result:
result: Dict[int, List[int]] =
{stream["id"]: [] for stream in stream_dicts}
If we changed `stream_dicts` to `target_stream_dicts` we
would faciliate a move toward `None`, but it would just cause
headaches for other server code as well as the frontends
(which, to reiterate, already prefer the empty array
for convenience).
Allowing any admins to create arbitrary users is not ideal because it
can lead to abuse issues. We should require something stronger that
requires the server operator's approval and thus we add a new
can_create_users permission.
The passwords generated for our development environment / test suite
include the `+` character, which needs to be quoted when encoded as an
HTTP POST parameter.
This is hopefully sufficient to fix the CI failures we've seen with
the tests for POST /api/v1/fetch_api_key; I haven't reproduced the
failure so am not completely sure.
Because of the very large `oneOf` clause of the formats of events
possible in Zulip's `GET /events` system, we had issues with
`test-backend` failures for missing documentation for a new event
format being like 1000 lines of output, which was very much unhelpful.
Fix this by limiting the output use only the oneOf variants that are
broadly similar to the actual payload received.
Fixes#16023.
We now can send an implied matrix of user/stream tuples
for peer_add and peer_remove events.
The client code basically does this:
for stream_id in event['stream_ids']:
for user_id in event['user_ids']:
update_sub(stream_id, user_id)
We used to send individual events, which gets real
expensive when you are creating new streams. For
the case of copy-to-stream case, we should see
events go from U to 1, where U is the number of users
added.
Note that we don't yet fully optimize the potential
of this schema. For adding a new user with lots
of default streams, we still send S peer_add events.
And if you subscribe a bunch of users to a bunch of
private streams, we only go from U * S to S; we can't
optimize it down to one event easily.
We used to send occupy/vacate events when
either the first person entered a stream
or the last person exited.
It appears that our two main apps have never
looked at these events. Instead, it's
generally the case that clients handle
events related to stream creation/deactivation
and subscribe/unsubscribe.
Note that we removed the apply_events code
related to these events. This doesn't affect
the webapp, because the webapp doesn't care
about the "streams" field in do_events_register.
There is a theoretical situation where a
third party client could be the victim of
a race where the "streams" data includes
a stream where the last subscriber has left.
I suspect in most of those situations it
will be harmless, or possibly even helpful
to the extent that they'll learn about
streams that are in a "quasi" state where
they're activated but not occupied.
We could try to patch apply_event to
detect when subscriptions get added
or removed. Or we could just make the
"streams" piece of do_events_register
not care about occupy/vacate semantics.
I favor the latter, since it might
actually be what users what, and it will
also simplify the code and improve
performance.
We add a new wildcard_mention_policy setting to handle wildcard
mentions in large streams, with a wide range of policies available to
organizations.
We set the default to the safe option for preventing accidental spam:
only stream administrators being able to use wildcard mentions in
large streams.
As explained in the previous commit, yamole preprocessed allOf with an
algorithm that is not standards compliant. We replicate that
algorithm, but importantly, we only use it for our own code and not
for building the openapi_core RequestValidator.
This improves the time taken by OpenAPISpec().check_reload() from
1.69s to 0.53s, nearly all of which is inside
openapi_core.create_spec.
Closes#10484. Significantly improves #16068.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
yamole preprocesses our schema by naïvely merging all the objects in
an allOf array together, but this fails to capture the meaning of
allOf according to the OpenAPI specification. allOf is supposed to be
a strict logical intersection of each subschema interpreted
independently. It does not combine their properties maps before
interpreting additionalProperties. So according to the old definition
of JsonSuccess, every response is invalid:
allOf:
- additionalProperties: false
properties:
result:
type: string
- required:
- result
- msg
properties:
msg:
type: string
because the first subschema disallowed msg and the second subschema
required msg.
To fix this, whenever we use allOf for schema “inheritence”, the base
schema must not specify additionalProperties, and the child schema
must explicitly list all properties recursively inherited from the
base schema in any subschema that uses additionalProperties.
Fixes#16109.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
This commit moves docs for users/{user_id}/subscriptions/{stream_id}
enndpoint to be after users/me/subscriptions/muted_topics docs.
We are rearranging the docs because after adding the new patch
endpoint for users/{user_id}/subscriptions/{stream_id}, openapi_core
validator tries to match 'users/me/subscriptions/muted_topics'
with 'users/{user_id}/subscriptions/{stream_id}' path in zulip.yaml
and thus gives error while running tests.
This is a bug in 'openapi_core' as it does not follows OpenAPI specs
to match concrete paths before their templated counterparts. Thus,
this commit rearranges the docs such that openapi_core validator
tries to match muted_topics endpoint with the correct path in
zulip.yaml docs.
$ref siblings are ignored according to the OpenAPI specification, and
the referenced definitions already have examples.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
This commit fixes examples in "400" response for deactivating user
endpoints to have msg as "Cannot deactivate the last organization
owner" instead of "Cannot deactivate the last organization
administrator".
We had already removed the restriction on deactivating last admin
and added it for last owner, while adding owner role.
The `typing: stop` event did not have any tests in test_events
hence its documentation wasn't added. So add tests and relevant
documentation for the typing stop event. Also edit the documentation
of `typing: start` to include the fact that servers should use
their own timeout incase `stop` event event isn't received.
Fixes#16122.
Improve OpenAPI documentation of /zulip-outgoing-webhook by moving
data and making appropriate additions from its couterpart in the
/outgoing-webhook docs. Then remove the redundant documentation
from the doc and add command to render OpenAPI documetation. Also
add a test to outgoing_webhooks_interface.py to ensure that OpenAPI
documentation is correct.
Fixes#16203.
This renames 'group_id' to 'user_group_id' in the api docs to remove
the naming mismatch between the url config and the docs and eventually
remove the 'user_groups' endpoints from 'pending_endpoints' in
test_openapi.py.
If you look at line number 1121 (new) of commit 14c0a387cf,
I seem to have accidently set the description for a status
200 response to "Bad Request" instead of "Success" which
is what it really is. It's basically an ugly typo (maybe
due to hastily copy-pasting the template).
Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
`update_message_flags` events used `operation` instead of `op`, the
latter being the standard field used in other events. So add `op`
field to `update_message_flags` and mark `operation` as deprecated,
so that it can be removed later.
This commit adds "role" field to the Subscription objects passed to
clients. This is important preparation for being able to work on the
frontend for this feature.
This adds 'user_id' to the simple success response for 'POST /users'
api endpoint, to make it convenient for API clients to get details
about users they just created. Appropriate changes have been made in
the docs and test_users.py.
Fixes#16072.
These escapes are valid YAML 1.2 (for JSON compatibility) but not
valid YAML 1.1, which means they don’t work with the faster
yaml.CSafeLoader that we’d like to transition to.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
To increase code reusability and reduce code redundancy, we move data
structures which occur multiple times in the OpenAPI documentation to
the `schemas` section. Note that this a pure data movement commit
without any changes to the data beyond removing over-specific
descriptions (E.g. that suggest the user group was just created).
(Future commits will use these)
Previously there was a documented_events set which provided for partial
OpenAPI documentation while documentation was still going on. But since
the documentation is complete now, remove it.
This giant commit completes basic OpenAPI documentation for all events
in Zulip's real-time events API.
Further work will be required in the near future to make
/api/get_events usable.
With many edits by tabbott for wording and correctness (especially
around which clients receive events, and their purpose).
ES and TypeScript modules are strict by default and don’t need this
directive. ESLint will remind us to add it to new CommonJS files and
remove it from ES and TypeScript modules.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
This commit modifies the /streams endpoint so that the web-public
streams are included in the default list of streams that users
have access to.
This is part of PR #14638 that aims to allow guest users to
browse and subscribe themselves to web public streams.
Document all events of `type`=stream i.e all `op`s. Also document some other
events.
Tweaked by tabbott to clarify some documentation details (especially
around who receives events).
Edit the function `validate_against_openapi_schema` and add some
helper functions to allow for validation of documented events.
Also add OpenAPI response validation in `verify_action` as it is
called in a large number of `/events` tests.
The parameter Stream.date_created is now sent down to the clients
for both:
- client.get_streams()
- client.list_subscriptions()
API docs updated for stream and subscriptions.
Fixes#15410
Change the condition for allowing failed validation to the condition
that `if the test fails, response status code begins with 4`. Also
add `intentionally_undocumented` argument in `validate_request` for
allowing passing of tests which return `200` responses but fail
validation due to some intentionally undocumented feature in
OpenAPI specification.
The stream schema is used in two locations so move it to the
components section. Also the `is_default` key returned by `/streams`
is not returned by `/events`. So handle it separately.
The `Messages` schema present in `#/components/schemas` was a
combination of all keys possible in any message object used in Zulip.
Edit it so that the original `Messages` contains just the keys present
in the `message` event. Also make another schema `GetMessages` which
adds a few other keys which are received when using `GET /messages`.
The message object in the `/zulip-outgoing-webhook` has also been
modified and corrected.
The `subscriptions` has use in multiple endpoints and hence instead
of redefining it at every point move it to the the components section
for easier reuse.
Prettier would do this anyway, but it’s separated out for a more
reviewable diff. Generated by ESLint.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
When you post to /json/users, we no longer
require or look at the short_name parameter,
since we don't use it in any meaningful way.
An upcoming commit will eliminate it from the
database.
OpenAPISpec is our main class for accessing OpenAPI objects and
has the capability of creating the OpenAPI objects only once, thus
saving time. Since the openapi-core request validator object is going
to be accessed for considerable time during testing, hence add it to
the class for faster testing.
We extract OptionalContent and RequiredContent since some endpoints
require it and other don't.
In an ideal world, we'd have a better way to express these small
variants.
Suyash Vardhan Mathur
sahil839
Aman Agrawal
Anders Kaseorg
Gaurav Pandey
Vishnu KS
Tim Abbott
Mateusz Mandera
Abhijeet Prasad Bodas
Alya Abbott
Sumanth V Rao
Dinesh
m-e-l-u-h-a-n
Alex Dehnert
100RABHpy
akshatdalton
Zeeshan Equbal
PIG208
LoopThrough-i-j
vagrant
shanukun
Tushar912
neiljp (Neil Pilgrim)
Steve Howell
Chris Bobbe
Harsh Srivastava
Harsh Srivastava
orientor
Kartik Srivastava
Hemanth V. Alluri
Clara Dantas
Rohitt Vashishtha