We allow clients to make existing streams web public via the API.
This feature is still disabled via settings in production
environments, because we may have additional policy rules or UI
warnings we wish to add to this sort of conversion.
User can now create web public stream via the /subscribe API.
So, when a web public stream present in the API request does not
exist, it will be created now by specifying the is_web_public
parameter. The parameter would have been ignored without this
commit.
Now, when we add a custom animated emoji to the realm
we also save a still image of it (1st frame of the gif). So
we can avoid showing an animated emoji every time.
This commit adds do_set_realm_user_default_setting which
will be used to change the realm-level defaults of settings
for new users.
We also add a new event type "realm_user_settings_defaults"
for these settings and a "realm_user_settings_default" object
in '/register' response containing all the realm-level default
settings.
This commit adds `demo_organization_scheduled_deletion_date` to
the `realm` section of the `/register` response so that it is
available to clients when enabled.
This is a part of #19523.
This commit indicates that the realm_message_retention_days field can have
a special value, similar to its stream counterpart, and also explains how
the special value changed over different server versions.
With an extension from tabbott to double-enter the changelog entry.
Related discussion: https://chat.zulip.org/#narrow/stream/378-api-design/topic/realm_message_retention_days
These changes are all independent of each other; I just didn’t feel
like making dozens of commits for them.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
This commit updates both the stream-level and realm-level message
retention setting to use 'unlimited' instead of 'forever' to set
message retention setting to "retain messages forever".
We added "user_settings" object containing all the user settings in
previous commit. This commit modifies the code to send the existing
setting fields in the top-level object only if user_settings_object
client_capabilities field is False.
This commit adds "user_settings_object" field to
client_capabilities which will be used to determine
if the client needs 'update_display_settings' and
'update_global_notifications' event.
We send a event with type 'user_settings' on updating user's display
and notification settings.
The old event types - 'update_global_notifications' and
'update_display_settings', are still supported for backwards
compatibility.
Return zulip_merge_base alongside zulip_version
in `/register`, `/event` and `/server_settings`
endpoint so that the value can be used by other
clients.
These were added at some point in the past, but were not complete, and
it makes sense to document the current feature level as and when they
become available, since clients should not use the drafts endpoints on
older feature levels.
In this commit:
* We update the `UserStatus` model to accept
`AbstractReaction` as a base class so, we can get all the
fields related to store status emoji.
* We update the user status endpoint
(`users/me/status`) to accept status emoji fields.
* We update the user status event to add status emoji
fields.
Co-authored-by: Yash Rathore <33805964+YashRE42@users.noreply.github.com>
This commit replaces boolean field add_emoji_by_admins_only with an
integer field add_custom_emoji_policy as we would also add full members
and moderators option for this setting in further commits.
Cross realm bots will soon stop being a thing. This param is responsible
for displaying "System Bot" in the user info popover - so this rename is the
right way to handle the situation.
We will likely want to rename the `cross_realm_bots` section as well,
but that is a more involved API migration.
We allow a maximum value of one week to make sure there aren't a huge
number of rows in the table for any user (this could happen if stream
notifications are enabled).
This commit also fixes a small error in the user_settings test.
This API change removes unnecessary complexity from a client that
wants to change a user's personal settings, and also saves developers
from needing to make decisions about what sort of setting something is
at the API level.
We preserve the old settings endpoints as mapping to the same function
as the new one for backwards-compatibility. We delete the
documentation for the old endpoints, though the documentation for the
merged /settings endpoint mentions how to use the old endpoints when
needed.
We migrate all backend tests to the new endpoints, except for
individual tests for each legacy endpoint to verify they still work.
Co-authored-by: sahil839 <sahilbatra839@gmail.com>
Sometime in the deep past, Zulip the GET /users/me/subscriptions
endpoint started returning subscribers. We noticed this and made it
optional via the include_subscribers parameter in
1af72a2745, however, we didn't notice
that they were being returned as emails rather than user IDs.
We migrated the core /register code paths to use subscriber IDs years
ago; this change completes that for the endpoints we forgot about.
The documentation allowed this error because we apparently had no
tests for this code path that used the actual API.
We remove the "full_name" and "account_email" fields from the response
of 'PATCH /settings' endpoint. These fields were part of the response
to make sure that we tell that the parameters not present in response
were ignored.
We can remove these fields as 'ignored_parameters_unsupported' now
specifies which parameters were ignored and not supported by the
endpoint.
We add "ignored_parameters_unsupported" field to the response object
of 'PATCH /settings' endpoint. This will contain the parameters
passed to the endpoint which are not changed by the endpoint and are
ignored.
This will help in removing the other fields like "full_name" from
response which was essentially present to specify that only these
fields were updated by the endpoint and rest were ignored.
We will also change other endpoints to follow this in future.
The operationId is directly used in URLs of API doc pages
to find the OpenAPI data to render. However, this is dash-
separated in the URLs, and having underscore_separated IDs
in OpenAPI data doesn't allow direct comparison of the two.
This commit changes all OperationIDs from underscore_separated
to dash-separated.
Previously, one needed to specifying all the HTTP status
codes that we want to render along with the operation,
but the primary use case just needs the responses of
all the status codes, and not just one.
This commit modifies the Markdown extension to render
all the responses of all status codes of a specified
operation in a loop.
Now, get-events template fits the standard
format of the api docs template, and so the
redundant markdown template can be deleted to
generate the page diretly from openapi data.
As we have changed the tab selector above from "Settings" to "Personal
settings", we can simply change "Your bots" to "Bots" as "Bots" is
clear enough given the personal settings context.
We also need to update the API documentation for bots accordingly.
The fixtures are now rendered differently, and using
just 400 in fixture renders all the 400 fixtures with
descriptions, whereas 400_1 and 400_0 are invalid now.
This commit removes the hardcoded descriptions, and fixes
the fixture rendering.
This commit fixes the documentation of settings as we have
replaced "Your account" section into two new sections -
"Profile" and "Account & privacy".
This commit also fixes a comment in the test for settings
documentation in test_middleware.py.
This PR adds a basic .md template that is followed by lot of /api
pages. Since we have recently done the migration work to ensure that
our REST API documentation pages for individual endpoints are almost
all identical files following a common pattern, we can now get the
payoff of deleting them all in favor of a shared template.
This removes 2000 lines of somewhat finicky configuration from the
codebase, and thus should save significant effort when documenting new
API endpoints in the future.
The markdown files for endpoints or other pages which deviate from the
standard template remain, and the docs are instead generated from
those files using the existing system.
As a part of goal of moving towards a common template,
the hardcoded python tabs need to be removed to ensure
that endpoints which don't have python examples can be
covered by the common template as well.
This commit also modifies the markdown extension for python
examples to render empty string in case the examples don't
exist, which would allow it to be called whether the endpoint
has python examples or not.
We use the "does not accept any parameters" language in the common
template that we'll be migrating to shortly, so we remove this
variance (And adjust its test).
This commit modifies the templates to
auto-generate general descriptions of
responses directly from the newly
added field of x-response-description
as a part of the goal of a common template.
This commit modifies the templates to
auto-generate general descriptions of
parameters directly from the newly
added field of x-parameter-description
as a part of the goal of a common template.
Now, all possible configurations for curl examples
are rendered directly from openapi data, and
the templates should just call the markdown
function once.
This commit adjusts all the templates by removing
hardcoded configurations and descriptions for
individual curl examples.
This commit replaces the allow_community_topic_editing boolean with
integer field edit_topic_policy and includes both frontend and
backend changes.
We also update settings_ui.disable_sub_settings_onchange to not
change the color of label as we did previously when the setting
was a checkbox. But now as the setting is dropdown we keep the
label as it is and we don't do anything with label when disabling
dropdowns. Also, this function was used only here so we can safely
change this.
Currently, the `admin_config` configuration was
hardcoded in the templates, but as a goal of creating
a common template, we need to move all configurations
outside.
Moved the checking for function and language which need
admin_config out of templates into the code and added a
boolean `x-admin-config` to store whehter the operation
requires admin config.
Also, added the banner for admin access to auto-generate
if admin_config is present, and fixed the admin_config
for endpoints that were earlier missing it in the templates.
The javascript tab in .md templates can be
generated along with the line that adds js
example.
Further, as a part of the effort of moving
towards a single template, the markdown extension
for javascrit examples is modified to return empty
string if javascript example doesn't exist for that
endpoint. This would make it possible to cover more
endpoints with a single template.
The js example tabs are now automatically generated
during generation of javascript code, and so need to
be removed. Also, the markdown function to render js
examples can be added in all templates, since it is
parses and returns an empty string if the examples
don't exist, and allows us to move towards a common
template.
The pages have been verified to be correct
by using diff between old and new pages' raw HTML.
The headings for return values were currently hardcoded
in cases where they occur, but they can be rendered directly
in the markdown extension if the return values exist.
The Hubot project looks to be abandoned; it hasn’t been updated in
years and its own installation instructions don’t work anymore.
Remove our special placement of Hubot alongside Zapier and IFTTT.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Now, the descriptions and all subschemas
are directly returned just by
`generate_code_example`, and so, these individual
subschemas and descriptions can be removed from the
templates.
All API endpoints docs are exactly the same after the
change, except few where missing 400 responses got added
due to the modification
Rename poll_timeout to event_queue_longpoll_timeout_seconds
and change its value from 90000 ms to 90 sec. Expose its
value in register api response when realm data is fetched.
Bump API_FEATURE_LEVEL to 74.
As a part of effort of removing .md files for /api pages, the
title of each page can be added into the summary OpenAPI
parameter. This also adds a nice summary visible in Swagger
and enhances the documentation. Added the parameter for all endpoints.
The previous description didn't make clear that you can edit a message
to add a mention.
Also, adjust indentation to be the normal 4-space style we use for HTML.
Since this is currently only useful to interpret presence data, we
send this only if presence is requested.
I'm not sure that server_timestamp is the right name for this field,
but ultimately it should match the main presence API format.
We change the title to be more descriptive and also link to the
overall Zulip changelog, since one can reasonably get to this page
when googling for "Zulip changelog".
* Remove unnecessary json_validator for string parameters.
* Update frontend to pass right parameter.
Bump api feature level and highlight the fix for `emojiset`
parameter of `settings/display` endpoint in zulip.yaml file.
Fixes part of #18035.
Update docs on how to build an incoming webhook integration to be more
accurate on exactly what is needed at this point in time as they were
quite out of date.
This removes unnecessary json_validator for string parameters in the
BigBlueButton video calls endpoints. Note that this breaks links to
video meetings sent before the upgrade; there's not much we can do
about that.
Since this is the last commit in this series, we update the
ZULIP_FEATURE_LEVEL for this batch of changes.
Fixes part of #18035.
Support for the timeouts, and tests for them, was added in
53a8b2ac87 -- though no code could have set them after 31597cf33e.
Add a 10-second default timeout. Observationally, p99 is just about
5s, with everything else being previously being destined to meet the
30s worker timeout; 10s provides a sizable buffer between them.
Fixes#17742.
Move `get_setup_webhook_message` to
`zerver/lib/webhooks/common.py` so multiple integrations can use this
rather than just those which import `zerver/lib/webhooks/git.py`. Also
added the documentation for this.
This commit adds both frontend and backend code to invite a user as
moderator. We allow only existing owners and admins to invite a user
as a moderator.
This commit modifies the user objects returned by 'GET /users',
'GET /users/me', 'GET /users/{user_id}' and 'GET /users/{email}'
endpoints to include role field.
We also include role field in the page_params['realm_users'] dict
and in the person object sent in (type="realm_user", op="add")
event.
This help mobile and terminal clients understand whether a server
restart changed API feature levels or not, which in turn determines
whether they will need to resynchronize their data.
Also add tests and documentation for this previously undocumented
event type.
Fixes: #18205.
This extends the /json/typing endpoint to also accept
stream_id and topic. With this change, the requests
sent to /json/typing should have these:
* `to`: a list set to
- recipients for a PM
- stream_id for a stream message
* `topic`, in case of stream message
along with `op`(start or stop).
On receiving a request with stream_id and topic, we send
typing events to clients with stream_typing_notifications set
to True for all users subscribed to that stream.
This commit adds an API to `zproject/urls.py` to edit/update
the realm linkifier. Its helper function to update the
database is added in `zerver/lib/actions.py`.
`zulip.yaml` is documented accordingly as well, clearly
stating that this API updates one linkifier at a time.
The tests are added for the API and helper function which
updates the realm linkifier.
Fixes#10830.
django.utils.translation.ugettext is a deprecated alias of
django.utils.translation.gettext as of Django 3.0, and will be removed
in Django 4.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Organization admins can use this setting to restrict the maximum
rating of GIFs that will be retrieved from GIPHY. Also, there
is option to disable GIPHY too.
* This introduces a new event type `realm_linkifiers` and
a new key for the initial data fetch of the same name.
Newer clients will be expected to use these.
* Backwards compatibility is ensured by changing neither
the current event nor the /register key. The data which
these hold is the same as before, but internally, it is
generated by processing the `realm_linkifiers` data.
We send both the old and the new event types to clients
whenever the linkifiers are changed.
Older clients will simply ignore the new event type, and
vice versa.
* The `realm/filters:GET` endpoint (which returns tuples)
is currently used by none of the official Zulip clients.
This commit replaces it with `realm/linkifiers:GET` which
returns data in the new dictionary format.
TODO: Update the `get_realm_filters` method in the API
bindings, to hit this new URL instead of the old one.
* This also updates the webapp frontend to use the newer
events and keys.
The possible errors had already been documented in
`zulip.yaml` in 3bfcaa3968.
I had missed adding some of them to the markdown files
in the API docs. This commit fixes that.
Commit 4a3ad0d introduced some extra stream-level parameters
to the `realm` object. This commit extends that to add a
max_message_length paramter too in the same server_level.
Previously, you had to request the `stream` event type in order to get
the stream-level parameters; this was a bad design in part because the
`subscription` event type has similar data and is preferred by most
clients.
So we move these to the `realm` object. We also add the maximum topic
length, as an adjacent parameter.
While changing this, we also fix these to better match the names of
similar API parameters.
* Don't require strings to be unnecessarily JSON-encoded.
* Use check_capped_string rather than custom code for length checks.
* Update frontend to pass the right parameters.
With a much simplified populate_data_for_request design suggested by
Anders; we only support a handful of data types, all of which are
correctly encoded automatically by jQuery.
Fixes part of #18035.
This commit adds backend code for passing can_invite_others_to_realm
field to clients using the fetch_initial_state_data in the page_params
object.
Though this field is not used by webapp as of now, but will be used
to fix a bug of incorreclty showing the invite users option in
settings overlay in the next commit.
This commit replaces invite_by_admins_policy, which was a bool field,
with a new enum field invite_by_realm_policy.
Though the final goal is to add moderators and full members option
using COMMON_POLICY_TYPES, but this will be done in a separate
commit to make this easy for review.
We send the whole data set as a part of the event rather than
doing an add/remove operation for couple of reasons:
* This would make the client logic simpler.
* The playground data is small enough for us to not worry
about performance.
Tweaked both `fetch_initial_state_data` and `apply_events` to
handle the new playground event.
Tests added to validate the event matches the expected schema.
Documented realm_playgrounds sections inside /events and
/register to support our openapi validation system in test_events.
Tweaked other tests like test_event_system.py and test_home.py
to account for the new event being generated.
Lastly, documented the changes to the API endpoints in
api/changelog.md and bumped API_FEATURE_LEVEL.
Tweaked by tabbott to add an `id` field in RealmPlayground objects
sent to clients, which is essential to sending the API request to
remove one.
Similar to the previous commit, we have added a `do_*` function
which does the deletion from the DB. The next commit handles sending
the events when both adding and deleting a playground entry.
Added the openAPI format data to zulip.yaml for DELETE
/realm/playgrounds/{playground_id}. Also added python and curl
examples to remove-playground.md.
Tests added.
This endpoint will allow clients to create a playground entry
containing the name, pygments language and url_prefix for the
playground of their choice.
Introduced the `do_*` function in-charge of creating the entry in
the model. Handling the process of sending events which will be
done in a follow up commit.
Added the openAPI format data to zulip.yaml for POST
/realm/playgrounds. Also added python and curl examples for using
the endpoint in its markdown documented (add-playground.md).
Tests added.
Adds backend code for the mute users feature.
This is just infrastructure work (database
interactions, helpers, tests, events, API docs
etc) and does not involve any behavioral/semantic
aspects of muted users.
Adds POST and DELETE endpoints, to keep the
URL scheme mostly consistent in terms of `users/me`.
TODOs:
1. Add tests for exporting `zulip_muteduser` database table.
2. Add dedicated methods to python-zulip-api to be used
in place of the current `client.call_endpoint` implementation.
`call_on_each_event` now supports additional params other than
`event_type` and `narrow`; Ex: `all_public_streams` to fetch events
of all public streams.
Also add a bit of explanation of how this parameter works.
Fixeszulip/python-zulip-api#647
We use GIPHY web SDK to create popover containing GIFs in a
grid format. Simply clicking on the GIFs will insert the GIF in the compose
box.
We add GIPHY logo to compose box action icons which opens the GIPHY
picker popover containing GIFs with "Powered by GIPHY"
attribution.
The Available flags table in update_message_flags.md was using
markdown for certain content inside HTML, which resulted in the
table not being rendered properly.
This commit fixes the table to use proper HTML for content rendering
instead of markdown, since the table was written in markdown's HTML
syntax.
In `validate_account_and_subdomain` we check
if user's realm is not deactivated. In case
of failure of this check, we raise our standard
JsonableError. While this works well in most
cases but it creates difficulties in handling
of users with deactivated realms for non-browser
clients.
So we register a new REALM_DEACTIVATED error
code so that clients can distinguish if error
is because of deactivated account. Following
these changes `validate_account_and_subdomain`
raises RealmDeactivatedError if user's realm
is deactivated.
This error is also documented in
`/api/rest-error-handling`.
Testing: I have mostly relied on automated
backend tests to test this.
Fixes#17763.
In validate_account_and_subdomain we check if
user's account is not deactivated. In case of
failure of this check we raise our standard
JsonableError. While this works well in most
cases but it creates difficulties in handling
of deactivated accounts for non-browser clients.
So we register a new USER_DEACTIVATED error
code so that clients can distinguish if error
is because of deactivated account. Following
these changes `validate_account_and_subdomain`
raises UserDeactivatedError if user's account
is deactivated.
This error is also documented in
`/api/rest-error-handling`.
Testing: I have mostly relied on automated
backend tests to test this.
Partially addresses issue #17763.
* `op` (operation) field, added in f6fb88549f, was never intended for
`custom_profile_fields` event. This commit removes the `op` as it doesn't
have any use in the code.
* As a part of cleanup, this also eliminates the schema check warnings
for `custom_profile_fields` event, mentioned in #17568.
Currently, there was no markdown page for deactivate-own-user API
endpoint. Created deactivate-own-user.md for the API page and
created a new owner client in test-api to reactivate the client
deactivated during testing.
Also changed endpoint name from deactivate-my-account to
deactivate-own-user, for better consistency with other endpoints.
Fixes#16163.
This is preparatory work for investigating reports of missing unread
messages.
It's a little surprising that not test failed after adding the code
without API documentation.
Co-Author-By: Tushar Upadhyay (tushar912).
This is part of our general process of replacing emails, which are not
static with time, with user_ids when referring to users in the API.
We still keep the `email` reference option, since it can be useful for
linking third-party applications to Zulip on an intranet that might
have a user's corporate email handy and not want to do the extra round
trip to lookup the user.
The name of the parameter, user_id_or_email, was chosen to to make it
clear that the default/preferred option is user_id.
Fixes#14304.
TextField is used to allow users to set long stream + topic narrow
names in the urls.
We currently restrict users to only set "all_messages" and
"recent_topics" as narrows.
This commit achieves 3 things:
* Removes recent topics as the default view which loads when
hash is empty.
* Loads default_view when hash is empty.
* Loads default_view on pressing escape key when it is unhandled by
other present UI elements.
NOTE: After this commit loading zulip with an empty hash will
automatically set hash to default_view. Ideally, we'd just display
the default view without a hash, but that involves extra complexity.
One exception is when user is trying to load an overlay directly,
i.e. zulip is loaded with an overlay hash. In this case,
we render recent topics is background irrespective of default_view.
We consider this last detail to be a bug not important enough to block
adding this setting.
The description of request parameter of update-subscription-settings was
wrongly pasted in yaml and wasn't completely removed from the md file.
Made appropriate fixes in yaml and md file.
Add new rest api endpoint GET users/{email} for looking up a user by
email, which is useful especially for corporate API applications that
might already have a user's email address.
Fixes#14302.
The responses for the API weren't being rendered from yaml, and were
incorrectly formatted in yaml. The parameters also weren't completely
included in yaml and needed to be moved. Made appropriate fixes in
yaml and markdown file.
We eliminate some redundant checks.
We also consistently provide a `subscribers` field
in our stream data with `[]`, even if our users
can't access subscribers. We therefore bump
the API version and tweak the docs. (See further
down for a detailed justification of the change.)
Even though it is sometimes fine to have redundant code
that is defensive in nature, some upcoming changes are gonna
move subscriber-related logic out of build_stream_dict_for_sub
for certain codepaths as part of our effort to streamline
the payload for subscribers within page_params.
So we can't rely on the code that I removed here
inside of build_stream_dict_for_sub.
Anyway, it makes more sense to do these checks explicitly
in the validate function.
The code in build_stream_dict_for_sub was almost effectively
a noop, since the validation function was already preventing
us from getting subscriber info. The only difference it
made was sometimes converting `[]` to `None`, and then
subsequently omitting the subscribers field.
Neither ZT nor the webapp make any distinction between
`[]` or <missing key> for the `subscribers` data in
`page_params`.
The webapp has had this code for a long time (and now
equivalent code elsewhere in this PR):
if (!Object.prototype.hasOwnProperty.call(sub, "subscribers")) {
sub.subscribers = new LazySet([]);
}
The webapp calculates access based on booleans, anyway:
sub.can_access_subscribers =
page_params.is_admin || sub.subscribed ||
(!page_params.is_guest && !sub.invite_only);
And ZT would choke if `subscribers` were missing, except that
it never gets to the relevant code due to other checks:
def get_other_subscribers_in_stream(<snip>):
assert stream_id is not None or stream_name is not None
if stream_id:
assert self.is_user_subscribed_to_stream(stream_id)
return [sub
for sub in self.stream_dict[stream_id]['subscribers']
if sub != self.user_id]
else:
return [sub
for _, stream in self.stream_dict.items()
for sub in stream['subscribers']
if stream['name'] == stream_name
if sub != self.user_id]
You could make a semantic argument that we should prefer
<missing key> to `[]` when subscribers aren't even available, but
we have precedent from the way that `bulk_get_subscriber_user_ids`
has traditionally populated its result:
result: Dict[int, List[int]] =
{stream["id"]: [] for stream in stream_dicts}
If we changed `stream_dicts` to `target_stream_dicts` we
would faciliate a move toward `None`, but it would just cause
headaches for other server code as well as the frontends
(which, to reiterate, already prefer the empty array
for convenience).
Aman Agrawal
Riken Shah
Anders Kaseorg
Sahil Batra
Tim Abbott
Eeshan Garg
Zeeshan
Gaurav Pandey
Steve Howell
Hemanth V. Alluri
sahil839
Mateusz Mandera
Abhijeet Prasad Bodas
Suyash Vardhan Mathur
Suyash Vardhan Mathur
Vishnu KS
Signior-X
Adam Birds
Alex Vandiver
m-e-l-u-h-a-n
Dinesh
akshatdalton
Zeeshan Equbal
Sumanth V Rao
LoopThrough-i-j
shanukun
Tushar912