Commit Graph

87 Commits

Author SHA1 Message Date
Tim Abbott 64aa8f80a0 events: Add heartbeat events to tests and documentation.
Heartbeat events are an important part of the API, even though they
are noops, so it's important to document them.
2021-07-01 17:14:32 -07:00
sahil839 828759d2ba models: Replace allow_community_topic_editing with edit_topic_policy.
This commit replaces the allow_community_topic_editing boolean with
integer field edit_topic_policy and includes both frontend and
backend changes.

We also update settings_ui.disable_sub_settings_onchange to not
change the color of label as we did previously when the setting
was a checkbox. But now as the setting is dropdown we keep the
label as it is and we don't do anything with label when disabling
dropdowns. Also, this function was used only here so we can safely
change this.
2021-06-16 14:59:36 -07:00
Vishnu KS 5db53029a5 api: Include is_billing_admin as an attribute in user response.
This is sufficiently useful that it should be made available to clients.
2021-06-03 10:27:07 -07:00
Aman Agrawal 70c0abc2e5 do_change_stream_invite_only: Ensure stream is not web public.
When changing stream permissions to invite_only or public, ensure
that stream doesn't have is_web_public set to True.
2021-05-20 15:23:19 -07:00
Anders Kaseorg e015f3ed7d docs: Correct “webapp” to “web app”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-14 13:15:38 -07:00
sahil839 34f134d58d users: Add role field to user objects returned by format_user_row.
This commit modifies the user objects returned by 'GET /users',
'GET /users/me', 'GET /users/{user_id}' and 'GET /users/{email}'
endpoints to include role field.

We also include role field in the page_params['realm_users'] dict
and in the person object sent in (type="realm_user", op="add")
event.
2021-04-29 15:13:50 -07:00
m-e-l-u-h-a-n 65c400e06d api: Add zulip_version and zulip_feature_level in restart event.
This help mobile and terminal clients understand whether a server
restart changed API feature levels or not, which in turn determines
whether they will need to resynchronize their data.

Also add tests and documentation for this previously undocumented
event type.

Fixes: #18205.
2021-04-29 12:08:15 -07:00
Tim Abbott 1afc0496ca event_schema: Fix translation of typing events message_type.
This was incorrectly/lazily marked as a string, not as an enum of two
string values, in the previous commit.
2021-04-27 21:35:32 -07:00
Dinesh 27e4f5da92 typing: Support sending stream/topic typing status.
This extends the /json/typing endpoint to also accept
stream_id and topic. With this change, the requests
sent to /json/typing should have these:
* `to`: a list set to
    - recipients for a PM
    - stream_id for a stream message
* `topic`, in case of stream message
along with `op`(start or stop).

On receiving a request with stream_id and topic, we send
typing events to clients with stream_typing_notifications set
to True for all users subscribed to that stream.
2021-04-27 20:52:21 -07:00
Abhijeet Prasad Bodas 3947b0c80a linkifiers: Update API to send data using dictionaries.
* This introduces a new event type `realm_linkifiers` and
a new key for the initial data fetch of the same name.
Newer clients will be expected to use these.

* Backwards compatibility is ensured by changing neither
the current event nor the /register key. The data which
these hold is the same as before, but internally, it is
generated by processing the `realm_linkifiers` data.
We send both the old and the new event types to clients
whenever the linkifiers are changed.
Older clients will simply ignore the new event type, and
vice versa.

* The `realm/filters:GET` endpoint (which returns tuples)
is currently used by none of the official Zulip clients.
This commit replaces it with `realm/linkifiers:GET` which
returns data in the new dictionary format.
TODO: Update the `get_realm_filters` method in the API
bindings, to hit this new URL instead of the old one.

* This also updates the webapp frontend to use the newer
events and keys.
2021-04-13 12:16:07 -07:00
Sumanth V Rao 1ac8fe7538 events/tests/api: Send realm_playground events to clients.
We send the whole data set as a part of the event rather than
doing an add/remove operation for couple of reasons:
    * This would make the client logic simpler.
    * The playground data is small enough for us to not worry
      about performance.

Tweaked both `fetch_initial_state_data` and `apply_events` to
handle the new playground event.

Tests added to validate the event matches the expected schema.

Documented realm_playgrounds sections inside /events and
/register to support our openapi validation system in test_events.

Tweaked other tests like test_event_system.py and test_home.py
to account for the new event being generated.

Lastly, documented the changes to the API endpoints in
api/changelog.md and bumped API_FEATURE_LEVEL.

Tweaked by tabbott to add an `id` field in RealmPlayground objects
sent to clients, which is essential to sending the API request to
remove one.
2021-04-06 20:56:58 -07:00
Abhijeet Prasad Bodas 3bfcaa3968 mute user: Add backend infrastructure code.
Adds backend code for the mute users feature.
This is just infrastructure work (database
interactions, helpers, tests, events, API docs
etc) and does not involve any behavioral/semantic
aspects of muted users.

Adds POST and DELETE endpoints, to keep the
URL scheme mostly consistent in terms of `users/me`.

TODOs:
1. Add tests for exporting `zulip_muteduser` database table.
2. Add dedicated methods to python-zulip-api to be used
   in place of the current `client.call_endpoint` implementation.
2021-04-06 18:44:08 -07:00
Sumanth V Rao e12f682e2e markdown: Include text & url in `topic_links` parameter of our API.
The linkifier code now includes both the shortened text and the expanded
URL, sorted by the order of the occurrence in a topic. This list is passed
back in the `topic_links` parameter of the /messages and the /events APIs.

topic_links earlier vs now:

earlier: ['https://www.google.com', 'https://github.com/zulip/zulip/32']

now: [{'url': 'https://www.google.com', 'text': 'https://www.google/com},
      {'url': 'https://github.com/zulip/zulip/32', 'text': '#32'}]

Similarly, the topic_links local echo logic in the frontend now returns
back an object.

Fixes: #17109.
2021-03-30 15:53:07 -07:00
shanukun bc2d58ad4a custom_profile_fields: Remove op field for the event.
* `op` (operation) field, added in f6fb88549f, was never intended for
`custom_profile_fields` event. This commit removes the `op` as it doesn't
have any use in the code.

* As a part of cleanup, this also eliminates the schema check warnings
for `custom_profile_fields` event, mentioned in #17568.
2021-03-26 16:28:33 -07:00
shanukun d68a2677d2 event_schema: Add checker for restart event.
Part of #17568.
2021-03-23 12:16:20 -07:00
shanukun cfe0fa3788 event_schema: Add schema check for realm/deactivated event.
This add the schema checker, openapi schema, and also a test for
realm/deactivated event.

With several block comments by tabbott explaining the logic behind our
behavior here.

Part of #17568.
2021-03-23 12:16:16 -07:00
sahil839 9a432b0c3b events: Remove name field from update subscription events.
This commit removes name field from update subscription
events, as it is not used by any of the clients, and use
stream_id in the events code instead.
2021-03-07 22:03:24 -08:00
sahil839 b53c773987 events: Remove email field from update subscription events.
This commit removes email field from update subscription
events, as email field is of no use in this case.
2021-03-01 14:52:06 -08:00
Anders Kaseorg 11741543da python: Reformat with Black, except quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Steve Howell 1498b2ef69 apply_event: Fix broken deepcopy attempt for subs.
When we were getting an apply_event call for
a subscription/add event, we were trying not to
mutate the event itself, but this clumsy code
was still mutating the actual event:

    # Avoid letting 'subscribers' entries end up in the list
    for i, sub in enumerate(event['subscriptions']):
        event['subscriptions'][i] = \
            copy.deepcopy(event['subscriptions'][i])
        del event['subscriptions'][i]['subscribers']

This is only a theoretical bug.

The only person who receives a subscription/add
event is the current user.

And it wouldn't have affected the current user,
since the apply_event was correctly updating the
state, and we wouldn't actually deliver the event
to the client (because the whole point of apply_event
is to prevent us from having to piggyback the
super-recent events on to our payload or put
them into the event queue and possibly race).

The new code just cleanly makes a copy of each
sub, if necessary, as we add them to state["subscriptions"].

And I updated the event schemas to reflect that
subscribers is always present in subscription/add
event.

Long term we should probably avoid sending subscribers
on this event when the clients don't set something
like include_subscribers.  That's a fairly complicated
fix that involves passing in flags to ClientDescriptor.
Alternatively, we could just say that our policy is
that we never send subscribers there, but we instead
use peer_add events.  See issue #17089 for more
details.
2021-01-21 15:04:07 -08:00
Steve Howell c693ae8982 event tests: Cover do_update_user_status better.
We often send only one field (away or status_text)
to be updated.

So we have to make our schema support optional
keys.

As a result of the more flexible schema, we no
longer need to exempt the node fixtures from
our schema checks.
2021-01-20 13:17:32 -08:00
Steve Howell 7ff3859136 subscriber events: Change schema for peer_add/peer_remove.
We now can send an implied matrix of user/stream tuples
for peer_add and peer_remove events.

The client code basically does this:

    for stream_id in event['stream_ids']:
        for user_id in event['user_ids']:
            update_sub(stream_id, user_id)

We used to send individual events, which gets real
expensive when you are creating new streams. For
the case of copy-to-stream case, we should see
events go from U to 1, where U is the number of users
added.

Note that we don't yet fully optimize the potential
of this schema.  For adding a new user with lots
of default streams, we still send S peer_add events.

And if you subscribe a bunch of users to a bunch of
private streams, we only go from U * S to S; we can't
optimize it down to one event easily.
2020-10-22 11:19:53 -07:00
Steve Howell def3dac6ae event_schema: Add comments to top of the file.
The comments basically explain the common coding
patterns for making the checkers.
2020-09-28 12:19:28 -07:00
Tim Abbott 94a9fa1891 event_schema: Add documentation and rename a few functions.
This should help make this revised subsystem readable for more new
contributors.  We still need to make updates to the high-level
documentation.
2020-09-25 12:53:00 -07:00
Steve Howell 5b7c9c4714 test_events: Add check_realm_user_remove. 2020-09-25 11:43:20 -07:00
Steve Howell 7bb7f2943f event_schema: Finish extraction with realm_emoji/update.
We now no longer define any schemas in test_events--all
of them are in event_schema, which helps our tooling
cross-check schemas for openapi and node tests.
2020-09-25 11:43:20 -07:00
Steve Howell ae4d083a5a event_schema: Extract check_realm_domains_*. 2020-09-25 11:43:20 -07:00
Steve Howell 298bed9fa1 event_schema: Split check_update_message_flags. 2020-09-25 11:43:20 -07:00
Steve Howell f6e0171d02 event_schema: Split check_reaction into add/remove.
It happens that whether you add a reaction or remove
a reaction, we send the exact same fields, just using
a different op code.

This sort of symmetry is actually kind of rare, as
usually "add" events have more fields, and "remove" events
might just send an id of something to remove.

Our openapi schema treats these as two seperate events,
so we are more consistent with it, and it helps our
schema-checking tooling for node fixtures, too.

Note that we now have to exempt the two events from
our openapi checks, due to the is_mirror_dummy field
in the deprecated user block.  We can decide how to
handle this later--one possibility is to just add it
as an optional field on the event_schema side.
2020-09-25 11:43:20 -07:00
Steve Howell b7b2546f44 event_schema: Extract check_subscription_update.
Note that we use value_type for value instead of
bool, since properties can be non-bool things
like color, which we just don't test now.  We
should test them.

We more than compensate for this by checking
the actual value of the value in
check_subscription_update.
2020-09-25 11:43:20 -07:00
Steve Howell b920ebce81 event_schema: Extract check_has_zoom_token. 2020-09-25 11:43:20 -07:00
Steve Howell 0c4286222f event_schema: Extract check_realm_update_dict. 2020-09-25 11:43:20 -07:00
Steve Howell 6ec6525624 event_schema: Extract check_delete_message.
There is a legacy format where we send
singular "message_id" instead of plural
"message_ids".

Then there are different fields for "private"
and "stream" message types.
2020-09-25 11:43:20 -07:00
Steve Howell 88165aee6b event_schema: Extract check_user_group_update. 2020-09-25 11:43:20 -07:00
Steve Howell aaaac11661 event_schema: Extract check_user_group_remove. 2020-09-25 11:43:20 -07:00
Steve Howell 1b7af13f37 event_schema: Extract check_user_group_remove_members. 2020-09-25 11:43:20 -07:00
Steve Howell 19b7739065 event_schema: Extract check_user_group_add_members. 2020-09-25 11:43:20 -07:00
Steve Howell 4084f0b949 event_schema: Extract check_realm_user_add.
Note that we make the schema for profile_data
slightly more realistic, but it doesn't actually get
exercised by our current tests (apart from
making sure it's a dict), since we don't have
profile data for our test realm.

We also don't have the optional fields for bots,
since our tests don't exercise that, nor
delivery_email.

So we exempt realm_user_add_event from openapi
checks for now.

When we try to match the openapi specs better, we
will probably want to add a few tests to test_events.

Obviously getting good coverage for adding users
would be nice for all these scenarios:

    * delivery_email matters
    * bots
    * realm has profile fields
2020-09-25 11:43:19 -07:00
Steve Howell dc2176a965 event_schema: Extract check_presence. 2020-09-25 11:43:19 -07:00
Steve Howell 6c74a44697 data_types: Generalize StringDictType.
This is a prep commit for supporting "presence"
events, where the key of the dictionary is some
arbitrary string like "website" but the value
of the dictionary is another dictionary itself
with keys that are more like variable names.
2020-09-25 11:43:19 -07:00
Steve Howell 4f3d5f2d87 event_schema: Extract check_realm_filters.
We have some known issues with representing
tuples in openapi, so we exempt realm_filters
from the relevant check.
2020-09-25 11:43:19 -07:00
Steve Howell e40a5400e5 event_schema: Extract check_muted_topics.
This also forces us to create TupleType.

We exempt this from the openapi check,
since we haven't figured out how to model
tuples in openapi with the same precision
as event_schema (and it may be impossible).

Long term we just want to stop dealing in
tuples, of course.
2020-09-25 11:43:19 -07:00
orientor 91ca1afe98 data_type: Add StringDict data type.
StringDict is a data type for representing dictionaries where
all keys and values are strings. Add this data type to data_types.py
and edit other files so that this data type is put to use and tested.

(slightly tweaked by @showell to remove a comment and shorten
a var name now that we have a proper data type)
2020-09-25 11:43:19 -07:00
Steve Howell 78a2059b8d event schema: Extract attachment checkers. 2020-09-25 11:43:19 -07:00
Steve Howell 4a947c971d event_schema: Extract check_realm_export.
These are all trivial transformations.

Note that we don't insist timestamps are
floats; the NumberType class allows ints
too.
2020-09-25 11:43:19 -07:00
Steve Howell d28c01284c event_schema: Extract check_hotspots.
This forces us to introduce a NumberType.
2020-09-25 11:43:19 -07:00
Steve Howell cf26151cea event_schema: Use realm_user_person_types.
For realm_user events, we now structure the
person type as a union of dicts, which is
more consistent with how we model this in
our openapi spec.
2020-09-25 11:43:19 -07:00
Steve Howell 10952394b0 test_events: Use int value of message_retention_days.
We also make our schema in event_schema reflect this,
which in turn makes us match the already accurate
openapi spec, so we no longer need to exempt four
types of events from our sanity checks.
2020-09-25 11:43:19 -07:00
Steve Howell 2b76eb767f event_schema: Change propagate_mode to an enum. 2020-09-25 11:43:19 -07:00
Steve Howell aca641a4d1 refactor: Extract data_types module.
Defining types with an object hierarchy
of type classes will allow us to build
functionality that was impossible (or
really janky) with the validators.py
approach of composing functions.

Most of the changes to event_schema.py
were automated search/replaces.

This patch doesn't really yet take
advantage of the new FooType classes,
but we will use it soon to audit our
openapi specs.
2020-09-25 11:43:19 -07:00