Commit Graph

13198 Commits

Author SHA1 Message Date
Abhijeet Prasad Bodas 2b9f2cc8ff mute user: Add some comments on message fetch.
These explain why we don't consider user mutes
in message fetching/unread data.
2021-04-13 09:15:49 -07:00
Abhijeet Prasad Bodas 8b098b95bb mute user: Mark as read old messages immediately.
When a user is muted, in the same request,
we mark any existing unreads from that user
as read.

This is done for all types of messages
(PM/huddle/stream) and regardless of whether
the user was mentioned in them.

This will not break the unread count logic
of the web frontend, because that algorithm
decides which messages to mark as read based
only on the pointer location and the whitespace
at the bottom, not on what messages have already
been marked as read.
2021-04-13 09:08:47 -07:00
Abhijeet Prasad Bodas 2f56f8d0ed mute user: Mark as read new messages.
Messages sent by muted users are marked as read
as soon as they are sent (or, more accurately,
while creating the database entries itself), regardless
of type (stream/huddle/PM).

ede73ee4cd, makes it easy to
pass a list to `do_send_messages` containing user-ids for
whom the message should be marked as read.
We add the contents of this list to the set of muter IDs,
and then pass it on to `create_user_messages`.

This benefits from the caching behaviour of `get_muting_users`
and should not cause performance issues long term.

The consequence is that messages sent by muted users will
not contribute to unread counts and notifications.

This commit does not affect the unread messages
(if any) present just before muting, but only handles
subsequent messages. Old unreads will be handled in
further commits.
2021-04-13 09:08:47 -07:00
Abhijeet Prasad Bodas b140c17441 mute user: Cache list of muter IDs.
This commit defines a new function `get_muting_users`
which will return a list of IDs of users who have muted
a given user.
Whenever someone mutes/unmutes  a user, the cache will be
flushed, and subsequently when that user sends a message,
the cache will be populated with the list of people who
have muted them (maybe empty).

This data is a good candidate for caching because-

1. The function will later be called from the message send
codepath, and we try to minimize database queries there.

2. The entries will be pretty tiny.

3. The entries won't churn too much. An average user will
send messages much more frequently than get muted/unmuted,
and the first time penalty of hitting the db and populating
the cache should ideally get amortized by avoiding several
DB lookups on subsequent message sends.

The actual code to call this function will be written in
further commits.
2021-04-13 09:08:47 -07:00
Abhijeet Prasad Bodas 9602aa1467 mute user: Record entries in RealmAuditLog.
This makes it so that RealmAuditLog entries are
created when a user mutes/unmutes someone.

We don't really need to store the time, but we
do so anyways, because the `event_time` field
is currently a non-nullable one in the `RealmAuditLog`
model, and making it nullable would risk allowing
not specifying the time in other more important
code which also creates `RealmAuditLog` entries.

This also fixes an incorrect test of successfully
unmuting with the API. Earlier it did not mock
the time in the `views/muting.py` code to return
`mute_time`.
2021-04-13 09:08:47 -07:00
Anders Kaseorg 1fe29aad42 queue_processors: Simplify unnecessary use of Optional.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-13 08:54:26 -07:00
Anders Kaseorg 9a2140d60e storage: Fix get_storage and remove_storage with empty keys list.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-13 08:53:03 -07:00
Zeeshan Equbal 2da4443cc5
api: Add max_message_length field to API data.
Commit 4a3ad0d introduced some extra stream-level parameters
to the `realm` object. This commit extends that to add a
max_message_length paramter too in the same server_level.
2021-04-12 16:03:31 -07:00
Tim Abbott 4a3ad0da06 api: Improve encoding of stream/topic max field lengths.
Previously, you had to request the `stream` event type in order to get
the stream-level parameters; this was a bad design in part because the
`subscription` event type has similar data and is preferred by most
clients.

So we move these to the `realm` object.  We also add the maximum topic
length, as an adjacent parameter.

While changing this, we also fix these to better match the names of
similar API parameters.
2021-04-10 10:07:57 -07:00
Tim Abbott cdbcb43706 api: Fix encoding of strings in realm endpoint.
* Don't require strings to be unnecessarily JSON-encoded.
* Use check_capped_string rather than custom code for length checks.
* Update frontend to pass the right parameters.

With a much simplified populate_data_for_request design suggested by
Anders; we only support a handful of data types, all of which are
correctly encoded automatically by jQuery.

Fixes part of #18035.
2021-04-09 16:27:30 -07:00
Abhijeet Prasad Bodas 224a8e0408 mute user: Add tests for realm import.
This covers the code added in 3bfcaa3968.
2021-04-08 23:04:28 -07:00
Abhijeet Prasad Bodas 152508e346 mute user: Reduce two database fetches when unmuting to one.
Previously, when unmuting a user, we used to make
two database fetches - one to verify that the user
is has been muted before, and one while actually
unmuting the user.

This reduces that to one, by passing around the
`MutedUser` object fetched in the first round.

Since the new function returns `Optional[MutedUser]`,
we need to use a hack for events tests, because
mypy does not yet use the type inferred from
`assert foo is not None` in nested functions like lambdas.
See python/mypy@8780d45507.
2021-04-08 23:04:28 -07:00
Abhijeet Prasad Bodas 32ab9872b1 refactor: Use API instead of functions in test_muting_users.py.
Instead of using internal functions for data setup,
we use the API so that these tests are more
end-to-end.

This commit also removes a now unnecessary
`if date_muted is None` check.
2021-04-08 23:04:28 -07:00
Abhijeet Prasad Bodas b500ff39c1 refactor: Respect example user conventions in test_muting_users.py.
This makes it consistent with the rest of the codebase.
Hamlet and Cordelia are usually the main users which do
things in automated tests.
2021-04-08 23:04:28 -07:00
Abhijeet Prasad Bodas e912bee6b6 refactor: Use variables for dates in test_muting_users.py.
This cleans up some code added in 3bfcaa3968.

Also fixes some indentation to be more readable:
- `mock.patch` is in a single line.
- Dictionaries are one field per line.
2021-04-08 23:04:28 -07:00
shanukun e65e39764a refactor: Make acting_user a mandatory kwarg for do_remove_realm_domain. 2021-04-08 17:50:10 -07:00
shanukun dd0a8738a3 refactor: Make acting_user a mandatory kwarg for do_change_notification_settings. 2021-04-08 17:50:10 -07:00
shanukun 0a21476004 refactor: Make acting_user a mandatory kwarg for do_change_default_all_public_streams. 2021-04-08 17:50:10 -07:00
shanukun 8d3c6a2feb refactor: Make acting_user a mandatory kwarg for do_change_default_event_register_stream. 2021-04-08 17:50:10 -07:00
shanukun 8f70386df3 refactor: Make acting_user a mandatory kwarg for do_change_default_sending_stream. 2021-04-08 17:50:10 -07:00
shanukun 6b8dfc2888 refactor: Make acting_user a mandatory kwarg for do_change_logo_source. 2021-04-08 17:50:10 -07:00
shanukun a3ac1a6306 refactor: Make acting_user a mandatory kwarg for do_change_icon_source. 2021-04-08 17:50:10 -07:00
shanukun 0abb493626 refactor: Make acting_user a mandatory kwarg for do_delete_avatar_image. 2021-04-08 17:50:10 -07:00
shanukun 2d336d38b9 refactor: Make acting_user a mandatory kwarg for do_change_avatar_fields. 2021-04-08 17:50:10 -07:00
shanukun 0b3324ba77 refactor: Make acting_user a mandatory kwarg for do_change_subscription_property. 2021-04-08 17:50:10 -07:00
Tim Abbott 1470dd9105 api: Delete ancient fetch_google_client_id endpoint.
This was used by the old native Zulip Android app
(zulip/zulip-android).  That app has been undeveloped for enough years
that we believe it no longer functions; as a result, there's no reason
to keep a prototype API endpoint for it (that we believe never worked).
2021-04-08 06:10:39 -07:00
Tim Abbott fdf44f19b2 api: Delete ancient generate_204 endpoint.
This endpoint was needed by the ancient pre-electron desktop app
written in QT; we removed support for that in practice a long time
ago, and even the custom error messages for it in
5a22e73cc6.

So we can delete this endpoint as well.
2021-04-08 06:10:39 -07:00
Tim Abbott 0f9970fd39 test_openapi: Update comments on undocumented endpoints. 2021-04-08 06:10:39 -07:00
Anders Kaseorg de014a30e0 dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-07 21:05:01 -07:00
sahil839 39c9845cb0 streams: Raise same error for all values of invite_to_stream_policy.
We keep the error message same for all cases when a user is not
allowed to subscribe others for all values of invite_to_stream_policy.

We raise error with different message for guest cases because it
is handled by decorators. We aim to change this behavior in future.

Explaining the details in error message isn't much important as
we do not show errors probably in API only, as we do not the show
the options itself in the frontend.
2021-04-07 14:51:00 -07:00
sahil839 1ffcb0f8b9 streams: Raise same error for all values of create_stream_policy.
We keep the error message same for all cases when a user is not
allowed to create streams for all values of create_stream_policy.

We raise error with different message for guest cases because it
is handled by decorators. We aim to change this behavior in future.

Explaining the details in error message isn't much important as
we do not show errors probably in API only, as we do not the show
the options itself in the frontend.
2021-04-07 14:51:00 -07:00
sahil839 adec45777d invite: Raise same error for all values of invite_to_realm_policy.
We keep the error message same for all cases when a user is not
allowed to invite others for all values of invite_to_realm_policy.

We raise error with different message for guest cases because it
is handled by decorators. We aim to change this behavior in future.

Explaining the details in error message isn't much important as
we do not show errors probably in API only, as we do not the show
the options itself in the frontend.
2021-04-07 14:51:00 -07:00
Anders Kaseorg f0e655f1d8 request: Rename validator parameter of REQ to json_validator.
This makes it much more clear that this feature does JSON encoding,
which previously was only indicated in the documentation.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-07 14:13:06 -07:00
Anders Kaseorg 93d2ae8092 request: Remove redundant str_validator=check_string from REQ().
REQ(str_validator=check_string) is equivalent to the default behavior
of REQ().

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-07 14:13:03 -07:00
Tim Abbott 33a732708e api docs: Record surprising required encoding for emojiset.
This fixes the API documentation tests following recent changes, but
isn't the right solution -- we probably want to change the API itself
to not require this strange JSON-encoding-of-a-string.

But this is necessary to have CI pass.
2021-04-07 12:37:51 -07:00
m-e-l-u-h-a-n 24800fca12 openapi: Add links to display settings in register documentation.
This may let us spend a bit more effort documenting these in one
place, like adding links.
2021-04-07 12:10:35 -07:00
m-e-l-u-h-a-n 079f016a35 api docs: Document display settings endpoint.
With substantial changes by tabbott to fix bugs, clarify the
documentation, and add links.

Fixes #16048.
2021-04-07 12:10:25 -07:00
Tim Abbott c6f27440c6 openapi: Fix display for boolean example values.
The comments explain in some detail, but basically we were displaying
the types for booleans incorrectly, and the types for strings in a
somewhat confusing fashion.  Fix this with comments explaining the logic.

Using JSON dumping also results in our showing strings inside
quotation marks in our examples, which seems net helpful.

Thanks to ArunSankarKs for finding where we needed to change the
codebase.

Fixes #18021.
2021-04-07 09:52:42 -07:00
Sumanth V Rao 75e539b7a7 openapi: Add feature level information for playground APIs/events.
We missed this when adding the new API endpoints/events for
configuring realm "playground" options in the series starting
with 251b415987.
2021-04-07 09:27:36 -07:00
Tim Abbott 2c9a1e085c server: Add defensive assertions for permission checks. 2021-04-07 09:22:45 -07:00
sahil839 d9fc424e12 tests: Include moderator role in events and realm update tests.
The moderator role was not included in the tests for create_stream_policy
and invite_to_stream_policy. The tests are do_set_realm_property_test
in test_events.py and do_test_realm_update_api in test_realm.py.
This should have been added for create_stream_policy in 5b32dcd and
in 5b32dcd for invite_to_stream_policy, but was missed by mistake.
2021-04-07 09:05:16 -07:00
sahil839 86f7b6af60 events: Pass can_invite_others_to_realm to client using page_params.
This commit adds backend code for passing can_invite_others_to_realm
field to clients using the fetch_initial_state_data in the page_params
object.

Though this field is not used by webapp as of now, but will be used
to fix a bug of incorreclty showing the invite users option in
settings overlay in the next commit.
2021-04-07 09:05:16 -07:00
sahil839 554ce5bf0d realm: Add moderators and full members option in invite_to_realm_policy.
We add moderators and full members option to invite_to_realm_policy
by using COMMON_POLICY_TYPES and use can_invite_others_to_realm helper
added in previous commit. This commit only does the backend work,
frontend work will be done in separate commit.
2021-04-07 09:05:16 -07:00
sahil839 d6165225c4 models: Add can_invite_others_to_realm for checking who can send invites.
This commit adds can_invite_others_to_realm helper which will be used in
further in next commit when invite_to_realm_policy will be modified to
support all values of COMMON_POLICY_TYPES.

It is important for this commit's correctness that
INVITE_TO_REALM_POLICY_TYPES was initialized to use the same values.
2021-04-07 09:04:47 -07:00
sahil839 4c8339fa8c settings: Replace invite_by_admins_policy with invite_to_realm_policy.
This commit replaces invite_by_admins_policy, which was a bool field,
with a new enum field invite_by_realm_policy.

Though the final goal is to add moderators and full members option
using COMMON_POLICY_TYPES, but this will be done in a separate
commit to make this easy for review.
2021-04-07 09:02:33 -07:00
Tim Abbott 54999f629c openapi: Fix missing example for typing notifications parameter. 2021-04-07 01:14:10 -07:00
Dinesh cd62b40f20 typing: Refactor view to support message_type.
For extending typing notifications support for streams.
2021-04-07 00:17:30 -07:00
Signior-X f7aab9c92c send_message: Ensure sender receives message events first.
This commit implements a subtle optimization (described in more detail
in the comment) that can save a few hundred milliseconds in when the
sender sees that their message has sent when sending to very large
streams.

Fixes #17898.
2021-04-06 22:51:49 -07:00
sahil839 9ad6a856a9 tests: Extract a single function to test functions using has_permission.
The tests for can_create_streams and can_subscribe_other_users shares a
lot of code and we deduplicate the code by extracting most of the code
as check_has_permission_policies which will now be called by the two
tests test_can_create_streams and test_can_subscribe_other_users.

This will also help in avoiding the duplication of code when we will
convert more policies to use COMMON_POLICY_TYPES.
2021-04-06 21:23:13 -07:00
Sumanth V Rao 1ac8fe7538 events/tests/api: Send realm_playground events to clients.
We send the whole data set as a part of the event rather than
doing an add/remove operation for couple of reasons:
    * This would make the client logic simpler.
    * The playground data is small enough for us to not worry
      about performance.

Tweaked both `fetch_initial_state_data` and `apply_events` to
handle the new playground event.

Tests added to validate the event matches the expected schema.

Documented realm_playgrounds sections inside /events and
/register to support our openapi validation system in test_events.

Tweaked other tests like test_event_system.py and test_home.py
to account for the new event being generated.

Lastly, documented the changes to the API endpoints in
api/changelog.md and bumped API_FEATURE_LEVEL.

Tweaked by tabbott to add an `id` field in RealmPlayground objects
sent to clients, which is essential to sending the API request to
remove one.
2021-04-06 20:56:58 -07:00