Commit Graph

13214 Commits

Author SHA1 Message Date
Anders Kaseorg 5fed442bb1 templates: Remove context argument from {{#tr}} block helper.
It only had one nontrivial use, and it’s easily replaced using the
builtin {{#with}} block helper.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-13 18:59:46 -07:00
100RABHpy 2ead0fa824 openapi: Refactor the way we find uncalled curl test functions.
In "test_curl_examples.py" we find the functions that registered but
never called. To improve readablity, now we have the full
implementation in curl_param_value_generators, rather than inspecting
its fields from another module.
2021-04-13 18:21:38 -07:00
sahil839 e40ebe1a5d test_users: Refactor user role change tests to use a single helper.
Currently, there are separate tests for testing change of one role
to other, precisely 8, with most of them having similar structure
of code. This commit adds a helper function check_user_role_change
which contains all the code for testing and the tests for different
role just use this helper function to avoid duplication of code.

This refactor is helpful considering we would want to add tests
for moderators also, which would contain multiple tests for
testing changing different user roles to moderator and vice versa.

Tweaked by timabbott to make the code more readable by checking for
every user role flag instead of just checking the certain flags and
using conditionals.

Co-authored-by: Tim Abbott
2021-04-13 17:49:36 -07:00
sahil839 d8c58527cc models: Remove unused function can_access_all_realm_members.
This commit removes can_access_all_realm_members function as
it is not used anywhere in code other than tests.

This function was originally added in 4483e33102 and was
only used in digest.py other than the tests, but its use
in diget.py was removed in 735b6cb761 and the function
itself was not removed from models.py.
2021-04-13 17:49:36 -07:00
sahil839 685fbffd91 tests: Refactor check_has_permission_policies to check for all user roles.
We refactor check_has_permission_policies to check for all user roles for
each value of policy. This will help in handle a case where a guest is
allowed to do something but moderator isn't.

We need to do user_profile.refresh_from_db() in validation_func because
the realm object from user_profile is used in has_permission and we need
updated realm instance after changing the policy.

This is a follow-up commit to 9a4c58cb.
2021-04-13 17:48:23 -07:00
Anders Kaseorg d8a36d0702 i18n: Remove i18next.
Fixes #17890.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-13 17:41:10 -07:00
Anders Kaseorg 2c3787196d makemessages: Remove custom _plural handling.
Plurals are handled natively by the ICU MessageFormat syntax, so I
think we don’t have to do anything here.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-13 17:41:10 -07:00
Anders Kaseorg ef4c07356f makemessages: Adjust regex for new {{#tr}} syntax.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-13 17:41:10 -07:00
Anders Kaseorg 810ba7fce1 makemessages: Call ‘formatjs extract’ for JavaScript extraction.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-13 17:41:10 -07:00
Anders Kaseorg d0c13736c7 makemessages: Annihilate opposing \\n kludges.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-13 17:41:10 -07:00
Abhijeet Prasad Bodas 3947b0c80a linkifiers: Update API to send data using dictionaries.
* This introduces a new event type `realm_linkifiers` and
a new key for the initial data fetch of the same name.
Newer clients will be expected to use these.

* Backwards compatibility is ensured by changing neither
the current event nor the /register key. The data which
these hold is the same as before, but internally, it is
generated by processing the `realm_linkifiers` data.
We send both the old and the new event types to clients
whenever the linkifiers are changed.
Older clients will simply ignore the new event type, and
vice versa.

* The `realm/filters:GET` endpoint (which returns tuples)
is currently used by none of the official Zulip clients.
This commit replaces it with `realm/linkifiers:GET` which
returns data in the new dictionary format.
TODO: Update the `get_realm_filters` method in the API
bindings, to hit this new URL instead of the old one.

* This also updates the webapp frontend to use the newer
events and keys.
2021-04-13 12:16:07 -07:00
Arun Sankar 146b32d63a test users: Add an escape char to a test username.
Changed the name of the test-user cordelia from `Cordelia Lear` to
`Cordelia, Lear's daughter`.

This change will enable us to test users with escape characters in
their names.

I also updated the Node, Puppeteer, Backend tests and Fixtures to
support this change.
2021-04-13 11:42:06 -07:00
Tim Abbott 4e8a5f4da8 test_signup: Fix test making DNS query to public Internet.
This appears to be the result of our having copied parts of the
previous test without copying its DNS query decorator.
2021-04-13 10:47:35 -07:00
Anders Kaseorg b01d43f339 mypy: Fix strict_equality violations.
puppet/zulip/files/nagios_plugins/zulip_postgresql/check_postgresql_replication_lag:98: error: Non-overlapping equality check (left operand type: "List[List[str]]", right operand type: "Literal[0]")  [comparison-overlap]
zerver/tests/test_realm.py:650: error: Non-overlapping container check (element type: "Dict[str, Any]", container item type: "str")  [comparison-overlap]

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-13 09:18:18 -07:00
Tim Abbott 2e928a0853 markdown: Remove logic for creating markdown engines for all realms.
This logic likely never ran due to a combination of bugs.

* Running `maybe_update_markdown_engines` unconditionally meant that
  `if md_engine_key in md_engines` was likely always true.
* Introduced in 65838bb: DEFAULT_MARKDOWN_KEY could never be in
  md_engines, so should we have ever reached that code path, we'd have
  tried to rebuild all markdown engines every time.

And it also wasn't clearly helpful -- because we fetch all linkifiers
for a realm on every request anyway, we don't really save database
queries by doing a bulk fetch on startup, and doing so would likely
result in a material regression to Zulip's overall startup time that
we were creating markdown engines for large numbers of realms in bulk
during process startup.
2021-04-13 09:18:18 -07:00
Tim Abbott c6a50499f7 management: Use better fetch query in edit_linkifiers. 2021-04-13 09:18:18 -07:00
Abhijeet Prasad Bodas 2b9f2cc8ff mute user: Add some comments on message fetch.
These explain why we don't consider user mutes
in message fetching/unread data.
2021-04-13 09:15:49 -07:00
Abhijeet Prasad Bodas 8b098b95bb mute user: Mark as read old messages immediately.
When a user is muted, in the same request,
we mark any existing unreads from that user
as read.

This is done for all types of messages
(PM/huddle/stream) and regardless of whether
the user was mentioned in them.

This will not break the unread count logic
of the web frontend, because that algorithm
decides which messages to mark as read based
only on the pointer location and the whitespace
at the bottom, not on what messages have already
been marked as read.
2021-04-13 09:08:47 -07:00
Abhijeet Prasad Bodas 2f56f8d0ed mute user: Mark as read new messages.
Messages sent by muted users are marked as read
as soon as they are sent (or, more accurately,
while creating the database entries itself), regardless
of type (stream/huddle/PM).

ede73ee4cd, makes it easy to
pass a list to `do_send_messages` containing user-ids for
whom the message should be marked as read.
We add the contents of this list to the set of muter IDs,
and then pass it on to `create_user_messages`.

This benefits from the caching behaviour of `get_muting_users`
and should not cause performance issues long term.

The consequence is that messages sent by muted users will
not contribute to unread counts and notifications.

This commit does not affect the unread messages
(if any) present just before muting, but only handles
subsequent messages. Old unreads will be handled in
further commits.
2021-04-13 09:08:47 -07:00
Abhijeet Prasad Bodas b140c17441 mute user: Cache list of muter IDs.
This commit defines a new function `get_muting_users`
which will return a list of IDs of users who have muted
a given user.
Whenever someone mutes/unmutes  a user, the cache will be
flushed, and subsequently when that user sends a message,
the cache will be populated with the list of people who
have muted them (maybe empty).

This data is a good candidate for caching because-

1. The function will later be called from the message send
codepath, and we try to minimize database queries there.

2. The entries will be pretty tiny.

3. The entries won't churn too much. An average user will
send messages much more frequently than get muted/unmuted,
and the first time penalty of hitting the db and populating
the cache should ideally get amortized by avoiding several
DB lookups on subsequent message sends.

The actual code to call this function will be written in
further commits.
2021-04-13 09:08:47 -07:00
Abhijeet Prasad Bodas 9602aa1467 mute user: Record entries in RealmAuditLog.
This makes it so that RealmAuditLog entries are
created when a user mutes/unmutes someone.

We don't really need to store the time, but we
do so anyways, because the `event_time` field
is currently a non-nullable one in the `RealmAuditLog`
model, and making it nullable would risk allowing
not specifying the time in other more important
code which also creates `RealmAuditLog` entries.

This also fixes an incorrect test of successfully
unmuting with the API. Earlier it did not mock
the time in the `views/muting.py` code to return
`mute_time`.
2021-04-13 09:08:47 -07:00
Anders Kaseorg 1fe29aad42 queue_processors: Simplify unnecessary use of Optional.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-13 08:54:26 -07:00
Anders Kaseorg 9a2140d60e storage: Fix get_storage and remove_storage with empty keys list.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-13 08:53:03 -07:00
Zeeshan Equbal 2da4443cc5
api: Add max_message_length field to API data.
Commit 4a3ad0d introduced some extra stream-level parameters
to the `realm` object. This commit extends that to add a
max_message_length paramter too in the same server_level.
2021-04-12 16:03:31 -07:00
Tim Abbott 4a3ad0da06 api: Improve encoding of stream/topic max field lengths.
Previously, you had to request the `stream` event type in order to get
the stream-level parameters; this was a bad design in part because the
`subscription` event type has similar data and is preferred by most
clients.

So we move these to the `realm` object.  We also add the maximum topic
length, as an adjacent parameter.

While changing this, we also fix these to better match the names of
similar API parameters.
2021-04-10 10:07:57 -07:00
Tim Abbott cdbcb43706 api: Fix encoding of strings in realm endpoint.
* Don't require strings to be unnecessarily JSON-encoded.
* Use check_capped_string rather than custom code for length checks.
* Update frontend to pass the right parameters.

With a much simplified populate_data_for_request design suggested by
Anders; we only support a handful of data types, all of which are
correctly encoded automatically by jQuery.

Fixes part of #18035.
2021-04-09 16:27:30 -07:00
Abhijeet Prasad Bodas 224a8e0408 mute user: Add tests for realm import.
This covers the code added in 3bfcaa3968.
2021-04-08 23:04:28 -07:00
Abhijeet Prasad Bodas 152508e346 mute user: Reduce two database fetches when unmuting to one.
Previously, when unmuting a user, we used to make
two database fetches - one to verify that the user
is has been muted before, and one while actually
unmuting the user.

This reduces that to one, by passing around the
`MutedUser` object fetched in the first round.

Since the new function returns `Optional[MutedUser]`,
we need to use a hack for events tests, because
mypy does not yet use the type inferred from
`assert foo is not None` in nested functions like lambdas.
See python/mypy@8780d45507.
2021-04-08 23:04:28 -07:00
Abhijeet Prasad Bodas 32ab9872b1 refactor: Use API instead of functions in test_muting_users.py.
Instead of using internal functions for data setup,
we use the API so that these tests are more
end-to-end.

This commit also removes a now unnecessary
`if date_muted is None` check.
2021-04-08 23:04:28 -07:00
Abhijeet Prasad Bodas b500ff39c1 refactor: Respect example user conventions in test_muting_users.py.
This makes it consistent with the rest of the codebase.
Hamlet and Cordelia are usually the main users which do
things in automated tests.
2021-04-08 23:04:28 -07:00
Abhijeet Prasad Bodas e912bee6b6 refactor: Use variables for dates in test_muting_users.py.
This cleans up some code added in 3bfcaa3968.

Also fixes some indentation to be more readable:
- `mock.patch` is in a single line.
- Dictionaries are one field per line.
2021-04-08 23:04:28 -07:00
shanukun e65e39764a refactor: Make acting_user a mandatory kwarg for do_remove_realm_domain. 2021-04-08 17:50:10 -07:00
shanukun dd0a8738a3 refactor: Make acting_user a mandatory kwarg for do_change_notification_settings. 2021-04-08 17:50:10 -07:00
shanukun 0a21476004 refactor: Make acting_user a mandatory kwarg for do_change_default_all_public_streams. 2021-04-08 17:50:10 -07:00
shanukun 8d3c6a2feb refactor: Make acting_user a mandatory kwarg for do_change_default_event_register_stream. 2021-04-08 17:50:10 -07:00
shanukun 8f70386df3 refactor: Make acting_user a mandatory kwarg for do_change_default_sending_stream. 2021-04-08 17:50:10 -07:00
shanukun 6b8dfc2888 refactor: Make acting_user a mandatory kwarg for do_change_logo_source. 2021-04-08 17:50:10 -07:00
shanukun a3ac1a6306 refactor: Make acting_user a mandatory kwarg for do_change_icon_source. 2021-04-08 17:50:10 -07:00
shanukun 0abb493626 refactor: Make acting_user a mandatory kwarg for do_delete_avatar_image. 2021-04-08 17:50:10 -07:00
shanukun 2d336d38b9 refactor: Make acting_user a mandatory kwarg for do_change_avatar_fields. 2021-04-08 17:50:10 -07:00
shanukun 0b3324ba77 refactor: Make acting_user a mandatory kwarg for do_change_subscription_property. 2021-04-08 17:50:10 -07:00
Tim Abbott 1470dd9105 api: Delete ancient fetch_google_client_id endpoint.
This was used by the old native Zulip Android app
(zulip/zulip-android).  That app has been undeveloped for enough years
that we believe it no longer functions; as a result, there's no reason
to keep a prototype API endpoint for it (that we believe never worked).
2021-04-08 06:10:39 -07:00
Tim Abbott fdf44f19b2 api: Delete ancient generate_204 endpoint.
This endpoint was needed by the ancient pre-electron desktop app
written in QT; we removed support for that in practice a long time
ago, and even the custom error messages for it in
5a22e73cc6.

So we can delete this endpoint as well.
2021-04-08 06:10:39 -07:00
Tim Abbott 0f9970fd39 test_openapi: Update comments on undocumented endpoints. 2021-04-08 06:10:39 -07:00
Anders Kaseorg de014a30e0 dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-07 21:05:01 -07:00
sahil839 39c9845cb0 streams: Raise same error for all values of invite_to_stream_policy.
We keep the error message same for all cases when a user is not
allowed to subscribe others for all values of invite_to_stream_policy.

We raise error with different message for guest cases because it
is handled by decorators. We aim to change this behavior in future.

Explaining the details in error message isn't much important as
we do not show errors probably in API only, as we do not the show
the options itself in the frontend.
2021-04-07 14:51:00 -07:00
sahil839 1ffcb0f8b9 streams: Raise same error for all values of create_stream_policy.
We keep the error message same for all cases when a user is not
allowed to create streams for all values of create_stream_policy.

We raise error with different message for guest cases because it
is handled by decorators. We aim to change this behavior in future.

Explaining the details in error message isn't much important as
we do not show errors probably in API only, as we do not the show
the options itself in the frontend.
2021-04-07 14:51:00 -07:00
sahil839 adec45777d invite: Raise same error for all values of invite_to_realm_policy.
We keep the error message same for all cases when a user is not
allowed to invite others for all values of invite_to_realm_policy.

We raise error with different message for guest cases because it
is handled by decorators. We aim to change this behavior in future.

Explaining the details in error message isn't much important as
we do not show errors probably in API only, as we do not the show
the options itself in the frontend.
2021-04-07 14:51:00 -07:00
Anders Kaseorg f0e655f1d8 request: Rename validator parameter of REQ to json_validator.
This makes it much more clear that this feature does JSON encoding,
which previously was only indicated in the documentation.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-07 14:13:06 -07:00
Anders Kaseorg 93d2ae8092 request: Remove redundant str_validator=check_string from REQ().
REQ(str_validator=check_string) is equivalent to the default behavior
of REQ().

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-07 14:13:03 -07:00