Commit Graph

5971 Commits

Author SHA1 Message Date
Abhijeet Prasad Bodas 152508e346 mute user: Reduce two database fetches when unmuting to one.
Previously, when unmuting a user, we used to make
two database fetches - one to verify that the user
is has been muted before, and one while actually
unmuting the user.

This reduces that to one, by passing around the
`MutedUser` object fetched in the first round.

Since the new function returns `Optional[MutedUser]`,
we need to use a hack for events tests, because
mypy does not yet use the type inferred from
`assert foo is not None` in nested functions like lambdas.
See python/mypy@8780d45507.
2021-04-08 23:04:28 -07:00
Abhijeet Prasad Bodas 32ab9872b1 refactor: Use API instead of functions in test_muting_users.py.
Instead of using internal functions for data setup,
we use the API so that these tests are more
end-to-end.

This commit also removes a now unnecessary
`if date_muted is None` check.
2021-04-08 23:04:28 -07:00
shanukun e65e39764a refactor: Make acting_user a mandatory kwarg for do_remove_realm_domain. 2021-04-08 17:50:10 -07:00
shanukun dd0a8738a3 refactor: Make acting_user a mandatory kwarg for do_change_notification_settings. 2021-04-08 17:50:10 -07:00
shanukun 0a21476004 refactor: Make acting_user a mandatory kwarg for do_change_default_all_public_streams. 2021-04-08 17:50:10 -07:00
shanukun 8d3c6a2feb refactor: Make acting_user a mandatory kwarg for do_change_default_event_register_stream. 2021-04-08 17:50:10 -07:00
shanukun 8f70386df3 refactor: Make acting_user a mandatory kwarg for do_change_default_sending_stream. 2021-04-08 17:50:10 -07:00
shanukun 6b8dfc2888 refactor: Make acting_user a mandatory kwarg for do_change_logo_source. 2021-04-08 17:50:10 -07:00
shanukun a3ac1a6306 refactor: Make acting_user a mandatory kwarg for do_change_icon_source. 2021-04-08 17:50:10 -07:00
shanukun 0abb493626 refactor: Make acting_user a mandatory kwarg for do_delete_avatar_image. 2021-04-08 17:50:10 -07:00
shanukun 2d336d38b9 refactor: Make acting_user a mandatory kwarg for do_change_avatar_fields. 2021-04-08 17:50:10 -07:00
shanukun 0b3324ba77 refactor: Make acting_user a mandatory kwarg for do_change_subscription_property. 2021-04-08 17:50:10 -07:00
sahil839 1ffcb0f8b9 streams: Raise same error for all values of create_stream_policy.
We keep the error message same for all cases when a user is not
allowed to create streams for all values of create_stream_policy.

We raise error with different message for guest cases because it
is handled by decorators. We aim to change this behavior in future.

Explaining the details in error message isn't much important as
we do not show errors probably in API only, as we do not the show
the options itself in the frontend.
2021-04-07 14:51:00 -07:00
Anders Kaseorg f0e655f1d8 request: Rename validator parameter of REQ to json_validator.
This makes it much more clear that this feature does JSON encoding,
which previously was only indicated in the documentation.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-07 14:13:06 -07:00
Tim Abbott c6f27440c6 openapi: Fix display for boolean example values.
The comments explain in some detail, but basically we were displaying
the types for booleans incorrectly, and the types for strings in a
somewhat confusing fashion.  Fix this with comments explaining the logic.

Using JSON dumping also results in our showing strings inside
quotation marks in our examples, which seems net helpful.

Thanks to ArunSankarKs for finding where we needed to change the
codebase.

Fixes #18021.
2021-04-07 09:52:42 -07:00
sahil839 86f7b6af60 events: Pass can_invite_others_to_realm to client using page_params.
This commit adds backend code for passing can_invite_others_to_realm
field to clients using the fetch_initial_state_data in the page_params
object.

Though this field is not used by webapp as of now, but will be used
to fix a bug of incorreclty showing the invite users option in
settings overlay in the next commit.
2021-04-07 09:05:16 -07:00
sahil839 554ce5bf0d realm: Add moderators and full members option in invite_to_realm_policy.
We add moderators and full members option to invite_to_realm_policy
by using COMMON_POLICY_TYPES and use can_invite_others_to_realm helper
added in previous commit. This commit only does the backend work,
frontend work will be done in separate commit.
2021-04-07 09:05:16 -07:00
sahil839 4c8339fa8c settings: Replace invite_by_admins_policy with invite_to_realm_policy.
This commit replaces invite_by_admins_policy, which was a bool field,
with a new enum field invite_by_realm_policy.

Though the final goal is to add moderators and full members option
using COMMON_POLICY_TYPES, but this will be done in a separate
commit to make this easy for review.
2021-04-07 09:02:33 -07:00
Signior-X f7aab9c92c send_message: Ensure sender receives message events first.
This commit implements a subtle optimization (described in more detail
in the comment) that can save a few hundred milliseconds in when the
sender sees that their message has sent when sending to very large
streams.

Fixes #17898.
2021-04-06 22:51:49 -07:00
sahil839 9ad6a856a9 tests: Extract a single function to test functions using has_permission.
The tests for can_create_streams and can_subscribe_other_users shares a
lot of code and we deduplicate the code by extracting most of the code
as check_has_permission_policies which will now be called by the two
tests test_can_create_streams and test_can_subscribe_other_users.

This will also help in avoiding the duplication of code when we will
convert more policies to use COMMON_POLICY_TYPES.
2021-04-06 21:23:13 -07:00
Sumanth V Rao 1ac8fe7538 events/tests/api: Send realm_playground events to clients.
We send the whole data set as a part of the event rather than
doing an add/remove operation for couple of reasons:
    * This would make the client logic simpler.
    * The playground data is small enough for us to not worry
      about performance.

Tweaked both `fetch_initial_state_data` and `apply_events` to
handle the new playground event.

Tests added to validate the event matches the expected schema.

Documented realm_playgrounds sections inside /events and
/register to support our openapi validation system in test_events.

Tweaked other tests like test_event_system.py and test_home.py
to account for the new event being generated.

Lastly, documented the changes to the API endpoints in
api/changelog.md and bumped API_FEATURE_LEVEL.

Tweaked by tabbott to add an `id` field in RealmPlayground objects
sent to clients, which is essential to sending the API request to
remove one.
2021-04-06 20:56:58 -07:00
Sumanth V Rao d2e5b62dce realm/playground: Add API endpoint for deleting a playground entry.
Similar to the previous commit, we have added a `do_*` function
which does the deletion from the DB. The next commit handles sending
the events when both adding and deleting a playground entry.

Added the openAPI format data to zulip.yaml for DELETE
/realm/playgrounds/{playground_id}. Also added python and curl
examples to remove-playground.md.

Tests added.
2021-04-07 08:20:53 +05:30
Sumanth V Rao 251b415987 realm/playground: Add API endpoint for creating playground entry.
This endpoint will allow clients to create a playground entry
containing the name, pygments language and url_prefix for the
playground of their choice.

Introduced the `do_*` function in-charge of creating the entry in
the model. Handling the process of sending events which will be
done in a follow up commit.

Added the openAPI format data to zulip.yaml for POST
/realm/playgrounds. Also added python and curl examples for using
the endpoint in its markdown documented (add-playground.md).

Tests added.
2021-04-07 08:20:53 +05:30
Sumanth V Rao 40228972b9 models/realm: Add a model for storing realm playground information.
Tweaked exports.py to add the config object there so that our export
tool can include the table when exporting. Also includes all the
changes required to import the new table from the exported data.

Helper function `get_realm_playgrounds` added to fetch all
playgrounds in a realm.

Tests amended.
2021-04-07 08:20:53 +05:30
Anders Kaseorg 8bd0ec2fca events: Include all active streams in initial state data for admins.
Realm administrators already get creation and deletion events for all
streams, including private streams.  So these should be reflected in
the initial state data.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-06 19:45:11 -07:00
Abhijeet Prasad Bodas 3bfcaa3968 mute user: Add backend infrastructure code.
Adds backend code for the mute users feature.
This is just infrastructure work (database
interactions, helpers, tests, events, API docs
etc) and does not involve any behavioral/semantic
aspects of muted users.

Adds POST and DELETE endpoints, to keep the
URL scheme mostly consistent in terms of `users/me`.

TODOs:
1. Add tests for exporting `zulip_muteduser` database table.
2. Add dedicated methods to python-zulip-api to be used
   in place of the current `client.call_endpoint` implementation.
2021-04-06 18:44:08 -07:00
PIG208 c6dfe7bf40 api: Add bot name to outgoing webhook payload.
Fixes: #12282
2021-04-06 17:37:50 -07:00
Abhijeet Prasad Bodas 52a86d9604 linkifiers: Use dictionaries for internal structures.
This change does not affect the API in anyway.
All internal code now uses dictionaries to
denote a linkifier, instead of tuples.
2021-04-05 18:16:08 -07:00
Abhijeet Prasad Bodas 68fe912c63 refactor: Rename most of "filter" to "linkifier".
After this only the database table, events,
and API endpoints remain.
2021-04-05 18:14:07 -07:00
Abhijeet Prasad Bodas f896a7667f refactor: Update some uses of "filter" to "linkifier".
This updates some comments and local variables
which could be changed without breaking other
stuff.
2021-04-05 18:14:07 -07:00
Aman Agrawal 5e83965e80 giphy: Use GIPHY web SDK to allow inserting GIFs in compose box.
We use GIPHY web SDK to create popover containing GIFs in a
grid format. Simply clicking on the GIFs will insert the GIF in the compose
box.

We add GIPHY logo to compose box action icons which opens the GIPHY
picker popover containing GIFs with "Powered by GIPHY"
attribution.
2021-04-05 15:04:49 -07:00
LoopThrough-i-j ffd0d822fe actions: Send reaction events to subscribers with history access.
Previously, if a user subscribed to a stream with
history_public_to_subscribers, and then was looking at old messages in
the stream, they would not get live-updates for that stream, because
of the structure in how notify_reaction_update only looked at
UserMessage rows (we had a previous workaround involving the
`historical` field in `UserMessage` which had already made it work if
the user themselves added the reaction).

We fix this by including all subscribers with history access in the
set of recipients for update events.

Fixes a bug that was confused with #16942.
2021-04-05 13:43:39 -07:00
LoopThrough-i-j 277fbb3f02 stream_subscription: Add subscribe_ids_with_stream_history_access.
This new function returns the set of `user_ids` with access to the
stream's full history, for use in send_event calls.
2021-04-05 13:23:11 -07:00
PIG208 e86b2c8d58 embedded bot: Use server settings for storage_size_limit. 2021-04-04 18:05:30 -07:00
Cyril Pletinckx b7fa41601d emails: Truncate overly-long From fields for RFC compatibility.
Amazon SES has a limit on the size of address fields, and rejects
emails with too-long "From" combinations of name and address. This
limit is set to 320 bytes and comes from an RFC limitation on the
size of addresses. This RFC standard states that an email address
should not be composed of a local part (before the '@') longer than
64 bytes and a domain part (after the '@') longer than 255 bytes.
It is possible that Amazon SES misinterprets this limitation as it
checks the length of the combination of the name and the email
address of the sender.

To ensure that this problem is not encountered in the send_email
module of Zulip the length of this combination is now checked
against this limit and the from_name field is removed to only
keep the from_address field when it is necessary in order to
stay below 320 bytes.

If the from_address field alone is longer than 320 bytes the
sending process will raise an SMTPDataError exception.

Tests for this new check are added to the backend test suite in
order to test if build_email correctly outputs an email with filled
from_name and from_address fields when the total length is lower
than 320 bytes and that it correctly throws the from_name field
away when necessary.

Fixes: #17558.
2021-04-03 08:13:26 -07:00
Siddharth Asthana 47e478945b support: Create RealmAuditLog when updating realm_subdomain. 2021-04-03 08:01:46 -07:00
Siddharth Asthana ddbc6d7662 support: Create RealmAuditLog when realm reactivation email is sent. 2021-04-03 08:01:46 -07:00
Siddharth Asthana c3f37c2a64 support: Pass acting_user to do_change_plan_type.
acting_user is now a mandatory field, so wherever this function is used,
we are passing acting_user as well.
2021-04-03 08:01:46 -07:00
Mateusz Mandera ba7f2a22e6 docs: Add redirects for moved pages about stream archiving.
We're renaming "stream deletion" language to "stream archiving"
and these pages were moved in the process, so we should keep redirects
for them for a while.
2021-04-02 22:08:15 -07:00
Mateusz Mandera d5871f1005 streams: Allow admins to fetch private streams via the get streams API.
We send a list of all private streams to realm admins in
fetch_initial_state since 73c30774cb
anyway and this API code just didn't catch up.
2021-04-02 16:27:50 -07:00
Anders Kaseorg ceb7e2d2bd Revert "markdown: Add support to shorten GitHub links."
This reverts commit 9c6d8d9d81 (#16916).

This feature has known bugs, and also wants some design changes to
make it customizable like linkifiers, so we’re retargeting this to
post-4.x.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-02 15:52:34 -07:00
shanukun bcc3bb03fe refactor: Make acting_user a mandatory kwarg for bulk_remove_subscriptions. 2021-04-02 14:44:41 -07:00
shanukun 0bf067b681 refactor: Make acting_user a mandatory kwarg for bulk_add_subscriptions. 2021-04-02 14:44:41 -07:00
shanukun 790085832c refactor: Make acting_user a mandatory kwarg for ensure_stream. 2021-04-02 14:44:41 -07:00
shanukun c39ffe8811 refactor: Make acting_user a mandatory kwarg for do_deactivate_stream. 2021-04-02 14:44:41 -07:00
shanukun f442e9fb03 refactor: Make acting_user a mandatory kwarg for do_scrub_realm. 2021-04-02 14:44:41 -07:00
shanukun 4b00e5da72 refactor: Make acting_user a mandatory kwarg for do_deactivate_realm. 2021-04-02 14:44:41 -07:00
shanukun 626cf52723 refactor: Make acting_user a mandatory kwarg for do_set_realm_signup_notifications_stream. 2021-04-02 14:44:41 -07:00
shanukun 00d998b955 refactor: Make acting_user a mandatory kwarg for do_set_realm_notifications_stream. 2021-04-02 14:44:41 -07:00
WookieMonkeys 1b6f68bb59 stream: Add entropy to deactivated streams.
Adding an additional `!` to the stream name each time a stream is
deactivated, to a maximum of 21 times, effectively limits number of
times a stream with a given name can be deactivated.  This is unlikely
to come up in common usage, but may be confusing when testing.

Change what we prepend to deactivated stream names to something with
more entropy than just `!`, by instead prepending a substring of hash
of the stream's ID.  `!`s.  Using 128 bits of the hash means that it
will require more than 10^18th renames to have a 1% chance of collision.

Because too-long stream names are also truncated at 60 characters,
having this entropy in the beginning of the name also helps address
potential issues from stream names that differed only in, e.g. the
60th character.

Fixes #17016.
2021-04-01 17:16:35 -07:00