Commit Graph

14078 Commits

Author SHA1 Message Date
rht 1bbd36d181 slack_import: Remove obsolete SlackImportAttachment placeholder.
This was introduced in f4ad464d82, and
incompletely removed in e037c2f93e649c28a71c02559b5ae7a3333f42a8; here
we finish removing it.
2021-08-02 13:13:28 -07:00
rht 9441bca282 import_realm: Explain the usage of the relative_path var of attachment_path. 2021-08-02 13:13:19 -07:00
Tim Abbott 24b1439e93 email_notifications: Avoid trying to email to bot users.
This fixes a regression in de04f0ad67.

We'll do a proper test in a follow-up commit; this is a quick fix to
make sure master works.

The emails will bounce, but it'll create all sorts of infrastructure
headaches.
2021-08-02 13:13:01 -07:00
Sahil Batra 93b51088c0 api: Fix "Changes" entry for enter_sends in zulip.yaml.
This commit fixes the feature level for enter_sends in
`/regiser` response and also modifies the "Changes"
entry to match with other settings.
2021-08-02 09:23:56 -07:00
Sahil Batra c3f31bd507 api: Fix feature level for zulip_merge_base in zulip.yaml. 2021-08-02 09:23:56 -07:00
Tim Abbott d1dd34d7e0 send_custom_email: Add option for sending marketing emails. 2021-08-01 21:45:34 -07:00
Aman Agrawal c1c6674809 emails: Retheme to purple colors to match present logo. 2021-08-01 21:27:42 -07:00
Tim Abbott 65d60f4a25 api docs: Improve documentation for user_settings transition. 2021-08-01 15:30:40 -07:00
Sahil Batra 998d710275 frontend: Add new user_settings module for user's settings.
We add a new user_settings module similar to page_params
module in frontend and use it to access user's personal
settings instead of page_params.
2021-08-01 15:30:17 -07:00
Sahil Batra e6f828a8e2 register: Send user_settings in response based on client capabilities.
We added "user_settings" object containing all the user settings in
previous commit. This commit modifies the code to send the existing
setting fields in the top-level object only if user_settings_object
client_capabilities field is False.
2021-08-01 15:30:17 -07:00
Sahil Batra 84742a040e api: Add "user_settings" object to '/register' response.
We add a "user_settings" object, containing all the user
settings, in the '/register' endpoint response for
better readability.
2021-08-01 15:30:17 -07:00
Sahil Batra 0364d0c8ca events: Add "user_settings_object" to client_capabilities.
This commit adds "user_settings_object" field to
client_capabilities which will be used to determine
if the client needs 'update_display_settings' and
'update_global_notifications' event.
2021-08-01 15:30:17 -07:00
Sahil Batra 7959ae3fab events: Add new event type 'user_settings' for updating user settings.
We send a event with type 'user_settings' on updating user's display
and notification settings.

The old event types - 'update_global_notifications' and
'update_display_settings', are still supported for backwards
compatibility.
2021-08-01 15:30:17 -07:00
Sahil Batra e73d2fff97 tests: Remove separate event tests for "enable_drafts_synchronization".
We do not require separate tests for checking events when changing
"enable_drafts_synchronization" as we already do this in the display
settings test because this setting is included in property_types.
2021-08-01 15:30:17 -07:00
Anders Kaseorg aeb0dc7269 video_calls: Support separate testing credentials for Zoom.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-01 14:26:46 -07:00
Gaurav Pandey d89b4dcd0b api: Return zulip_merge_base alongside zulip_version.
Return zulip_merge_base alongside zulip_version
in `/register`, `/event` and `/server_settings`
endpoint so that the value can be used by other
clients.
2021-07-30 12:03:41 -07:00
Hemanth V. Alluri 6a3e98d14b drafts: Add API documentation for the core drafts endpoints.
These were added at some point in the past, but were not complete, and
it makes sense to document the current feature level as and when they
become available, since clients should not use the drafts endpoints on
older feature levels.
2021-07-30 10:41:52 -07:00
Tim Abbott 472c55a1ff populate_db: Create some drafts in the initial database. 2021-07-30 10:41:52 -07:00
Hemanth V. Alluri 6fee946a43 drafts: Send events to clients when drafts change.
With this, the core of the new drafts system is complete.
2021-07-30 10:41:52 -07:00
Hemanth V. Alluri c00089ac28 drafts: Add support for toggling drafts synchronization.
With changes mostly to the API documentation by tabbott.
2021-07-30 10:00:27 -07:00
Anders Kaseorg 1f5fc12989 video_calls: Remove call to deprecated Zoom Data Compliance API.
https://marketplace.zoom.us/docs/guides/stay-up-to-date/announcements#data-compliance

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-30 09:19:22 -07:00
Suyash Vardhan Mathur ea6b21cd8c openapi: Document update-status endpoint. 2021-07-29 11:50:25 -07:00
Hemanth V. Alluri f1f0a26c37 drafts: Extract all of the real work to lib functions.
The main reason why this is needed is because this seems to be
convention and because we can't easily test event creation without
doing this.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2021-07-28 16:33:01 -07:00
Priyansh Garg 044fe547d3 data_import: Add huddle import support for Rocket.Chat. 2021-07-28 15:45:54 -07:00
Priyansh Garg 24dd0ff96c data_import: Add rocket chat import tool.
This commit allows to import the following from rocketchat:
* All users
* All public/private channels
* All teams and its public/private channels
* All discussion rooms as topics in their parent channel
* All the messages in all the channels
* All private conversations
* Reactions on messages (except for custom emojis)
* Mentions in messages (except @all, @here mentions)
2021-07-28 15:28:56 -07:00
Mateusz Mandera 4161e0caeb rate_limit: Show html page when rate limited at /new/ endpoint.
Previously this showed a json error, but this is an endpoint that human
users use in the browser, so a proper HTML page is more appropriate.
2021-07-28 15:24:04 -07:00
Abhijeet Prasad Bodas de04f0ad67 notifications: Calculate PMs/mentions settings like other settings.
Previously, we checked for the `enable_offline_email_notifications` and
`enable_offline_push_notifications` settings (which determine whether the
user will receive notifications for PMs and mentions) just before sending
notifications. This has a few problem:

1. We do not have access to all the user settings in the notification
handlers (`handle_missedmessage_emails` and `handle_push_notifications`),
and therefore, we cannot correctly determine whether the notification should
be sent. Checks like the following which existed previously, will, for
example, incorrectly not send notifications even when stream email
notifications are enabled-
```
if not receives_offline_email_notifications(user_profile):
    return
```
With this commit, we simply do not enqueue notifications if the "offline"
settings are disabled, which fixes that bug.

Additionally, this also fixes a bug with the "online push notifications"
feature, which was, if someone were to:
* turn off notifications for PMs and mentions (`enable_offline_push_notifications`)
* turn on stream push notifications (`enable_stream_push_notifications`)
* turn on "online push" (`enable_online_push_notifications`)

then, they would still receive notifications for PMs when online.
This isn't how the "online push enabled" feature is supposed to work;
it should only act as a wrapper around the other notification settings.

The buggy code was this in `handle_push_notifications`:
```
if not (
    receives_offline_push_notifications(user_profile)
    or receives_online_push_notifications(user_profile)
):
    return

    // send notifications
```

This commit removes that code, and extends our `notification_data.py` logic
to cover this case, along with tests.

2. The name for these settings is slightly misleading. They essentially
talk about "what to send notifications for" (PMs and mentions), and not
"when to send notifications" (offline). This commit improves this condition
by restricting the use of this term only to the database field, and using
clearer names everywhere else. This distinction will be important to have
non-confusing code when we implement multiple options for notifications
in the future as dropdown (never/when offline/when offline or online, etc).

3. We should ideally re-check all notification settings just before the
notifications are sent. This is especially important for email notifications,
which may be sent after a long time after the message was sent. We will
in the future add code to thoroughly re-check settings before sending
notifications in a clean manner, but temporarily not re-checking isn't
a terrible scenario either.
2021-07-28 13:55:25 -07:00
Abhijeet Prasad Bodas 683c8507e4 models: Remove MutedTopic alias for UserTopic.
Part of #19272
2021-07-28 10:25:54 -07:00
Abhijeet Prasad Bodas 798defc046 models: Rename MutedTopic to UserTopic.
Part of #19272

We still keep refering to this model with "MutedTopic" to reduce the
diff size of this commit. The alias will be removed in the next commit.

This commit skips on renaming the `date_muted` field to something more
general. That will be done in further commits, along with the code and
API changes.
2021-07-28 10:25:53 -07:00
Riken Shah 9fadd43830 user_status: Add backend changes to support status emoji.
In this commit:

* We update the `UserStatus` model to accept
`AbstractReaction` as a base class so, we can get all the
fields related to store status emoji.

* We update the user status endpoint
(`users/me/status`) to accept status emoji fields.

* We update the user status event to add status emoji
fields.

Co-authored-by: Yash Rathore <33805964+YashRE42@users.noreply.github.com>
2021-07-27 17:57:15 -07:00
Tim Abbott ed01ffadba tests: Fix leading slash in test exclusion as well. 2021-07-27 17:01:18 -07:00
sahil839 0d6f378ad7 help: Update documentation for add_custom_emoji_policy. 2021-07-27 16:41:23 -07:00
sahil839 1ce09d9db6 realm: Extend add_custom_emoji_policy to use COMMON_POLICY_TYPES.
This commit adds full members and moderators option for
add_custom_emoji_policy by using COMMON_POLICY_TYPES.
2021-07-27 16:41:23 -07:00
sahil839 5efc854f66 models: Add can_add_custom_emoji helper.
This commit adds can_add_custom_emoji
helper to check whether the user can
add custom emoji or not.

This function will be used further when
add_custom_emoji_policy will be extended
to include all COMMON_POLICY_VALUES.
2021-07-27 16:41:23 -07:00
sahil839 4b1313a92b models: Replace add_emoji_by_admins_only with add_custom_emoji_policy.
This commit replaces boolean field add_emoji_by_admins_only with an
integer field add_custom_emoji_policy as we would also add full members
and moderators option for this setting in further commits.
2021-07-27 16:41:22 -07:00
Anders Kaseorg 17749cb608 archive: Remove non-functional archive code.
This removes a bunch of non-functional duplicate JavaScript, HTML, and
CSS that was interfering with maintenance on the functional originals,
because it was never clear how to update the duplicates or how to
check that you’d updated the duplicates correctly.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-27 16:25:24 -07:00
Sahil Batra 79b0ff5b23 events: Send enter_sends if update_display_settings in fetch_event_types.
We now send 'enter_sends' in response of '/register' endpoint if
'update_display_settings' is present in 'fetch_event_types' instead
of 'realm_user'.
2021-07-27 15:54:16 -07:00
Sahil Batra 355f6e9b53 settings: Move 'enter_sends' setting to property_types dict.
This commit moves "enter_sends" setting to property_types dict.
With this change, changing enter_sends setting also sends an
event of type "update_display_settings" and thus enables us
to live-update the UI.
2021-07-27 15:52:53 -07:00
Mateusz Mandera 7af789c48f models: Make realm_id arg of get_system_bot required. 2021-07-27 15:51:59 -07:00
Mateusz Mandera 013d6fa5e6 bots: Assume STAFF_SUBDOMAIN to be the realm where ERROR_BOT lives.
Now that get_system_bot requires a realm argument, a subdomain
needs to be made explicit to know in which organization to look up the
bot.
2021-07-27 15:51:59 -07:00
PIG208 e517f967b8 typing: Amend inaccurate type annotations.
This amend some type annotations that turn out to be inaccurate with
django-stubs.
2021-07-27 12:17:00 -07:00
PIG208 8121d2d58d typing: Fix misuse of HttpResponse.
Amend usage of HttpResponse when appropriate.
2021-07-27 14:31:19 +08:00
Tim Abbott 9ada794ce1 test_openapi: Fix mis-resolved merge conflict. 2021-07-26 18:22:09 -07:00
Suyash Vardhan Mathur ec6739ff29 openapi: Document get-subscribers endpoint. 2021-07-26 18:12:12 -07:00
Suyash Vardhan Mathur 2d2b089066 openapi: Document delete-topic endpoint. 2021-07-26 17:56:04 -07:00
sahil839 c7e756f984 setting: Allow only owners to change "Who can invite users" setting. 2021-07-26 17:48:01 -07:00
sahil839 8ced74192a invite: Add 'Nobody' option to invite_to_realm_policy in backend. 2021-07-26 17:48:01 -07:00
sahil839 ace527af0a api: Remove incorrect realm setting in 'realm/update_dict' schema.
We incorrectly include many realm settings in the data section of
'realm/update_dict' schema. It should only contain the settings
related to message edit, realm icon, realm logo and authentication
methods and not other settings, becausea all the other settings send
'realm/update' event and not 'realm/update_dict' event.

This commit only removes 'invite_to_realm_policy' and others will
be removed separately.
2021-07-26 17:43:51 -07:00
Mateusz Mandera 8f588dcbab models: Pass realm to get_user_including_cross_realm calls. 2021-07-26 15:33:13 -07:00
Mateusz Mandera 43329b6a34 bots: Pass realm to get_system_bot calls in export/import. 2021-07-26 15:33:13 -07:00
Mateusz Mandera 7a36ac6eca bots: Pass realm to get_system_bot call in server_initialization.py. 2021-07-26 15:33:13 -07:00
Mateusz Mandera 0416f0385a bots: Pass realm to get_system_bot calls in onboarding.py. 2021-07-26 15:33:13 -07:00
Mateusz Mandera c34260426a bots: Pass realm to remaining get_system_bot calls in tests. 2021-07-26 15:33:13 -07:00
Mateusz Mandera a6924e23fa bots: Pass realm to get_system_bot calls in streams.py. 2021-07-26 15:33:13 -07:00
Mateusz Mandera a124a4dfcf bots: Pass realm to get_system_bot calls in actions.py. 2021-07-26 15:33:13 -07:00
Mateusz Mandera 6f6be04f2e bots: Pass realm to get_system_bot call in test_users. 2021-07-26 15:33:13 -07:00
Mateusz Mandera 994ee70497 bots: Pass realm to self.notification_bot test helper. 2021-07-26 15:33:13 -07:00
Mateusz Mandera a01594e72b bots: Pass realm to get_system_bot call in DeferredWorker. 2021-07-26 15:33:13 -07:00
Mateusz Mandera f790a32d13 bots: Pass realm to get_system_bot calls in test_cache.py. 2021-07-26 15:33:13 -07:00
Mateusz Mandera 104139657b bots: Pass realm to get_system_bot calls in email_mirror. 2021-07-26 15:33:13 -07:00
Mateusz Mandera d45f3eecaa models: Add optional realm_id argument to get_system_bot. 2021-07-26 15:31:10 -07:00
PIG208 22c84e69ac tests: Fix inappropriate assertEqual.
We do not give a second argument to `self.assertEqual` here. We only need
to invoke the function.
2021-07-26 15:09:07 -07:00
PIG208 91de2cbe03 typing: Avoid scoped redefinition of different types.
Mypy doesn't allow redefinition of a variable using a different type
within the same scope.
https://github.com/python/mypy/issues/1174
2021-07-26 15:09:07 -07:00
PIG208 178a4b4eff tests: Initialize requests with different post data.
Instead of directly changing the `POST` attribute of a request, we
utilize the `HostRequestMock` initializer to produce requests with
different post data.
2021-07-26 15:09:07 -07:00
PIG208 c3abc0b472 tests: Correctly mock and invoke view functions.
The decorators require the decorated function to be a valid view
function. This changes the way the mocked view functions and requests
are implemented such that we can invoke view functions without future
type errors.
2021-07-26 15:09:07 -07:00
PIG208 57f46db09f tests: Pass HostRequestMock to export_realm.
`export_realm` accepts an HttpRequest as the first argument,
while `self.client_post` conflicts with it. Though the argument is
unused in `export_realm`, we keep it to be compliant with the
view function type.
2021-07-26 15:09:07 -07:00
PIG208 f280ebe4b5 decorator: Type zulip_login_required with overload.
As we only return the actual decorator as-is only if `function` is
`None`, we can use `@overload` to accurately annotate the return type
for the decorator.
2021-07-26 15:09:07 -07:00
PIG208 d2af20eb2a typing: Do explicit type conversion when appropriate.
When calling some functions or assigning values to certain attributes,
the arguments/right operand do not match the exact type that the
functions/attributes expect, and thus we fix that by converting types
beforehand.
2021-07-26 15:09:07 -07:00
PIG208 7d1c475f69 typing: Use assertions for function arguments.
Utilize the assert_is_not_None helper to eliminate errors of
'Argument x to "Foo" has incompatible type "Optional[Bar]"...'
2021-07-26 14:48:45 -07:00
PIG208 c4179428ac caching: Make sender type optional for flush_cache.
That's the correct Django interface for this function.
2021-07-26 14:48:07 -07:00
PIG208 2972e921b7 utils: Add a helper function for None-checks. 2021-07-26 14:46:45 -07:00
PIG208 cf8687662f confirmation: Use the correct type hints for create_confirmation_link.
Previously we annotate the first argument as `ContentType`, which
is wrong as suggested by django-stubs.
2021-07-26 14:46:45 -07:00
Dinesh beadb5ec7f test_middleware: Replace logging mocks with assertLogs. 2021-07-26 14:46:01 -07:00
Dinesh 2846d18854 test_decorators: Replace webhook logger mock with assertLogs.
Of the two other logging mocks left in this file, one checks
a logging call isn't made and another makes sure errors
aren't allowed by raising an exception as a side_effect
to the logger.
2021-07-26 14:46:01 -07:00
Dinesh 199f522808 test_classes.py: Replace markdown_logger mock with assertLogs.
Set level to 'ERROR' since exceptions create logs with that level.
2021-07-26 14:46:01 -07:00
Dinesh b195cc3635 test_push_notifications.py: Replace logging mocks with assertLogs.
Left the mocks which are used to assert a logging call isn't made.
2021-07-26 14:46:01 -07:00
Aman Agrawal 7d8f62e7a0 portico: Add a developer-community page.
This is intended to replace the ReadTheDocs page of this form, but
this commit does not yet update any links.
2021-07-25 17:01:26 -07:00
Tim Abbott 09b5bb7930 export: Improve error message for missing registrations. 2021-07-24 17:36:15 -07:00
Mateusz Mandera 1c64bed8e4 rate_limiter: Rate limit the /new/ endpoint. 2021-07-24 15:52:06 -07:00
Mateusz Mandera 4418aefde4 decorators: Extract rate_limit_request_by_ip function. 2021-07-24 15:52:06 -07:00
Mateusz Mandera 119f1da04a bots: Rename is_cross_realm_bot API field to is_system_bot.
Cross realm bots will soon stop being a thing. This param is responsible
for displaying "System Bot" in the user info popover - so this rename is the
right way to handle the situation.

We will likely want to rename the `cross_realm_bots` section as well,
but that is a more involved API migration.
2021-07-24 15:46:40 -07:00
Mateusz Mandera 1652e5c031 test_webhooks_common: Compare users by id not email.
Comparing by email can be incorrect, as different UserProfile can share
the same .email if they're in different realm.
2021-07-24 15:39:00 -07:00
Mateusz Mandera 86c330b752 social_auth: Fix handling of user errors in the authentication process.
The code didn't account for existence of SOCIAL_AUTH_SUBDOMAIN. So the
redirects would happen to endpoints on the SOCIAL_AUTH_SUBDOMAIN, which
is incorrect. The redirects should happen to the realm from which the
user came.
2021-07-24 15:17:52 -07:00
PIG208 5359d20a13 webhooks: Add support to event filtering system for webhooks.
This add support to event filtering system for most webhooks that
require trivial changes to adapt this feature.
2021-07-24 15:10:09 -07:00
PIG208 987f859681 webhooks: Detect view function with default naming convention. 2021-07-24 15:10:09 -07:00
PIG208 66b1a4e7ca backend: Add None-checks with assertions and if-elses.
This fixes a batch of mypy errors of the following format:
'Item "None" of "Optional[Something]" has no attribute "abc"'
2021-07-24 15:00:21 -07:00
PIG208 fffd4ed8d5 invitations: Correctly compare realms for revoke_user_invite. 2021-07-24 14:59:19 -07:00
PIG208 de2678a319 tests: Fix missed mypy errors in tests.
This fixes a batch of mypy errors of the following format:
'Item "None" of "Optional[Something]" has no attribute "abc"
as a follow-up for 9892951703.
2021-07-24 14:59:19 -07:00
Tim Abbott 01ce58319d mypy: Fix most AnonymousUser type errors.
This commit fixes several mypy errors with Django stubs, by telling
mypy that we know in a given code path that the user is authenticated.
2021-07-24 14:55:46 -07:00
Tim Abbott d603b0641e mypy: Extend type for post_data in send_to_push_bouncer. 2021-07-24 12:55:15 -07:00
Tim Abbott bd134019e4 mypy: Fix return type for num_push_devices_for_user. 2021-07-24 12:55:15 -07:00
Tim Abbott 4aba6c9833 mypy: Use Sequence in push notifications types.
This is important for variance reasons.
2021-07-24 12:55:15 -07:00
Tim Abbott 6e26c8912e actions: Fix stream_dict/sub_dict types and variable names. 2021-07-24 11:02:08 -07:00
Tim Abbott 3790146c75 actions: Fix typing for raw stream dictionaries. 2021-07-24 11:02:08 -07:00
PIG208 495a8476be tests: Use assertion to enforce None-checks in tests.
This fixes a batch of mypy errors of the following format:
'Item "None" of "Optional[Something]" has no attribute "abc"

Since we have already been recklessly using these attritbutes
in the tests, adding assertions beforehand is justified presuming
that they oughtn't to be None.
2021-07-24 09:54:21 -07:00
PIG208 442adfaff3 sessions: Use SessionBase instead of Session. 2021-07-24 09:54:21 -07:00
Tim Abbott 12737fdc02 models: Add models for emoji in user statuses. 2021-07-23 13:11:21 -07:00
Hemanth V. Alluri b5cd232a2e drafts: Use enable_drafts_synchronization for access control.
If a user doesn't have enable_drafts_synchronization set to True, then
don't let them access the drafts API.  This will help protect us
against client bugs accidentally sending drafts to the server when the
feature is disabled.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2021-07-23 12:34:15 -07:00
Hemanth V. Alluri 08e9e48205 models: Add a new field controlling drafts synchronization.
This field will control whether or not a user wants to sync their
drafts between different clients.  Defaults to enabled.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2021-07-23 12:31:00 -07:00
Abhijeet Prasad Bodas 5db4fe8652 api: Allow setting email_notifications_batching_period_seconds.
We allow a maximum value of one week to make sure there aren't a huge
number of rows in the table for any user (this could happen if stream
notifications are enabled).

This commit also fixes a small error in the user_settings test.
2021-07-23 12:22:05 -07:00