Commit Graph

5983 Commits

Author SHA1 Message Date
Hemanth V. Alluri 6a3e98d14b drafts: Add API documentation for the core drafts endpoints.
These were added at some point in the past, but were not complete, and
it makes sense to document the current feature level as and when they
become available, since clients should not use the drafts endpoints on
older feature levels.
2021-07-30 10:41:52 -07:00
Tim Abbott 472c55a1ff populate_db: Create some drafts in the initial database. 2021-07-30 10:41:52 -07:00
Hemanth V. Alluri 6fee946a43 drafts: Send events to clients when drafts change.
With this, the core of the new drafts system is complete.
2021-07-30 10:41:52 -07:00
Hemanth V. Alluri c00089ac28 drafts: Add support for toggling drafts synchronization.
With changes mostly to the API documentation by tabbott.
2021-07-30 10:00:27 -07:00
Anders Kaseorg 1f5fc12989 video_calls: Remove call to deprecated Zoom Data Compliance API.
https://marketplace.zoom.us/docs/guides/stay-up-to-date/announcements#data-compliance

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-30 09:19:22 -07:00
Suyash Vardhan Mathur ea6b21cd8c openapi: Document update-status endpoint. 2021-07-29 11:50:25 -07:00
Priyansh Garg 044fe547d3 data_import: Add huddle import support for Rocket.Chat. 2021-07-28 15:45:54 -07:00
Priyansh Garg 24dd0ff96c data_import: Add rocket chat import tool.
This commit allows to import the following from rocketchat:
* All users
* All public/private channels
* All teams and its public/private channels
* All discussion rooms as topics in their parent channel
* All the messages in all the channels
* All private conversations
* Reactions on messages (except for custom emojis)
* Mentions in messages (except @all, @here mentions)
2021-07-28 15:28:56 -07:00
Mateusz Mandera 4161e0caeb rate_limit: Show html page when rate limited at /new/ endpoint.
Previously this showed a json error, but this is an endpoint that human
users use in the browser, so a proper HTML page is more appropriate.
2021-07-28 15:24:04 -07:00
Abhijeet Prasad Bodas de04f0ad67 notifications: Calculate PMs/mentions settings like other settings.
Previously, we checked for the `enable_offline_email_notifications` and
`enable_offline_push_notifications` settings (which determine whether the
user will receive notifications for PMs and mentions) just before sending
notifications. This has a few problem:

1. We do not have access to all the user settings in the notification
handlers (`handle_missedmessage_emails` and `handle_push_notifications`),
and therefore, we cannot correctly determine whether the notification should
be sent. Checks like the following which existed previously, will, for
example, incorrectly not send notifications even when stream email
notifications are enabled-
```
if not receives_offline_email_notifications(user_profile):
    return
```
With this commit, we simply do not enqueue notifications if the "offline"
settings are disabled, which fixes that bug.

Additionally, this also fixes a bug with the "online push notifications"
feature, which was, if someone were to:
* turn off notifications for PMs and mentions (`enable_offline_push_notifications`)
* turn on stream push notifications (`enable_stream_push_notifications`)
* turn on "online push" (`enable_online_push_notifications`)

then, they would still receive notifications for PMs when online.
This isn't how the "online push enabled" feature is supposed to work;
it should only act as a wrapper around the other notification settings.

The buggy code was this in `handle_push_notifications`:
```
if not (
    receives_offline_push_notifications(user_profile)
    or receives_online_push_notifications(user_profile)
):
    return

    // send notifications
```

This commit removes that code, and extends our `notification_data.py` logic
to cover this case, along with tests.

2. The name for these settings is slightly misleading. They essentially
talk about "what to send notifications for" (PMs and mentions), and not
"when to send notifications" (offline). This commit improves this condition
by restricting the use of this term only to the database field, and using
clearer names everywhere else. This distinction will be important to have
non-confusing code when we implement multiple options for notifications
in the future as dropdown (never/when offline/when offline or online, etc).

3. We should ideally re-check all notification settings just before the
notifications are sent. This is especially important for email notifications,
which may be sent after a long time after the message was sent. We will
in the future add code to thoroughly re-check settings before sending
notifications in a clean manner, but temporarily not re-checking isn't
a terrible scenario either.
2021-07-28 13:55:25 -07:00
Abhijeet Prasad Bodas 683c8507e4 models: Remove MutedTopic alias for UserTopic.
Part of #19272
2021-07-28 10:25:54 -07:00
Riken Shah 9fadd43830 user_status: Add backend changes to support status emoji.
In this commit:

* We update the `UserStatus` model to accept
`AbstractReaction` as a base class so, we can get all the
fields related to store status emoji.

* We update the user status endpoint
(`users/me/status`) to accept status emoji fields.

* We update the user status event to add status emoji
fields.

Co-authored-by: Yash Rathore <33805964+YashRE42@users.noreply.github.com>
2021-07-27 17:57:15 -07:00
sahil839 1ce09d9db6 realm: Extend add_custom_emoji_policy to use COMMON_POLICY_TYPES.
This commit adds full members and moderators option for
add_custom_emoji_policy by using COMMON_POLICY_TYPES.
2021-07-27 16:41:23 -07:00
sahil839 5efc854f66 models: Add can_add_custom_emoji helper.
This commit adds can_add_custom_emoji
helper to check whether the user can
add custom emoji or not.

This function will be used further when
add_custom_emoji_policy will be extended
to include all COMMON_POLICY_VALUES.
2021-07-27 16:41:23 -07:00
sahil839 4b1313a92b models: Replace add_emoji_by_admins_only with add_custom_emoji_policy.
This commit replaces boolean field add_emoji_by_admins_only with an
integer field add_custom_emoji_policy as we would also add full members
and moderators option for this setting in further commits.
2021-07-27 16:41:22 -07:00
Anders Kaseorg 17749cb608 archive: Remove non-functional archive code.
This removes a bunch of non-functional duplicate JavaScript, HTML, and
CSS that was interfering with maintenance on the functional originals,
because it was never clear how to update the duplicates or how to
check that you’d updated the duplicates correctly.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-27 16:25:24 -07:00
Sahil Batra 355f6e9b53 settings: Move 'enter_sends' setting to property_types dict.
This commit moves "enter_sends" setting to property_types dict.
With this change, changing enter_sends setting also sends an
event of type "update_display_settings" and thus enables us
to live-update the UI.
2021-07-27 15:52:53 -07:00
PIG208 8121d2d58d typing: Fix misuse of HttpResponse.
Amend usage of HttpResponse when appropriate.
2021-07-27 14:31:19 +08:00
Tim Abbott 9ada794ce1 test_openapi: Fix mis-resolved merge conflict. 2021-07-26 18:22:09 -07:00
Suyash Vardhan Mathur ec6739ff29 openapi: Document get-subscribers endpoint. 2021-07-26 18:12:12 -07:00
Suyash Vardhan Mathur 2d2b089066 openapi: Document delete-topic endpoint. 2021-07-26 17:56:04 -07:00
sahil839 c7e756f984 setting: Allow only owners to change "Who can invite users" setting. 2021-07-26 17:48:01 -07:00
sahil839 8ced74192a invite: Add 'Nobody' option to invite_to_realm_policy in backend. 2021-07-26 17:48:01 -07:00
Mateusz Mandera c34260426a bots: Pass realm to remaining get_system_bot calls in tests. 2021-07-26 15:33:13 -07:00
Mateusz Mandera a124a4dfcf bots: Pass realm to get_system_bot calls in actions.py. 2021-07-26 15:33:13 -07:00
Mateusz Mandera 6f6be04f2e bots: Pass realm to get_system_bot call in test_users. 2021-07-26 15:33:13 -07:00
Mateusz Mandera 994ee70497 bots: Pass realm to self.notification_bot test helper. 2021-07-26 15:33:13 -07:00
Mateusz Mandera f790a32d13 bots: Pass realm to get_system_bot calls in test_cache.py. 2021-07-26 15:33:13 -07:00
Mateusz Mandera 104139657b bots: Pass realm to get_system_bot calls in email_mirror. 2021-07-26 15:33:13 -07:00
PIG208 22c84e69ac tests: Fix inappropriate assertEqual.
We do not give a second argument to `self.assertEqual` here. We only need
to invoke the function.
2021-07-26 15:09:07 -07:00
PIG208 91de2cbe03 typing: Avoid scoped redefinition of different types.
Mypy doesn't allow redefinition of a variable using a different type
within the same scope.
https://github.com/python/mypy/issues/1174
2021-07-26 15:09:07 -07:00
PIG208 178a4b4eff tests: Initialize requests with different post data.
Instead of directly changing the `POST` attribute of a request, we
utilize the `HostRequestMock` initializer to produce requests with
different post data.
2021-07-26 15:09:07 -07:00
PIG208 c3abc0b472 tests: Correctly mock and invoke view functions.
The decorators require the decorated function to be a valid view
function. This changes the way the mocked view functions and requests
are implemented such that we can invoke view functions without future
type errors.
2021-07-26 15:09:07 -07:00
PIG208 57f46db09f tests: Pass HostRequestMock to export_realm.
`export_realm` accepts an HttpRequest as the first argument,
while `self.client_post` conflicts with it. Though the argument is
unused in `export_realm`, we keep it to be compliant with the
view function type.
2021-07-26 15:09:07 -07:00
Dinesh beadb5ec7f test_middleware: Replace logging mocks with assertLogs. 2021-07-26 14:46:01 -07:00
Dinesh 2846d18854 test_decorators: Replace webhook logger mock with assertLogs.
Of the two other logging mocks left in this file, one checks
a logging call isn't made and another makes sure errors
aren't allowed by raising an exception as a side_effect
to the logger.
2021-07-26 14:46:01 -07:00
Dinesh b195cc3635 test_push_notifications.py: Replace logging mocks with assertLogs.
Left the mocks which are used to assert a logging call isn't made.
2021-07-26 14:46:01 -07:00
Aman Agrawal 7d8f62e7a0 portico: Add a developer-community page.
This is intended to replace the ReadTheDocs page of this form, but
this commit does not yet update any links.
2021-07-25 17:01:26 -07:00
Mateusz Mandera 1c64bed8e4 rate_limiter: Rate limit the /new/ endpoint. 2021-07-24 15:52:06 -07:00
Mateusz Mandera 119f1da04a bots: Rename is_cross_realm_bot API field to is_system_bot.
Cross realm bots will soon stop being a thing. This param is responsible
for displaying "System Bot" in the user info popover - so this rename is the
right way to handle the situation.

We will likely want to rename the `cross_realm_bots` section as well,
but that is a more involved API migration.
2021-07-24 15:46:40 -07:00
Mateusz Mandera 1652e5c031 test_webhooks_common: Compare users by id not email.
Comparing by email can be incorrect, as different UserProfile can share
the same .email if they're in different realm.
2021-07-24 15:39:00 -07:00
Mateusz Mandera 86c330b752 social_auth: Fix handling of user errors in the authentication process.
The code didn't account for existence of SOCIAL_AUTH_SUBDOMAIN. So the
redirects would happen to endpoints on the SOCIAL_AUTH_SUBDOMAIN, which
is incorrect. The redirects should happen to the realm from which the
user came.
2021-07-24 15:17:52 -07:00
PIG208 de2678a319 tests: Fix missed mypy errors in tests.
This fixes a batch of mypy errors of the following format:
'Item "None" of "Optional[Something]" has no attribute "abc"
as a follow-up for 9892951703.
2021-07-24 14:59:19 -07:00
PIG208 495a8476be tests: Use assertion to enforce None-checks in tests.
This fixes a batch of mypy errors of the following format:
'Item "None" of "Optional[Something]" has no attribute "abc"

Since we have already been recklessly using these attritbutes
in the tests, adding assertions beforehand is justified presuming
that they oughtn't to be None.
2021-07-24 09:54:21 -07:00
Hemanth V. Alluri b5cd232a2e drafts: Use enable_drafts_synchronization for access control.
If a user doesn't have enable_drafts_synchronization set to True, then
don't let them access the drafts API.  This will help protect us
against client bugs accidentally sending drafts to the server when the
feature is disabled.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2021-07-23 12:34:15 -07:00
Abhijeet Prasad Bodas 5db4fe8652 api: Allow setting email_notifications_batching_period_seconds.
We allow a maximum value of one week to make sure there aren't a huge
number of rows in the table for any user (this could happen if stream
notifications are enabled).

This commit also fixes a small error in the user_settings test.
2021-07-23 12:22:05 -07:00
Abhijeet Prasad Bodas dd5e12d112 MissedMessageWorker: Use custom batching periods from UserProfile. 2021-07-23 12:13:46 -07:00
Abhijeet Prasad Bodas 9fcb6e51ce MissedMessageWorker: Handle deleted messages.
The test for the try-except block is hacky. See the comment for
explaination.
2021-07-23 12:13:46 -07:00
Anders Kaseorg 162e9d6c0b fenced_code: Optimize FENCE_RE to fix cubic worst-case complexity.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-22 16:40:44 -07:00
Sahil Batra 318d71469b api: Remove '/users/me/enter-sends' endpoint.
We remove the '/users/me/enter-sends' endpoint
and 'enter_sends' setting will now be edited
using the '/settings' endpoint.
2021-07-22 11:01:43 -07:00
Anders Kaseorg f9430674d2 github: Update macOS dmg filename pattern.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-21 19:03:39 -07:00
Tim Abbott 1f497bf8c8 test_docs: Update tests for recent plans page changes.
This should have been in b03cab8ced.
2021-07-21 18:03:55 -07:00
Aman Agrawal 56ce1f9f63 portico: Add pricing widget from /plans to for-education page. 2021-07-21 12:13:25 -07:00
Aman Agrawal 3c024b2fba portico: Add /for/education, /for/events, /for/research pages.
These modern landing pages cover use cases previously not detailed on
our website. Technically, we had a /for/research page before, but it
wasn't finished or linked everywhere.

Removed "function-url-quotes" stylelint rule
since I need to use quotes in url to use an
svg as list bullet point. There are spacing issues
using it as an image. Also, using quotes in url
is actually the recommended way to do it otherwise
there could be issue with escaping.
2021-07-20 22:37:52 -07:00
Tim Abbott 256091dc15 settings: Merge settings API endpoints.
This API change removes unnecessary complexity from a client that
wants to change a user's personal settings, and also saves developers
from needing to make decisions about what sort of setting something is
at the API level.

We preserve the old settings endpoints as mapping to the same function
as the new one for backwards-compatibility. We delete the
documentation for the old endpoints, though the documentation for the
merged /settings endpoint mentions how to use the old endpoints when
needed.

We migrate all backend tests to the new endpoints, except for
individual tests for each legacy endpoint to verify they still work.

Co-authored-by: sahil839 <sahilbatra839@gmail.com>
2021-07-19 21:14:32 -07:00
Tim Abbott b73356e891 portico: Add UCSD case study. 2021-07-18 21:55:57 -07:00
Alya Abbott 844c62c64e portico: Create new "Zulip for communities" page. 2021-07-18 12:10:33 -07:00
Tim Abbott 95606a7347 api: Return user IDs, not display emails, in subscribers endpoints.
Sometime in the deep past, Zulip the GET /users/me/subscriptions
endpoint started returning subscribers.  We noticed this and made it
optional via the include_subscribers parameter in
1af72a2745, however, we didn't notice
that they were being returned as emails rather than user IDs.

We migrated the core /register code paths to use subscriber IDs years
ago; this change completes that for the endpoints we forgot about.

The documentation allowed this error because we apparently had no
tests for this code path that used the actual API.
2021-07-18 11:32:28 -07:00
sahil839 d5a0c1ede5 api: Remove "full_name" and "account_email" from response of '/settings'.
We remove the "full_name" and "account_email" fields from the response
of 'PATCH /settings' endpoint. These fields were part of the response
to make sure that we tell that the parameters not present in response
were ignored.

We can remove these fields as 'ignored_parameters_unsupported' now
specifies which parameters were ignored and not supported by the
endpoint.
2021-07-16 16:10:15 -07:00
sahil839 05aff3f271 api: Add "ignored_parameters_unsupported" to response of '/settings'.
We add "ignored_parameters_unsupported" field to the response object
of 'PATCH /settings' endpoint. This will contain the parameters
passed to the endpoint which are not changed by the endpoint and are
ignored.

This will help in removing the other fields like "full_name" from
response which was essentially present to specify that only these
fields were updated by the endpoint and rest were ignored.

We will also change other endpoints to follow this in future.
2021-07-16 16:10:15 -07:00
Mateusz Mandera 50e9169680 oidc: Add auto_signup configuration option. 2021-07-16 14:40:07 -07:00
Anders Kaseorg fb3ddf50d4 python: Fix mypy no_implicit_reexport errors.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-16 14:02:31 -07:00
Anders Kaseorg 1ae56e466b cache: Fix typing for post_save and post_delete flush handlers.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-16 13:14:04 -07:00
Abhijeet Prasad Bodas 9901b72690 tests: Purge queue before starting tests, not at tearDown.
This is more robust towards reruning failed tests (which ran
partially and added some events to a queue before failing).

The tearDown code was added in 571f8b8664.
2021-07-16 09:00:02 -07:00
Anders Kaseorg a0e3de302f test_reactions: Fix ineffective mock due to indentation mistake.
Introduced by commit 90b6fa7881.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-15 17:44:59 -07:00
akshatdalton 6b5812082e markdown: Fix shebang line eliminating behaviour of Codehilite.
See the block comment explaining the motivation for this change, but
basically, the shebang feature of Python-Markdown's Codehilite
extension could be really confusing and is not part of the CommonMark
standard.

1. https://python-markdown.github.io/extensions/code_hilite/#shebang-no-path
2. eacff473a2/markdown/extensions/codehilite.py (L164-L180)

Fixes: #18591.
2021-07-15 15:18:33 -07:00
Priyansh Garg a21a280054 data_import: Rename mattermost_user to user_handler.
This logic can be readily reused for new data import tools.
2021-07-15 14:28:36 -07:00
Eeshan Garg faa695e86d registration: Collect organization type on sign-up. 2021-07-15 09:58:27 -07:00
Tim Abbott 4fb6f089e5 api: Remove error for no settings changes requested.
While this error message was vaguely useful, it wasn't reachable via
the UI at all, and it prevents merging this endpoint with related
endpoints.
2021-07-14 15:54:05 -07:00
sahil839 61acf68aa4 emoji: Allow emoji authors to delete the emoji always.
Previously, non-admin emoji authors were allowed to
delete the emoji only if add_emoji_by_admins_only
was false. But, as add_emoji_by_admins_only setting
is for who can add emoji and not delete emojis, it
should not affect the behavior of deleting emojis
and users should always be allowed to delete the
emojis which. they added themselves
2021-07-14 14:57:08 -07:00
sahil839 50240ca71b models: Use COMMON_POLICY_TYPES for user_group_edit_policy.
This commit adds moderators and full members options for
user_group_edit_policy by using COMMON_POLICY_TYPES.

Moderators do not require to be a member of user group in
order to edit or remove the user group if they are allowed
to do so according to user_group_edit_policy.
But full members need to be a member of user group to edit
or remove the user group.
2021-07-14 14:38:44 -07:00
sahil839 93a1479286 user_group: Change error message to "Insufficient permission".
There is no need to have a error message which specifies the
roles having permission to edit user-groups, we can simply
have error message as "Insufficient permission" as we already
show the roles having permission clearly in UI.
2021-07-14 14:38:43 -07:00
PIG208 26875cdf0b tests: Migrate mocked Request to HostMockRequest.
This migrates some mocked Request class and mocked request achieved
with namedtuple in test_decorators and test_mirror_users to use the
refactored HostMockRequest.

Since weakref cannot be used with namedtuple, this old way of mocking a
request object should be migrated to using HostRequestMock. Only after
this change we can extract client from the request object and store it
via ZulipRequestNotes.
2021-07-14 12:01:07 -07:00
PIG208 5167a93229 request: Move tornado_handler to ZulipRequestNotes. 2021-07-14 12:01:07 -07:00
PIG208 5475334b16 request: Refactor to store requestor_for_logs in ZulipRequestNotes. 2021-07-14 12:01:07 -07:00
PIG208 75cde8ebcf request: Refactor HostRequestMock to subclass HttpRequest. 2021-07-14 11:52:43 -07:00
akshatdalton f5c4d51ed2 resolve topic: Add `is:resolved` search keyword/filtering support.
This commit adds the backend support for `is:resolved` search keyword.
In the next commit, I will add the frontend support for the same.
2021-07-13 23:18:41 -07:00
akshatdalton 7ec406f39d refactor: Extract `RESOLVED_TOPIC_PREFIX` in topic.py.
This is a prep commit for #18990.
2021-07-13 23:18:41 -07:00
Abhijeet Prasad Bodas ebb4eab0f9 worker: Rewrite MissedMessageWorker to not be lossy.
Previously, we stored up to 2 minutes worth of email events in memory
before processing them. So, if the server were to go down we would lose
those events.

To fix this, we store the events in the database.

This is a prep change for allowing users to set custom grace period for
email notifications, since the bug noted above will aggravate with
longer grace periods.
2021-07-13 17:21:38 -07:00
Abhijeet Prasad Bodas 1709428cff models: Create MissedMessageEmailEntry table.
This will be used to store the missedmessage events received
during the waiting time for email notifications (which is currently
2 minutes, hardcoded).

The change in `test_retention` is because we've set `on_delete=CASCADE`
for the message field this table.
The new query is like so:
```
DELETE FROM "zerver_missedmessageemailentry"
    WHERE "zerver_missedmessageemailentry"."message_id" IN (
        1545, 1546, 1547, 1548, 1549, 1550, 1551, 1552, 1553
        )
```
2021-07-13 17:21:37 -07:00
Abhijeet Prasad Bodas c3319a5231 notification_data: Create common source for trigger strings.
This reduces loose strings in the codebase, and allows us to not worry
about the exact naming (`stream_email_enabled` or `stream_emails_enabled`?)
and tense (`mentioned` or `mention`?).

Ideally this new class should have been in `lib/notification_data.py`,
which is our file for things like this. But, the next commit requires
using this data in `models.py`, and importing from `notification_data.py`
to `models.py` causes recursive imports.
2021-07-13 17:16:32 -07:00
akshatdalton 3ea1ff7665 refactor: Use `assertGreaterEqual` helper instead of `assertTrue`. 2021-07-13 13:03:38 -07:00
akshatdalton 0b469e9e4e refactor: Use `assertGreater` helper instead of `assertTrue`. 2021-07-13 13:03:38 -07:00
akshatdalton e203112fd4 refactor: Use `assert_length` helper instead of `assertTrue/assertEqual`. 2021-07-13 13:03:38 -07:00
Mateusz Mandera db7d2ee713 test_upload: Fix unnecessarily hard-coded user id. 2021-07-13 08:31:11 -07:00
Mateusz Mandera af29bd77a0 test_sessions: Fix unnecessarily hard-coded user id. 2021-07-13 08:31:11 -07:00
Mateusz Mandera 91b3ba6d66 test_queue_worker: Fix unnecessarily hard-coded user id. 2021-07-13 08:31:11 -07:00
Mateusz Mandera ccd961b797 test_reactions: Fix unnecessarily hard-coded user id. 2021-07-13 08:31:11 -07:00
Mateusz Mandera 6ec5a9698f test_markdown: Fix unnecessarily hard-coded user id. 2021-07-13 08:31:11 -07:00
Abhijeet Prasad Bodas e63e86dcb2 worker: Ensure complete coverage for PushNotificationsWorker.
The `# nocoverage` was unnecessary apart from for the compatibility code,
so add a test for that code and remove the `# nocoverage`.

The `message_id` -> `message_ids` conversion was done in
9869153ae8.
2021-07-13 08:30:31 -07:00
aryanshridhar 7dc56fdcf2 settings_emoji: Allow only admins to override existing emojis.
Previously, even non-admins had the option to override built-in
emojis in the `Settings Emoji` UI.

This commits essentially limits the functionality of overriding
custom and allows only realm administrators to
override built-in emojis with their custom emojis by adding an
authorization check in the backend.

It also adds relevant tests in `test_realm_emoji` which tests
for the cases where an admin and non admin tries to override
the built-in emoji.

Fixes #18860.
2021-07-13 08:28:28 -07:00
Tim Abbott eb8283c089 test_docs: Test the new case study page. 2021-07-09 13:19:05 -07:00
Tim Abbott c804ab27d5 actions: Rename do_activate_user to do_activate_mirror_dummy_user.
This function had a confusing name, which could result in someone
using it unintentionally when they meant do_reactivate_user.

We also add docstrings for both functions.
2021-07-08 17:33:17 -07:00
Mateusz Mandera 093ca7a574 zilencer: Log warning when a remote server hits rate limits.
We don't want this rate limit to affect legitimate users so it being hit
should be abnormal - thus worth logging so that we can spot if we're
rate limiting legitimate users and can know to increase the limit.
2021-07-08 15:55:03 -07:00
Mateusz Mandera 85cbdc8904 rate_limit: Add rate limiting of ZulipRemoteServer. 2021-07-08 15:55:02 -07:00
Mateusz Mandera b9056d193d rate_limit: Implement IP-based rate limiting.
If the user is logged in, we'll stick to rate limiting by the
UserProfile. In case of requests without authentication, we'll apply the
same limits but to the IP address.
2021-07-08 15:46:52 -07:00
Mateusz Mandera df96e02732 test_external: Extract common logic for "hit rate limits" tests. 2021-07-08 15:35:12 -07:00
Mateusz Mandera 3b4f8cc85b saml: Add setting to skip the "continue to registration" page.
It's a smoother Just-In-Time provisioning process to allow
creating the account and getting signed in on the first login by the
user.
2021-07-08 15:21:20 -07:00
sahil839 a535ec6262 settings: Remove timezone from UserProfile.property_types.
We remove timezone setting from UserProfile.property_types
so that we can directly use UserProfile.property_types for
implementation of realm-default values of various user
settings.
2021-07-08 12:52:29 -07:00
aryanshridhar 9049fb3bd4 users: Remove redundant compute_show_invites function.
This commits removes the redundant `compute_show_invites` function
which computes the `show_invites` page parameter in `lib/users.py`.

It is so because, commit 13399833b0 removed
the `show_invites` context variable passed in index.html.
Hence, the `show_invites` page_param key is no
longer required to compute in backend as it can be switched with
`settings_data.user_can_invite_others_to_realm()` in the frontend.

This commits also removes the `test_compute*` tests in
`test_home` that concerned with the `show_invites` page parameter
as they are no longer required.
2021-07-08 12:37:47 -07:00
Abhijeet Prasad Bodas bf15c0235a notifications: Remove unused fields in queue events.
* `stream_name`: This field is actually redundant. The email/push
  notifications handlers don't use that field from the dict, and they
  anyways query for the message, so we're safe in deleting this field,
  even if in the future we end up needing the stream name.

* `timestamp`: This is totally unused by the email/push notification
  handlers, and aren't sent to push clients either.

* `type` is used only for the push notifications handler, since only
  push notifications can be revoked, so we move them to only run there.
2021-07-08 11:22:45 -07:00
Tim Abbott af47fa705e exceptions: Use HTTP 401 code for authentication errors. 2021-07-08 10:33:08 -07:00
Abhijeet Prasad Bodas 4f9c7cae0a push_notifications: Send mentioned user group ID and name in payload. 2021-07-08 10:19:43 -07:00
Abhijeet Prasad Bodas 9bd8fe01fc android push notifications: Display mentioned user group name.
Followup to 83399e2e72.
2021-07-08 10:03:07 -07:00
Abhijeet Prasad Bodas 4d24499317 android notifications: Differentiate personal vs wildcard mentions.
The code to also notify for wildcard mentions was added in
0ed0bb6828.

But that showed the same text for both the cases. This commit fixes
that.

This is more of change for correctness. The mobile app currently does
not rely on this text for notifications, but constructs the text by
itself from the data in the payload.

This also fixes the "stream_push_notify" case to consistently show
a `#` before the stream name.
2021-07-08 10:03:07 -07:00
Abhijeet Prasad Bodas ce6f6a3829 push_notifications: Pre-calculate mentioned_user_group_name.
Prep change for showing the mentioned user group name in Android
notifications also. This will avoid doing the user group fetch twice.
2021-07-08 10:03:07 -07:00
Dinesh c4463ee34c test_example.py: Add a new example to demonstrate mocking.
Took the help of our mocking docs:
https://zulip.readthedocs.io/en/latest/testing/testing-with-django.html#testing-with-mocks
and rewrote a part of
test_message_edit.EditMessageTest.test_edit_message_content_limit
using mock.patch().
2021-07-08 10:04:20 -04:00
Ganesh Pawar d6711e3cd2 muting: Add support for muting deactivated users.
Fixes #19141
2021-07-07 10:44:18 -07:00
Anders Kaseorg 98194b9316 blueslip: Remove SAVE_FRONTEND_STACKTRACES setting.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-07 09:26:02 -07:00
Anders Kaseorg 07fef56c74 logging_handlers: Remove STAGING_ERROR_NOTIFICATIONS setting.
Running notify_server_error directly from the logging handler can lead
to database queries running in a random context.  Among the many
potential problems that could cause, one actual problem is a
SynchronousOnlyOperation exception when running in an asyncio event
loop.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-07 09:26:02 -07:00
Mateusz Mandera cee4da64fa saml: Don't raise AssertionError if no name is provided in SAMLResponse.
This is an acceptable edge case for SAML and shouldn't raise any errors.
2021-07-07 09:23:21 -07:00
Vishnu KS 4ad592ed4f populate_db: Use do_create_realm for creating zulip realm.
Since do_create_realm also creates general and core team streams,
we rename general to verona right after the realm is created. Mostly
because we dont really want two additional streams and this might
probably make it easy to review things.

There are puppeteer test changes because, we have a new "core team"
stream in tests as well as there is a new default notification stream
"Verona". Because of this tests in message-basics for example have
to be changed since the newly added core team affects the order in
which we navigate through the streams using arrow keys.

The extra await for selector was added in subscriptions test to make
the tests wait. Without the await the tests were passing ocassionally
and failing in some other times.

Fixes #6967
2021-07-06 17:37:43 -07:00
Vishnu KS acffc0ae0a populate_db: Use do_create_realm for creating zephyr realm. 2021-07-06 17:22:00 -07:00
PIG208 8b9011dff8 json_error: Completely remove json_error.
This completes the migration from `return json_error` to
`raise JsonableError`.
2021-07-06 15:34:33 -07:00
Tim Abbott 331f34cc1f auth: Use standard classes for API key fetch exceptions.
This lets us reuse a bunch of code and error handling logic, resulting
in a more sensible and consistent API.
2021-07-06 15:28:38 -07:00
Tim Abbott ac568d303c test_queue_worker: Fix order-dependent assertions. 2021-07-06 14:37:16 -07:00
Mateusz Mandera 95b46549e1 embed_links: Only log warning if worker times out.
Throwing an exception is excessive in case of this worker, as it's
expected for it to time out sometimes if the urls take too long to
process.

With a test added by tabbott.
2021-07-06 14:17:24 -07:00
sahil839 c13f0626e3 tests: Use the list in models.py directly in do_test_realm_update_api.
The list of possible values of all settings was re-defined in
do_test_realm_update_api and we can instead use the list defined
in models.py which is used to validate values in views/realm.py.

There is no problem in order of values as we always initialize the
first value of the list.

This has also added some more values to test for a couple of
settings as a result.
2021-07-06 13:45:01 -07:00
sahil839 b4a0a33f4c tests: Test all values in the list in do_test_realm_update_api.
This commit changes do_test_realm_update_api to test all values
in the list of a given setting instead of checking only two
values.
2021-07-06 13:45:01 -07:00
sahil839 8314620314 tests: Change test_change_realm_default_language to test invalid value.
We change the test_change_realm_default_language to test only invalid
value and rename it to test_invalid_realm_default_language, because
we already test whether the value is changed correctly or not in
do_test_realm_update_api.
2021-07-06 13:45:01 -07:00
sahil839 6a3a07a359 tests: Remove test_change_bot_creation_policy from test_realm.py.
This commit removes test_change_bot_creation_policy which is used
to test changing bot_creation_policy using 'PATCH /realm' endpoint
as we already do this in do_test_realm_update_api and invalid value
is also tested in test_invalid_integer_attribute_values.
2021-07-06 13:45:01 -07:00
sahil839 77471a5b44 tests: Remove test_change_email_address_visibility from test_realm.py.
This commit removes test_change_email_address_visibility which is used
to test changing email_address_visibility using 'PATCH /realm' endpoint
as we already do this in do_test_realm_update_api and invalid value is
also tested in test_invalid_integer_attribute_values.
2021-07-06 13:45:01 -07:00
sahil839 74023bfacf tests: Remove test_change_invite_to_stream_policy from test_realm.py.
This commit removes test_change_invite_to_stream_policy which is used
to test changing invite_to_stream_policy using 'PATCH /realm' endpoint
as we already do this in do_test_realm_update_api and invalid value
is also tested in test_invalid_integer_attribute_values.
2021-07-06 13:45:01 -07:00
sahil839 99e5058bd6 tests: Remove test_change_invite_to_realm_policy from test_realm.py.
This commit removes test_change_invite_to_realm_policy which is used
to test changing invite_to_realm_policy using 'PATCH /realm' endpoint
as we already do this in do_test_realm_update_api and invalid value
is also tested in test_invalid_integer_attribute_values.
2021-07-06 13:45:01 -07:00
sahil839 1c0e67e7c4 tests: Remove test_change_move_messages_between_streams_policy.
This commit removes test_change_move_messages_between_streams_policy
which is used to test changing move_messages_between_streams_policy
using 'PATCH /realm' endpoint as we already do this in
do_test_realm_update_api and invalid value is also tested in
test_invalid_integer_attribute_values.
2021-07-06 13:45:01 -07:00
sahil839 8831fb22b1 tests: Remove test_user_group_edit_policy from test_realm.py.
This commit removes test_user_group_edit_policy which is used
to test changing user_group_edit_policy using 'PATCH /realm'
endpoint as we already do this in do_test_realm_update_api and
invalid value is also tested in test_invalid_integer_attribute_values.
2021-07-06 13:45:01 -07:00
sahil839 a1e6cbdc80 tests: Remove test_private_message_policy from test_realm.py.
This commit removes test_private_message_policy which is used to
test changing private_message_policy using 'PATCH /realm' endpoint
as we already do this in do_test_realm_update_api and invalid
value is also tested in test_invalid_integer_attribute_values.
2021-07-06 13:45:01 -07:00
sahil839 f95faaf732 tests: Remove test_change_wildcard_mention_policy from test_realm.py.
This commit removes test_change_wildcard_mention_policy which is
used to test changing wildcard_mention_policy using 'PATCH /realm'
endpoint as we already do this in do_test_realm_update_api and
invalid value is also tested in test_invalid_integer_attribute_values.
2021-07-06 13:45:01 -07:00
sahil839 8e26f920ab tests: Remove test_change_stream_creation_policy from test_realm.py.
This commit removes test_change_stream_creation_policy which is
used to test changing create_stream_policy using 'PATCH /realm'
endpoint as we already do this in do_test_realm_update_api and
invalid value is also tested in test_invalid_integer_attribute_values.
2021-07-06 13:45:01 -07:00
Riken Shah a96c614420 message: Fix the edit topic bug by removing leading \n from msg content.
The reason for this bug is because of different striping
processes in the backend and frontend, i.e The frontend
checks if the message's `raw_content` has changed to
decide if the `content` of the message should be sent in
the request to the backend, or not. So, it removes the
leading new line ('\n') from the message `raw_content`
when checking it, which is causing the "Error saving edit:
You don't have permission to edit this message" error.

This commit fixes it by removing the leading new line
when cleaning message content.

The bug was explained by @punchagan and its solution
by @timabbott.
2021-07-06 09:41:54 -07:00
Anders Kaseorg 08592c9731 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-06 08:40:32 -07:00
Anders Kaseorg 3853285241 push_notifications: Replace PyAPNs2 with aioapns.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-05 20:16:50 -07:00
Anders Kaseorg 0bc002270c push_notifications: Use lru_cache decorator.
This does the same thing in a simpler way.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-05 20:16:50 -07:00
aryanshridhar 13399833b0 right_sidebar: Migrate to handlebars.
This commit migrates the `right_sidebar.html` Django template
to handlebars by creating a new file as `right_sidebar.hbs`
which is then rendered using `ui_init` module.

It also removes the tests in `test_home` due to the template
migration, since these elements aren't rendered on the backend
anymore.

We also remove `test_compute_show_invites_and_add_streams*`.

Fixes part of #18792.
2021-07-05 18:58:24 -07:00
aryanshridhar 8fb7ff7637 left_sidebar: Migrate to handlebars.
This commit migrates the `left_sidebar.html` Django template
to handlebars by creating a new file as `left_sidebar.hbs`
which is then rendered using `ui_init` module.

These are the minor changes introduced by virtue of template
migration -
 - The `compute_show_invites_and_add_streams` function now
   only concerns with the invite_to_realm_policy.
 - Renamed the `compute_show_invites_and_add_streams` function
   to `compute_show_invites` due to the above change.
 - Fixes relevant `test_home.py` tests due to the above
   changes.

Fixes part of #18792.
2021-07-05 18:49:43 -07:00
Anders Kaseorg ee8724e436 docs: Correct Big Blue Button to BigBlueButton.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-05 16:59:00 -07:00
Hashir Sarwar 87d90f47ee email_notifications: Show mentioned user group in email notifications.
Fixes #13080.
Cherry-picked from #15011 with edits.

Co-authored-by: Abhijeet Bodas <abhijeetbodas2001@gmail.com>
2021-07-05 14:23:59 -07:00
Hashir Sarwar 83399e2e72 push_notifications: Show mentioned user group in mobile notifications.
Part of #13080.
Cherry-picked from #15011 with edits.

Co-authored-by: Abhijeet Bodas <abhijeetbodas2001@gmail.com>
2021-07-05 14:23:59 -07:00
Abhijeet Prasad Bodas 167be7dbdc mentions: Send user group mention data to notification notices.
We will later use this data to include text like:
`<sender> mentioned @<user_group>` instead of the current
`<sender> mentioned you` when someone mentions a user group
the current user is a part of in email/push notification.

Part of #13080.
2021-07-05 14:23:59 -07:00
Abhijeet Prasad Bodas 07d6ab9753 notification_data: Add `get_user_group_mentions_data` function.
We will use this later to display which user group was mentioned
in push and email notifications.

`mentioned_user_group_ids` is kept as a List (not Set) to ensure proper
test coverage of the function, since it depends on the order of iteration,
and we cannot change the order of iteration for a set (which we'll need
to do for proper testing).

Part of #13080.
2021-07-05 14:23:59 -07:00
PIG208 6a04648fd7 exceptions: Add ResourceNotFoundError. 2021-07-05 11:02:12 -07:00
PIG208 a6e88a5a76 exceptions: Add AccessDeniedError. 2021-07-05 11:02:09 -07:00
Vishnu KS e0f5fadb79 billing: Downgrade small realms that are behind on payments.
An organization with at most 5 users that is behind on payments isn't
worth spending time on investigating the situation.

For larger organizations, we likely want somewhat different logic that
at least does not void invoices.
2021-07-02 13:19:12 -07:00
Priyansh Garg 5b2e21965c data_import: Add import attachments support for Mattermost.
Add support for importing message attachments from Mattermost.

Fixes: #18959
2021-07-02 11:19:45 -07:00
Tim Abbott 64aa8f80a0 events: Add heartbeat events to tests and documentation.
Heartbeat events are an important part of the API, even though they
are noops, so it's important to document them.
2021-07-01 17:14:32 -07:00
sahil839 0c80d23857 help: Update help pages according to profile page redesign.
This commit fixes the documentation of settings as we have
replaced "Your account" section into two new sections -
"Profile" and "Account & privacy".

This commit also fixes a comment in the test for settings
documentation in test_middleware.py.
2021-06-30 16:56:03 -07:00
PIG208 dcbb2a78ca python: Migrate most json_error => JsonableError.
JsonableError has two major benefits over json_error:
* It can be raised from anywhere in the codebase, rather than
  being a return value, which is much more convenient for refactoring,
  as one doesn't potentially need to change error handling style when
  extracting a bit of view code to a function.
* It is guaranteed to contain the `code` property, which is helpful
  for API consistency.

Various stragglers are not updated because JsonableError requires
subclassing in order to specify custom data or HTTP status codes.
2021-06-30 16:22:38 -07:00
Tim Abbott 0be35f530b push_notifications: Replace 'REDACTED' string.
The previous string was bold, potentially confusing, and doesn't
explain clearly what's happening. We replace this with a string that's
more or less copied from what we do in email notifications with the
similar setting enabled.
2021-06-30 15:15:22 -07:00
PIG208 0740c94517 webhooks: Support configuring destination stream by id. 2021-06-30 09:31:07 -07:00
sahil839 ce133dc762 models: Add can_edit_user_group helper.
This helper will be used to check whether
the user is allowed to edit user groups or
not. Currently it is not used, but will
be used in next commit where we will
refactor the user_group_edit_policy to use
COMMON_POLICY_TYPES.
2021-06-29 17:25:42 -07:00
sahil839 c8849f8fe3 logo: Compute realm-logo url in frontend only.
This commit fixes the bug of always showing
day-mode realm logo when color scheme display
setting is set to automatic but the OS setting
is dark theme. This is because we cannot check
the OS setting on backend and we need to set
the logo url accordingly in frontend only.
So, we remove the logo url computation from
backend completely and instead compute it in
the frontend only.

Fixes #18778.
2021-06-29 13:42:44 -07:00
Steve Howell 9e1d98a512 widgets: Add range checks on backend for indexes. 2021-06-29 13:40:33 -07:00
Steve Howell c25dbf7020 widgets: Validate todo data on the backend. 2021-06-29 13:40:33 -07:00
aryanshridhar 8140a1a6ee info_overlay: Render search operators using info_overlay.
This commits ports the `search_operators.html` file from
./templates to handlebars, essentially creating a new file
as `search_operators.hbs` within /static/templates which is
then rendered using info_overlays.js.

As part of this migration, we rewrote the way internationalization was
done, since the previous implementation incorrectly did not support
languages with a different word order than English.

We also not consistently use periods at the end of the descriptions.

Co-authored-by: Tim Abbott <tabbott@zulip.com>

Fixes #18504.
2021-06-28 09:14:09 -07:00
aryanshridhar 942f4372f0 info_overlay: Render keyboard shortcuts using info_overlay.
This commits ports the `keyboard_shortcuts.html` file from
using the Django template to handlebars, essentially creating
a new file as `keyboard_shortcuts.hbs` within /static/templates
which is then rendered using info_overlays.js.

Fixes part of #18792.
2021-06-27 18:29:52 -07:00
Abhijeet Prasad Bodas e196ea7e64 event_queue: Consistently user `user_notifications_data` as variable name.
This disambiguates this object from the those sent via the `send_event`
calls, which are also called "user_data".
2021-06-25 08:54:01 -07:00
Abhijeet Prasad Bodas 733e0ae75e notification_data: Rename `sender_id` -> `acting_user_id`.
This better shows the situation for message edits, where we use the same
class.
2021-06-25 08:54:00 -07:00
PIG208 75cea329b4 markdown: Refactor out additional properties added to Message.
This adds a new class called MessageRenderingResult to contain the
additional properties we added to the Message object (like alert_words)
as well as the rendered content to ensure typesafe reference. No
behavioral change is made except changes in typing.

This is a preparatory change for adding django-stubs to the backend.

Related: #18777
2021-06-24 18:14:53 -07:00
Vishnu KS 4e0dac9e6c actions: Update default streams state when user role is changed. 2021-06-24 17:02:05 -07:00
Tim Abbott d7df1f208a test_signup: Add temporary nocoverage marker.
This was introduced in 23530ac81e and
will be used soon, but isn't used yet.
2021-06-24 15:24:15 -07:00
Tim Abbott abf2571133 test_signup: Support passing no password in verify_signup. 2021-06-24 14:55:05 -07:00
Gilbert Bishop-White 23530ac81e test_signup: Refactor test_signup_existing_email to use verify_signup.
Add a new `verify_signup` helper function, which currently implements
enough functionality to be used by `test_signup_existing_email`.

This is the first step towards #7564.
2021-06-24 14:52:25 -07:00
Ganesh Pawar eca2e10c4a test_realm: Extract out test_change_stream_creation_policy.
This is a prep commit in preparation of splitting
create_stream_policy into create_private_stream_policy
and create_public_stream_policy.

This extracts it in a way to make it possible to easily test
different stream policies in the upcoming stream policy split.
2021-06-24 14:08:49 -07:00
Ganesh Pawar 113e27615d test_subs: Extract out test_user_settings_for_adding_streams.
This is a prep commit in preparation of splitting
create_stream_policy into create_private_stream_policy
and create_public_stream_policy.

This extracts it in a way to make it possible to easily test
different stream policies in the upcoming stream policy split.
2021-06-24 14:08:49 -07:00
Ganesh Pawar a1ab79992c test_subs: Extract out test_user_settings_for_creating_streams.
This is a prep commit in preparation of splitting
create_stream_policy into create_private_stream_policy
and create_public_stream_policy.

This extracts it in a way to make it possible to easily test
different stream policies in the upcoming stream policy split.
2021-06-24 14:08:49 -07:00
Ganesh Pawar e74ad23091 test_subs: Remove test_create_stream_policy_setting.
test_create_stream_policy_setting (in class StreamAdminTest) and
test_user_settings_for_creating_streams (in class SubscriptionAPITest)
test essentially the same thing.

So, remove one of them.

Removing test_create_stream_policy_setting makes sense,
since class StreamAdminTest tests things admins can do, whereas
non-admin users can create streams.
2021-06-24 14:08:49 -07:00
Ganesh Pawar aec493a8d3 test_subs: Remove test_invite_to_stream_by_invite_period_threshold.
test_invite_to_stream_by_invite_period_threshold (in class StreamAdminTest)
and test_user_settings_for_subscribing_other_users
(in class SubscriptionAPITest) test essentially the same thing.

So, remove one of them.

Removing test_invite_to_stream_by_invite_period_threshold makes sense,
since class StreamAdminTest tests things admins can do, whereas
non-admin users can invite other users.
2021-06-24 14:08:49 -07:00
Ganesh Pawar d45e7274f9 test_subs: Avoid direct usage of list_to_streams.
This was used to test can_create_stream property of a guest user.
There are better ways to test it, which are already implemented in
test_can_create_streams.
2021-06-24 14:08:48 -07:00
Suyash Vardhan Mathur fe07f67360 apidocs: Standardize parameters text for get-own-user.
We use the "does not accept any parameters" language in the common
template that we'll be migrating to shortly, so we remove this
variance (And adjust its test).
2021-06-24 10:37:56 -07:00
Abhijeet Prasad Bodas 1cf1d147aa event_queue: Move notification trigger logic to notification_data.
This removes some complexity from the event_queue module.
To avoid code duplication, we reduce the `is_notifiable` methods to
internally just call the `trigger` methods and check their return value.
2021-06-24 09:35:18 -07:00
Abhijeet Prasad Bodas 66192825c0 maybe_enqueue_notifications: Take in notification_data dataclass.
* Modify `maybe_enqueue_notifications` to take in an instance of the
dataclass introduced in 951b49c048.

* The `check_notify` tests tested the "when to notify" logic in a way
which involved `maybe_enqueue_notifications`. To simplify things, we've
earlier extracted this logic in 8182632d7e.
So, we just kill off the `check_notify` test, and keep only those parts
which verify the queueing and return value behavior of that funtion.

* We retain the the missedmessage_hook and message
message_edit_notifications since they are more integration-style.

* There's a slightly subtle change with the missedmessage_hook tests.
Before this commit, we short-circuited the hook if the sender was muted
(5a642cea11).
With this commit, we delegate the check to our dataclass methods.
So, `maybe_enqueue_notifications` will be called even if the sender was
muted, and the test needs to be updated.

* In our test helper `get_maybe_enqueue_notifications_parameters` which
generates default values for testing `maybe_enqueue_notifications` calls,
we keep `message_id`, `sender_id`, and `user_id` as required arguments,
so that the tests are super-clear and avoid accidental false positives.

* Because `do_update_embedded_data` also sends `update_message` events,
we deal with that case with some hacky code for now. See the comment
there.

This mostly completes the extraction of the "when to notify" logic into
our new `notification_data` module.
2021-06-24 09:35:17 -07:00
Abhijeet Prasad Bodas dedc39f139 notifications_data: Rename `id` -> `user_id`.
We also make this a mandatory named argument for our test helper
for clarity.
2021-06-24 17:34:50 +05:30
Abhijeet Prasad Bodas f6e705d477 maybe_enqueue_notifications: Require all keyword arguments.
This is a more readable way to call the function.
2021-06-24 17:34:50 +05:30
sahil839 37bf160298 queue_processor: Add langauge to the events added to invites queue.
This is a prep commit for adding realm-level default for various
user settings. We add the language, in which the invite email will
be sent, to the dict added to queue itself to avoid making queries
in a loop when sending multiple emails from queue.

We also handle the case for old events in the queue.
2021-06-22 16:55:32 -07:00
sahil839 6a191a7bf0 queue_processor: Remove email_body from the invite email events.
We removed the use of email_body field in 47fcb27e39, but was
still passed in events from do_resend_user_invite_email and
in tests. So this commit removes the email_body field from
these places.
2021-06-22 16:55:32 -07:00
Abhijeet Prasad Bodas c3fb413119 message edit: Don't send mentioned user_ids in event dict.
We already have this data in the `flags` for each user, so no need to
send this set/list in the event dictionary.

The `flags` in the event dict represent the after-message-update state,
so we can't avoid sending `prior_mention_user_ids`.
2021-06-22 10:27:55 -07:00
Tim Abbott e231a03eff message_edit: Fix non-alternating resolve topic notifications.
Previously, it was possible for an unusual series of topic-edit
actions to result in Notification Bot reporting that a topic was
marked as resolved that had already been marked as resolved, etc.
2021-06-21 12:45:19 -07:00
Tim Abbott b2dd15fd86 message_edit: Reject buggy noop topic edit requests.
A buggy client might send a message_edit request to change the topic
field, sending the current topic as the new value. Previously, we
would treat that as a normal request to edit the topic; now we act as
though the API request had not requested a topic change.  In the
common case that only the topic was in the edit request, this now
results in an error that should help client implementations identify
their bug.

This fixes a bad interaction with the "unresolve topic" logic, which
assumed that upstream logic had verified that the topic was actually
changing.
2021-06-21 12:16:00 -07:00
Abhijeet Prasad Bodas 5c483e3b58 get_active_presence_idle_user_ids: Check notifiability more thoroughly.
* Have the `get_active_presence_idle_user_ids` function look at all the
user data, not just `private_message` and `mentioned`.
* Fix a couple of incorrect `missedmessage_hook` tests, which did not
catch the earlier behaviour.
* Add some comments to the tests for this function for clarity.
* Add a helper to create `UserMessageNotificationsData` objects from the
user ID lists. This will later help us deduplicate code in the event_queue
logic.

This fixes a bug which earlier existed, that if a user turned on stream
notifications, and received a message in that stream which did not mention
them, they wouldn't be in the `presence_idle_users` list, and hence would
never get notifications for that message.

Note that, after this commit, users might still not get notifications in
the above scenarios in some cases, because the downstream logic in the
notification queue consumers sometimes erroneously skips sending
notifications for stream messages.
2021-06-21 10:52:59 -07:00
Abhijeet Prasad Bodas 8182632d7e notification_data: Add methods to determine notifiability.
We will later consistently use these functions to check for notifiable
messages in the message send and event_queue code.

We have these functions accept the `sender_id` so that we can avoid the
`private_message = message["type"] == "private" and user_id != sender_id`
wizardy.
2021-06-21 10:52:59 -07:00
Abhijeet Prasad Bodas 6167c36adc event_queue: Translation code for user data migration.
This is separate from the next commit for ease of testing.
To verify that the compatibility code works correctly, all message send
and event_queue tests from our test suite should pass on just this commit.
2021-06-21 10:52:59 -07:00
Dinesh a71ee405e4 test_example: Add a few more common testing methods.
Adds an example to illustrate the use of:
* self.settings()
* self.assertLogs()
* self.assert_in_success_response()
2021-06-20 07:16:35 -04:00
Tim Abbott 696236b6fc left sidebar: Implement basic resolve topic option.
Fixes part of #18751.
2021-06-18 09:24:48 -07:00
Gaurav Pandey 8fc3715ea8 settings: Remove language_list_dbl_col from page_params.
The language_list_dbl_col parameter in the page_params
is used by only the web client frontend. The value is
calculated in the backend and then passed as a page_param
which is unnecessary considering that the whole process
is beneficial for the front_end only. Hence move the entire
calculation code to the frontend.

Fixes part of #18673.
2021-06-17 17:02:27 -07:00
Gaurav Pandey 56d85fb833 settings: Remove default_language_name from page_params.
default_language_name was a part of page_params which is actually
redundant considering that we already have language_list and
default_language available to frontend which can be used to
get the default_language_name and hence prevents the backend
from sending an additional parameter.

Fixes part of #18673.
2021-06-17 16:56:21 -07:00
Carlos Bederian 1b51792459 email_mirror: Improve filter_footer delimiter detection 2021-06-17 09:11:28 -07:00
Rohitt Vashishtha 98f8d94b25 linkifiers: Add joint validation for linkifier urls and patterns.
We now validate the linkifier urls and patterns together, and add
the following additional checks:

1. All groups in the pattern must be used in the URL format string.
2. All groups in the URL format string must be declared in the pattern.

Linkifier pattern is now validated inside the `clean` method.
`filter_pattern_validator` is moved from `clean_fields` to `clean`
method as a safe check. As a result of this, a Puppeteer test case
is updated.

NOTE: The changes here are IN ADDITION to the existing validations.

Fixes #16482.

Co-authored-by: akshatdalton <akshat.dak@students.iiit.ac.in>
2021-06-16 17:51:33 -07:00
sahil839 38fac6c359 settings: Add moderators and members options in edit_topic_policy.
This commit adds moderators, full members and members options for
edit_topic_policy in both the backend and frontend.
2021-06-16 15:04:29 -07:00
sahil839 02db407398 tests: Pass acting user name to helper functions in topic edit tests.
This is a prep commit for adding members, full members and moderator
options to edit_topic_policy. As we will be adding tests for these
options, we will need to add a login statment repeatedly and this
helps us in avoiding that.
2021-06-16 14:59:47 -07:00
sahil839 611de0faa9 tests: Login as iago in set_message_editing_params.
This is a prep commit for adding moderators, full
members and member roles in edit_topic_policy.

As we add these new options, we will add tests with
user with all these roles and thus we would need to
login as iago repeatedly when changing parameters.
So, to avoid this we instead login as Iago in
set_message_editing_params itself.
2021-06-16 14:59:47 -07:00
sahil839 828759d2ba models: Replace allow_community_topic_editing with edit_topic_policy.
This commit replaces the allow_community_topic_editing boolean with
integer field edit_topic_policy and includes both frontend and
backend changes.

We also update settings_ui.disable_sub_settings_onchange to not
change the color of label as we did previously when the setting
was a checkbox. But now as the setting is dropdown we keep the
label as it is and we don't do anything with label when disabling
dropdowns. Also, this function was used only here so we can safely
change this.
2021-06-16 14:59:36 -07:00
Tim Abbott 9aca1dffcd test_auth_backends: Add types for subclass fields of SocialAuthBase.
These are used in the tests on this base class, so they need to be
defined for mypy to be happy.  Discovered when we enabled Django
stubs.
2021-06-16 08:28:10 -07:00
Riken Shah e91054ef19 context_processors: Add test for `get_apps_page_url`.
This is a follow-up commit to 61b30e61ad.
2021-06-16 07:49:03 -07:00
Aman Agrawal 64f04e9913 web_public_view: Replace `web_public_visitor` with `spectator`.
We use `spectator` term instead of `web_public_visitor` as it is
more concise and better represents this type of users.
2021-06-16 07:46:34 -07:00
Eeshan Garg 512229cf7d plans_page: Fix broken redirection to /accounts/login/.
When an unauthenticated user tries to access the /plans page, we
redirect to /accounts/login/?next=plans (note the missing slash
before "plans"). After the user is authenticated, they are then
redirected to /accounts/login/plans, which is an invalid URL. The
correct URL should be just /plans.

This commit solves this by prefixing the "plans" in the query
parameter with a forward slash, which results in the correct
redirect URL, i.e., /plans.
2021-06-15 16:49:01 -07:00
Abhijeet Prasad Bodas 5a642cea11 missedmessage_hook: Don't enqueue notificationss if sender is muted.
This is a follow up to 71742dce24 to handle
muted senders in the missedmessage_hook too.
2021-06-15 12:30:31 -07:00
Riken Shah b2ab448b71 gear_menu: Extract the gear menu as a handlebars template.
For this extraction, we need to move some context
parameter (from home_real in `views/home.py`) to extra
page_params parameter (of
build_page_params_for_home_page_load in
`lib/home.py`) so handlebars template can access them.

While moving I confirmed that these parameters are not
used elsewhere if some parameter is used elsewhere
(like `apps_page_url`) then I didn't remove it from the
context list, I just added it to the page_params list.

Fixes: #18795.
2021-06-15 12:17:45 -07:00
Ganesh Pawar 1d4b6c1320 about_zulip: Migrate to handlebars.
This results in moving the `zulip_merge_base` parameter to
page_params, so that it's available to JavaScript.

Since this is technically a tiny overlay, it needs to be initialized
before hashchange.js.
2021-06-15 11:55:20 -07:00
Steve Howell e739bee00a poll widget: Add server validation. 2021-06-14 17:46:16 -07:00
Steve Howell 24ed5a2bb4 submessages: Add verify_submessage_sender.
Before this change a rogue actor could try to
widgetize another person's message. (The
rogue actor would already have access to read
the message.)
2021-06-14 17:46:16 -07:00