Commit Graph

5659 Commits

Author SHA1 Message Date
sahil839 6a191a7bf0 queue_processor: Remove email_body from the invite email events.
We removed the use of email_body field in 47fcb27e39, but was
still passed in events from do_resend_user_invite_email and
in tests. So this commit removes the email_body field from
these places.
2021-06-22 16:55:32 -07:00
Abhijeet Prasad Bodas c3fb413119 message edit: Don't send mentioned user_ids in event dict.
We already have this data in the `flags` for each user, so no need to
send this set/list in the event dictionary.

The `flags` in the event dict represent the after-message-update state,
so we can't avoid sending `prior_mention_user_ids`.
2021-06-22 10:27:55 -07:00
Tim Abbott e231a03eff message_edit: Fix non-alternating resolve topic notifications.
Previously, it was possible for an unusual series of topic-edit
actions to result in Notification Bot reporting that a topic was
marked as resolved that had already been marked as resolved, etc.
2021-06-21 12:45:19 -07:00
Tim Abbott b2dd15fd86 message_edit: Reject buggy noop topic edit requests.
A buggy client might send a message_edit request to change the topic
field, sending the current topic as the new value. Previously, we
would treat that as a normal request to edit the topic; now we act as
though the API request had not requested a topic change.  In the
common case that only the topic was in the edit request, this now
results in an error that should help client implementations identify
their bug.

This fixes a bad interaction with the "unresolve topic" logic, which
assumed that upstream logic had verified that the topic was actually
changing.
2021-06-21 12:16:00 -07:00
Abhijeet Prasad Bodas 5c483e3b58 get_active_presence_idle_user_ids: Check notifiability more thoroughly.
* Have the `get_active_presence_idle_user_ids` function look at all the
user data, not just `private_message` and `mentioned`.
* Fix a couple of incorrect `missedmessage_hook` tests, which did not
catch the earlier behaviour.
* Add some comments to the tests for this function for clarity.
* Add a helper to create `UserMessageNotificationsData` objects from the
user ID lists. This will later help us deduplicate code in the event_queue
logic.

This fixes a bug which earlier existed, that if a user turned on stream
notifications, and received a message in that stream which did not mention
them, they wouldn't be in the `presence_idle_users` list, and hence would
never get notifications for that message.

Note that, after this commit, users might still not get notifications in
the above scenarios in some cases, because the downstream logic in the
notification queue consumers sometimes erroneously skips sending
notifications for stream messages.
2021-06-21 10:52:59 -07:00
Abhijeet Prasad Bodas 8182632d7e notification_data: Add methods to determine notifiability.
We will later consistently use these functions to check for notifiable
messages in the message send and event_queue code.

We have these functions accept the `sender_id` so that we can avoid the
`private_message = message["type"] == "private" and user_id != sender_id`
wizardy.
2021-06-21 10:52:59 -07:00
Abhijeet Prasad Bodas 6167c36adc event_queue: Translation code for user data migration.
This is separate from the next commit for ease of testing.
To verify that the compatibility code works correctly, all message send
and event_queue tests from our test suite should pass on just this commit.
2021-06-21 10:52:59 -07:00
Dinesh a71ee405e4 test_example: Add a few more common testing methods.
Adds an example to illustrate the use of:
* self.settings()
* self.assertLogs()
* self.assert_in_success_response()
2021-06-20 07:16:35 -04:00
Tim Abbott 696236b6fc left sidebar: Implement basic resolve topic option.
Fixes part of #18751.
2021-06-18 09:24:48 -07:00
Gaurav Pandey 8fc3715ea8 settings: Remove language_list_dbl_col from page_params.
The language_list_dbl_col parameter in the page_params
is used by only the web client frontend. The value is
calculated in the backend and then passed as a page_param
which is unnecessary considering that the whole process
is beneficial for the front_end only. Hence move the entire
calculation code to the frontend.

Fixes part of #18673.
2021-06-17 17:02:27 -07:00
Gaurav Pandey 56d85fb833 settings: Remove default_language_name from page_params.
default_language_name was a part of page_params which is actually
redundant considering that we already have language_list and
default_language available to frontend which can be used to
get the default_language_name and hence prevents the backend
from sending an additional parameter.

Fixes part of #18673.
2021-06-17 16:56:21 -07:00
Carlos Bederian 1b51792459 email_mirror: Improve filter_footer delimiter detection 2021-06-17 09:11:28 -07:00
Rohitt Vashishtha 98f8d94b25 linkifiers: Add joint validation for linkifier urls and patterns.
We now validate the linkifier urls and patterns together, and add
the following additional checks:

1. All groups in the pattern must be used in the URL format string.
2. All groups in the URL format string must be declared in the pattern.

Linkifier pattern is now validated inside the `clean` method.
`filter_pattern_validator` is moved from `clean_fields` to `clean`
method as a safe check. As a result of this, a Puppeteer test case
is updated.

NOTE: The changes here are IN ADDITION to the existing validations.

Fixes #16482.

Co-authored-by: akshatdalton <akshat.dak@students.iiit.ac.in>
2021-06-16 17:51:33 -07:00
sahil839 38fac6c359 settings: Add moderators and members options in edit_topic_policy.
This commit adds moderators, full members and members options for
edit_topic_policy in both the backend and frontend.
2021-06-16 15:04:29 -07:00
sahil839 02db407398 tests: Pass acting user name to helper functions in topic edit tests.
This is a prep commit for adding members, full members and moderator
options to edit_topic_policy. As we will be adding tests for these
options, we will need to add a login statment repeatedly and this
helps us in avoiding that.
2021-06-16 14:59:47 -07:00
sahil839 611de0faa9 tests: Login as iago in set_message_editing_params.
This is a prep commit for adding moderators, full
members and member roles in edit_topic_policy.

As we add these new options, we will add tests with
user with all these roles and thus we would need to
login as iago repeatedly when changing parameters.
So, to avoid this we instead login as Iago in
set_message_editing_params itself.
2021-06-16 14:59:47 -07:00
sahil839 828759d2ba models: Replace allow_community_topic_editing with edit_topic_policy.
This commit replaces the allow_community_topic_editing boolean with
integer field edit_topic_policy and includes both frontend and
backend changes.

We also update settings_ui.disable_sub_settings_onchange to not
change the color of label as we did previously when the setting
was a checkbox. But now as the setting is dropdown we keep the
label as it is and we don't do anything with label when disabling
dropdowns. Also, this function was used only here so we can safely
change this.
2021-06-16 14:59:36 -07:00
Tim Abbott 9aca1dffcd test_auth_backends: Add types for subclass fields of SocialAuthBase.
These are used in the tests on this base class, so they need to be
defined for mypy to be happy.  Discovered when we enabled Django
stubs.
2021-06-16 08:28:10 -07:00
Riken Shah e91054ef19 context_processors: Add test for `get_apps_page_url`.
This is a follow-up commit to 61b30e61ad.
2021-06-16 07:49:03 -07:00
Aman Agrawal 64f04e9913 web_public_view: Replace `web_public_visitor` with `spectator`.
We use `spectator` term instead of `web_public_visitor` as it is
more concise and better represents this type of users.
2021-06-16 07:46:34 -07:00
Eeshan Garg 512229cf7d plans_page: Fix broken redirection to /accounts/login/.
When an unauthenticated user tries to access the /plans page, we
redirect to /accounts/login/?next=plans (note the missing slash
before "plans"). After the user is authenticated, they are then
redirected to /accounts/login/plans, which is an invalid URL. The
correct URL should be just /plans.

This commit solves this by prefixing the "plans" in the query
parameter with a forward slash, which results in the correct
redirect URL, i.e., /plans.
2021-06-15 16:49:01 -07:00
Abhijeet Prasad Bodas 5a642cea11 missedmessage_hook: Don't enqueue notificationss if sender is muted.
This is a follow up to 71742dce24 to handle
muted senders in the missedmessage_hook too.
2021-06-15 12:30:31 -07:00
Riken Shah b2ab448b71 gear_menu: Extract the gear menu as a handlebars template.
For this extraction, we need to move some context
parameter (from home_real in `views/home.py`) to extra
page_params parameter (of
build_page_params_for_home_page_load in
`lib/home.py`) so handlebars template can access them.

While moving I confirmed that these parameters are not
used elsewhere if some parameter is used elsewhere
(like `apps_page_url`) then I didn't remove it from the
context list, I just added it to the page_params list.

Fixes: #18795.
2021-06-15 12:17:45 -07:00
Ganesh Pawar 1d4b6c1320 about_zulip: Migrate to handlebars.
This results in moving the `zulip_merge_base` parameter to
page_params, so that it's available to JavaScript.

Since this is technically a tiny overlay, it needs to be initialized
before hashchange.js.
2021-06-15 11:55:20 -07:00
Steve Howell e739bee00a poll widget: Add server validation. 2021-06-14 17:46:16 -07:00
Steve Howell 24ed5a2bb4 submessages: Add verify_submessage_sender.
Before this change a rogue actor could try to
widgetize another person's message. (The
rogue actor would already have access to read
the message.)
2021-06-14 17:46:16 -07:00
Vishnu KS 5da3a69505 plans: Mention whether realm is on free trial. 2021-06-14 16:49:53 -07:00
akshatdalton c507931ac8 refactor: Export non-markdown logic in mention.py. 2021-06-14 13:26:30 -07:00
akshatdalton 1a76d06add test_push_notifications: Use responses module to mock HTTP responses. 2021-06-12 07:31:12 -07:00
Mateusz Mandera b1da5d9d76 ldap: Prevent creation of users with malformed email value. 2021-06-11 17:30:46 -07:00
Mateusz Mandera 18dab5619c ldap: Fix ldap access control tests.
These tests didn't configure ldap settings correctly and as a result,
the user involved in these tests wasn't actually hamlet@zulip.com, but a
new, malformed user with email "hamlet" that was being created by the
ldap auto-registration codepath. This wasn't caught because the codepath
didn't validate the email address and thus created such a malformed user
silently.
2021-06-11 17:30:46 -07:00
Abhijeet Prasad Bodas 10dd5f784b event_queue: Don't check for "read" flag when processing events.
* In `event_queue.py`, only the sender and recipient users who have muted
the sender will have the "read" flag set.

* We already skip enqueueing notifications for users who've muted the sender
after 58da384da3.

* The queue consume functions for email and push notifications already
check filter messages which have been read before sending notifications.

* So, the "read" logic in `event_queue.py` is unnecessary, and the
processing power saved from not enqueueing notifications for a single
user should be insignificant, so we remove these checks all toghether.
2021-06-11 08:07:37 -07:00
Abhijeet Prasad Bodas 3fd53d6f48 test_email_notifications: Add test for "read" flag return.
This logic was peviously untested. This is a prep change for us
to completely depend on the logic here for the "read" flag, and
not on the `event_queue` code.
2021-06-11 08:07:36 -07:00
Abhijeet Prasad Bodas 2b438fd7ce test_push_notifications: Fix incorrect "read" flag test.
It was unclear what the original test was testing, and more
importantly, the test passed even if one removed the `read` flag
check in the `handle_push_notifications` function, so we fix it
to be more comprehensive.
2021-06-11 08:05:27 -07:00
Abhijeet Prasad Bodas c6a31dcd9f event_queue: Extract local variables. 2021-06-11 08:05:27 -07:00
Vishnu KS 4e26746b98 tests: Move tests of /activity pages from zerver to analytics module.
This module deals with the testing of /activity, /realm_activity
and /user_activity. All these pages reside in analytics module.
Keeping these tests in zerver/tests is kind is not appropriate
since person who makes changes to /activity pages would not think
it is necessary to run tests in zerver. So better to keep them
in the analytics module.
2021-06-11 07:41:02 -07:00
Vishnu KS 46c902686d tests: Specify the update_fields in test_activity.
This is a prep commit. Else the lint will complain during the
refactoring of this function to a new location that update_fields
is not specified.
2021-06-11 07:41:02 -07:00
Vishnu KS 40ab415005 activity: Show effective rate of realms in /activity page. 2021-06-11 07:41:02 -07:00
Eeshan Garg 50a6887d94 plans_page: Compute sponsorship link for subdomain pages correctly.
Currently, in the FAQ on our /plans page, when the user clicks on
the sponsorship link in the answer for the first question, they
are always taken to /accounts/go, causing them to have to input
their organization URL even if they are on a subdomain page.

This commit makes it so that when the user is on a subdomain page,
they are taken to /upgrade#sponsorship directly. On the other
hand, when they are on a root domain (/) page, they have to go
through /accounts/go and specify their organization's name.
2021-06-10 09:10:38 -07:00
Vishnu KS 1938076f67 billing: Enforce license limit for plans on manual license management. 2021-06-09 17:42:38 -07:00
Anders Kaseorg 342834ee9c python: Simplify stdio flushing using print(…, flush=True).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-09 14:05:31 -07:00
Anders Kaseorg d8adeb66ca docs: Deemphasize Hubot as an integration bridge.
The Hubot project looks to be abandoned; it hasn’t been updated in
years and its own installation instructions don’t work anymore.
Remove our special placement of Hubot alongside Zapier and IFTTT.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-09 14:04:01 -07:00
akshatdalton 3cc6ff57db test_link_embed: Use responses module to mock HTTP responses. 2021-06-09 12:17:26 -07:00
akshatdalton bef7120390 test_slack_importer: Use responses module to mock HTTP responses. 2021-06-09 12:17:26 -07:00
Alex Vandiver 721546dfc0 subdomains: Extend "static" to include resources hosted on S3.
This causes avatars and emoji which are hosted by Zulip in S3 (or
compatible) servers to no longer go through camo.  Routing these
requests through camo does not add any privacy benefit (as the request
logs there go to the Zulip admins regardless), and may break emoji
imported from Slack before 1bf385e35f,
which have `application/octet-stream` as their stored Content-Type.
2021-06-08 15:28:10 -07:00
Abhijeet Prasad Bodas 58da384da3 muting: Don't enqueue notifications for messages from muted senders.
Earlier, the notification-blocking for messages from muted senders
was a side-effect of we never sending notifications for messages
with the "read" flag.

This commit decouples these two things, as a prep for having new
settings which will allow users to **always** receive email
notifications, including when/if they read the message during the
time the notifications is in the queue.

We still mark muted-sender messages as "read" when they are sent,
because that's desirable anyways.
2021-06-08 14:58:14 -07:00
Abhijeet Prasad Bodas 006b92ed6d message send: Plumb sender muters through `get_recipient_info`.
This will make it possible to share this code with the message
update codepath.
2021-06-08 14:58:14 -07:00
Mateusz Mandera c54b48452d saml: Support syncing custom profile fields with SAML attributes.
Fixes #17277.

The main limitation of this implementation is that the sync happens if
the user authing already exists. This means that a new user going
through the sign up flow will not have their custom fields synced upon
finishing it. The fields will get synced on their consecutive log in via
SAML in the future. This can be addressed in the future by moving the
syncing code further down the codepaths to login_or_register_remote_user
and plumbing the data through to the user creation process.

We detail that limitation in the documentation.
2021-06-08 14:47:33 -07:00
Abhijeet Prasad Bodas 30c993451d test_message_edit_notifications: Cleanup irrelevant parameters.
This is in the same spirit as the previous commit.
2021-06-08 11:10:18 -07:00
Abhijeet Prasad Bodas 2bbdd42e1a test_event_queue: Fill-up default data in missedmessage_hook tests.
This allows us to skip sending parameters which are irrelevant
to what we are testing, and only send the specific changed data.
2021-06-08 11:10:18 -07:00