Commit Graph

5300 Commits

Author SHA1 Message Date
Vishnu KS 7bf3d6f21f tests: Use do_change_plan_type in test_show_plans. 2021-03-23 15:51:50 -07:00
shanukun d68a2677d2 event_schema: Add checker for restart event.
Part of #17568.
2021-03-23 12:16:20 -07:00
shanukun cfe0fa3788 event_schema: Add schema check for realm/deactivated event.
This add the schema checker, openapi schema, and also a test for
realm/deactivated event.

With several block comments by tabbott explaining the logic behind our
behavior here.

Part of #17568.
2021-03-23 12:16:16 -07:00
m-e-l-u-h-a-n 830c4acedc markdown: Fix invalid mention bug for stream and stream topic mention.
Modifies `StreamPattern` and `StreamTopicPattern` to inherit
from InlineProcessor instead of Pattern. This change is done
because Pattern stopped checking for matching patterns as soon
as it found a match which was not a valid stream. Due to this
all the subsequent mention failed, even if they were valid.
This bug was only present in backend renderring due to
markdown.inlinepatterns.Pattern.

Due to above changes verbose_compile is no longer used for
precompiling STREAM_LINK_REGEX, STREAM_TOPIC_LINK_REGEX as
adds ^(.*?) and (.*?)$ which cause extra overhead of matching
pattern which is not required. With new InlineProcessor these
extra patterns at beggining and end are not required.
So, StreamPattern and StreamTopicPattern now define their own
__init__ method for precompiling the regex.

Fixes #17535.

These changes were tested locally in dev server and by adding
some new markdown tests to test these.
2021-03-23 01:28:30 -07:00
m-e-l-u-h-a-n dadbba0c25 markdown: Fix invalid mention bug for user group mention.
Modifies `UserGroupMentionPattern` to inherit from InlineProcessor
instead of Pattern. This change is done because Pattern
stopped checking for matching patterns as soon as it found
a match which was not a valid user group. Due to this all
the subsequent user group mention failed, even if they were
valid. This bug was only present in backend renderring due to
markdown.inlinepatterns.Pattern.

This was reported as issue #17535.

These changes were tested locally in dev server and by adding
some new markdown tests to test these.
2021-03-23 01:28:30 -07:00
m-e-l-u-h-a-n c8979a5100 markdown: Fix invalid mention bug for user mention.
Modifies `UserMentionPattern` to inherit from InlineProcessor
instead of Pattern. This change is done because Pattern
stopped checking for matching patterns as soon as it found
a match which was not a valid user. Due to this all the
subsequent user mention failed. This bug was only present in
backend renderring due to markdown.inlinepatterns.Pattern.

This was reported as issue #17535.

These changes were tested locally in dev server and by adding
some new markdown tests to test these.
2021-03-23 01:28:30 -07:00
Vishnu KS 5e0030927b models: Make get_human_admin_users optionally not return realm owners. 2021-03-22 18:10:03 -07:00
Vishnu KS 7a351edb60 models: Make get_admin_users_and_bots optionally not return owners. 2021-03-22 18:10:03 -07:00
Tim Abbott f121e40848 message: Record whether unread_msgs data is truncated.
This is preparatory work for investigating reports of missing unread
messages.

It's a little surprising that not test failed after adding the code
without API documentation.

Co-Author-By: Tushar Upadhyay (tushar912).
2021-03-21 19:48:13 -07:00
Steve Howell 7c9a766c29 tests: Test send_restart_events. 2021-03-21 10:46:55 -04:00
Siddharth Asthana ec28a7555c outgoing_webhook: modify outgoing_webhook's 407 error message.
The message from the bot which triggered the 407 error message notifies
the bot owner about the exceptions as well in the error message. This
commit handles it more gracefully and shows a generic message.
2021-03-18 14:39:26 -07:00
Siddharth Asthana 5ec0860a2f outgoing_webhook: Add bot name when a outgoing_webhook is triggered.
The messages from the bot which were triggered by the outgoing_webhooks
didn't have the bot name in them. This commit adds the bot name to it
and makes the corresponding changes in the tests.
2021-03-18 14:39:26 -07:00
Strifel 209c89be10 ldap: Add option to limit user access to certain realms.
This adds an option for restricting a ldap user
to only be allowed to login into certain realms.
This is done by configuring an attribute mapping of "org_membership"
to an ldap attribute that will contain the list of subdomains the ldap
user is allowed to access. This is analogous to how it's done in SAML.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2021-03-18 11:19:59 -07:00
Gaurav Pandey 44ff6da107 email-integration: Notify user on failure to send message via email.
On replying to an email notifcation from a stream where the user
does not come under the stream_post_policy will subsequently result
in a failure. In such a case, the user does not receive feedback
regarding the failure.

Notify the user via notification bot if their email
message failed to send.
Fixes #16642.
2021-03-17 14:56:17 -07:00
Abhijeet Prasad Bodas a1e75edb8a linkifiers: Remove outdated comment in tests.
After 34e39248fc, we do support
generic GitHub URLs. This test and the comment above
it were added in 043baa2af8.
2021-03-15 11:19:59 -07:00
Tim Abbott e42354c917 do_create_realm: Require passing kwargs by name. 2021-03-14 08:50:02 -07:00
Mateusz Mandera d91d3a05b9 tests: Use do_create_realm where possible.
Using do_create_realm should be preferred over manual creation where
possible, as it creates more realistic data.
2021-03-14 08:50:02 -07:00
Mateusz Mandera 20fd9b1dec tests: Eliminate use of get_user_profile_by_email in tests. 2021-03-11 20:30:57 -08:00
Mateusz Mandera 132ce525b1 test_realm: Fix test_do_deactivate_realm_clears_user_realm_cache.
self.example_user("hamlet") uses get_user_by_delivery_email, so it
doesn't actually cache anything. This should use a cached function, like
the test below: test_do_change_realm_subdomain_clears_user_realm_cache.
2021-03-11 20:30:57 -08:00
Mateusz Mandera faf314c8d3 test_retention: Delete redundant get_user_profile_by_email call.
This does absolutely nothing and must be in the code accidentally.
2021-03-11 20:30:57 -08:00
Tushar912 55de66f944 api: Support user_id in get_user_presence_endpoint.
This is part of our general process of replacing emails, which are not
static with time, with user_ids when referring to users in the API.

We still keep the `email` reference option, since it can be useful for
linking third-party applications to Zulip on an intranet that might
have a user's corporate email handy and not want to do the extra round
trip to lookup the user.

The name of the parameter, user_id_or_email, was chosen to to make it
clear that the default/preferred option is user_id.

Fixes #14304.
2021-03-11 20:14:04 -08:00
Aman Agrawal e587c029f6 display_settings: Allow user to set default_view.
TextField is used to allow users to set long stream + topic narrow
names in the urls.

We currently restrict users to only set "all_messages" and
"recent_topics" as narrows.

This commit achieves 3 things:
* Removes recent topics as the default view which loads when
  hash is empty.
* Loads default_view when hash is empty.
* Loads default_view on pressing escape key when it is unhandled by
  other present UI elements.

NOTE: After this commit loading zulip with an empty hash will
automatically set hash to default_view.  Ideally, we'd just display
the default view without a hash, but that involves extra complexity.

One exception is when user is trying to load an overlay directly,
i.e. zulip is loaded with an overlay hash. In this case,
we render recent topics is background irrespective of default_view.

We consider this last detail to be a bug not important enough to block
adding this setting.
2021-03-11 18:09:08 -08:00
Alex Vandiver 1926daa244 import/export: Fix black formatting. 2021-03-08 13:08:35 -08:00
Cyril Pletinckx ba7da6d5c0 import/export: Fix deprecated authentication method for Slack.
The query string parameter authentication method is now deprecated for
newly created Slack applications since the 24th of February[1].  This
causes Slack imports to fail, claiming that the token has none of the
required scopes.

Two methods can be used to solve this problem: either include the
authentication token in the header of an HTTP GET request, or include
it in the body of an HTTP POST request. The former is preferred, as
the code was already written to use HTTP GET requests.

Change the way the parameters are passed to the "requests.get" method
calls, to pass the token via the `Authorization` header.

[1] https://api.slack.com/changelog/2020-11-no-more-tokens-in-querystrings-for-newly-created-apps

Fixes: #17408.
2021-03-08 12:56:37 -08:00
Rex Ferrer d4c0578560 refactor: Integrate POSTRequestMock into HostRequestMock.
Minimized code duplication by integrating POSTRequestMock into
HostRequestMock and then updating the required files with
HostRequestMock.

Fixes part of #1211.
2021-03-03 21:52:05 -08:00
sahil839 4ca21a6982 users: Give moderators same permissions as that of full members.
This commit updates the stream creation, subscribing others to
stream, wildcard mention settings and stream post policy to allow
realm moderators even if they are new and the respective setting
is set to allow full members only.
2021-03-02 17:19:31 -08:00
sahil839 b4fd15d516 models: Rename is_new_member to is_provisional_member.
This commit renames the is_new_member property in models.py
to is_provisional_member which will return true for any user
who is not a full member. We will add a condition in further
commit such that this returns 'False' for a moderator as we
will initially give all the rights to moderator that a full
member has.
2021-03-02 17:19:31 -08:00
Mateusz Mandera 6f9f608225 test_home: Fix wrong bot references in test_people.
These are all referring to email_gateway_bot, when they're supposed to
refer to the notification and welcome bots, respectively. The values are
the same though, so the tests were passing anyway.
2021-02-28 17:02:37 -08:00
Sumanth V Rao 829f9272d2 hotspots: Extract INTRO_HOTSPOTS from ALL_HOTSPOTS.
Its likely that we would implement new hotspots that aren't
a part of the tutorial hotspots, in the future. For instance,
a hotspot to advertise new features. Hence, grouping them into
categories like INTRO_HOTSPOTS would be a good start. We also
have an aggregate of all types of hotspots we may add in the
future, under ALL_HOTSPOTS.
2021-02-26 15:02:48 -08:00
Mateusz Mandera 4b903c5dcd invites: Fix bug revoking user invites in other realms than intended.
Fixes #17238.
In process_new_human user, the queries were wrong, revoking all invites
sent to the email address, even in other realms than the one where the
new account just got created.
2021-02-26 08:26:43 -08:00
Mateusz Mandera b9c1fed18c invites: Delete old compat code in the invites queue worker.
1.7.* is old enough at this point that we can clean up this code.
2021-02-26 08:26:43 -08:00
shanukun 4b67946605 refactor: Make acting_user a mandatory kwarg for do_create_user. 2021-02-25 17:58:00 -08:00
Alex Vandiver e53be6d043 email: Set an envelope-from which may be different from the From: field.
The envelope-from is used by the MTA if the destination address is not
deliverable.  Route all such mail to the noreply address.
2021-02-24 17:32:28 -08:00
Mateusz Mandera 1d4badf6ad tests: Test internal_send_private_message can send to cross-realm bots. 2021-02-23 15:26:47 -08:00
Mateusz Mandera 51d7f24d20 actions: Remove realm argument to internal_send_stream_message.
The argument is redundant.
2021-02-23 15:26:47 -08:00
Mateusz Mandera 09fc79f911 actions: Remove realm argument to internal_send_private_message.
The argument is redundant.
2021-02-23 15:26:47 -08:00
Mateusz Mandera a652573169 tests: Fix tests causing internal_send_private_message with wrong realm.
test_signup: This test was wrong, because the inviter UserProfile was
from a different realm. Such a PreregistrationUser shouldn't be
considered valid.

test_tutorial: The direct call to internal_send_private_message was
using sender's realm as the realm argument which is not valid. It
doesn't lead to any error because the codepath seems to mostly not care
about the realm arg if the sender is a cross-realm bot. From my reading
of the code I think that wrong realm arg here would break user mentions,
because it makes its way to check_message() and then to
build_message_send_dict - but overall the message gets sent without
errors. Either way, this was a bug in the test and should be fixed.
2021-02-23 15:26:47 -08:00
sahil839 d71afc5a26 actions: Include ROLE_MODERATOR in realm_user_count_by_role.
This commmit includes ROLE_MODERATOR in realm_user_count_by_role.

We also update test_change_role in test_audit_log.py to include
changes for moderator role as well.
2021-02-23 15:01:14 -08:00
sahil839 6b5cf231a1 users: Add new user 'shiva' as realm moderator.
Note that at this point, it's not possible to create moderator users;
this just will make it easier to write tests for logic involving them
as we develop the feature.
2021-02-23 15:00:49 -08:00
sahil839 15e74a637c tests: Check cases when full members and their bots can send messages.
Currently there are only tests for verifying the error case and there
are no tests to check the case where messages are sent successfully
in 'STREAM_POST_POLICY_RESTRICT_NEW_MEMBERS' stream.

This commit adds tests for checking that full members and bots owned
by them can send message successfully in streams with post policy as
'STREAM_POST_POLICY_RESTRICT_NEW_MEMBERS'.
2021-02-18 18:38:52 -08:00
sahil839 3df87d0901 stream: Fix error handling in access_stream_for_send_message.
According to tests we should not allow bot without owners to
post in streams with STREAM_POST_POLICY_RESTRICT_NEW_MEMBERS.
But the code does not handle this and the related test passes
and raises error for case of bots without owner because the bot
is itself a new member.

This commit fixes this by adding a condition to check if there
is no bot owner and then raise error if there is no owner.
2021-02-18 18:38:52 -08:00
Tushar912 dfafdda9b3 api: Add REST API endpoint for looking up a user by email address.
Add new rest api endpoint GET users/{email} for looking up a user by
email, which is useful especially for corporate API applications that
might already have a user's email address.

Fixes #14302.
2021-02-15 17:38:33 -08:00
Anders Kaseorg d001676728 streams: Fix compose_views type safety.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-15 17:05:28 -08:00
Anders Kaseorg dd2a3b45cd test_service_bot_system: Strengthen for_all_bot_types decorator type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-15 17:05:28 -08:00
Anders Kaseorg 04a5e0c339 test_report: Avoid Any type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-15 17:05:28 -08:00
Shanu 7f196967ad event_queue: Remove internal fields being leaked to the API.
A few internal fields used for tracking which types of notifications
have already been sent for a given message, like `hander_id` and the
`push_notified` bundle of fields were being incorrectly included in
message events delivered to clients clients.

One could argue these fields might be useful hints to clients, but
because notifications can be triggered later on via
`missedmessage_hook`, they have no useful purpose in the API.

This commit move these extended event field on a `internal_data`
object within the event object, and delete this field in `contents()`
for call points that would serve data to clients.

Tweaked by tabbott to provide a cleaner interface.

We're not bumping API_FEATURE_LEVEL because these fields have always
been documented as being present only due to a bug, so no clients
should be expecting or relying on them.

Fixes: #15947.
2021-02-14 21:42:19 -08:00
Anders Kaseorg 6e4c3e41dc python: Normalize quotes with Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 11741543da python: Reformat with Black, except quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 5028c081cb python: Merge concatenated string literals that Black would uglify.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Mateusz Mandera b8c8ea5262 tests: Fix bugs confusing recipient.type_id for other ids.
These tests were accidentally passing due to numbers coinciding.
2021-02-09 17:45:34 -08:00