Commit Graph

5313 Commits

Author SHA1 Message Date
m-e-l-u-h-a-n 5ae896758d logging: use assertEqual to assert logging output.
This commit migrates some tests in test_stripe.py and
test_embedded_bot_system.py to use assertEqual instead of
assertRegexpMatches for asserting logs, as suggested here:
https://github.com/zulip/zulip/pull/16818#discussion_r569888862
2021-03-25 01:10:03 -07:00
m-e-l-u-h-a-n ffd4abaa4e logging: migrate test_import_export to use assertLogs.
This commit  migrates some of the backend tests in test_import_export
to use assertLogs(), instead of mock.patch() as planned in #15331.

Logs for tests in this file are suppressed and are not asserted as
that made changes to import/export codebase more fragile. As we
already have checks for the actual functionalities, it made less
sense to assert those logs.
2021-03-25 01:10:03 -07:00
Mateusz Mandera a094b80fb1 test_subs: Remove redundant cache_delete calls.
These caches should be getting automatically invalidated upon
realm.save().
2021-03-25 00:47:42 -07:00
m-e-l-u-h-a-n 2699048208 markdown: Extend user mention syntax to support user_id for mentioning.
Extend our markdown system to support mentioning of users
by id also. Following these changes, it would be possible
to mention users with @**|user_id** and silently mention
using @_**|user_id**.

Main intention for extending the mention syntax is to make
it convenient for bots to mention a users using their ids. It
is to be noted that previous syntax are also supported.

Documentation tweaked by tabbott for better readability.

The changes were tested manually in development server, and also
by adding some new backend and frontend tests.

Fixes: #17487.
2021-03-25 00:44:56 -07:00
akshatdalton 9c6d8d9d81 markdown: Add support to shorten GitHub links.
We add support to shorten links and test their shortening in
well-organized, clean manner that makes it trivial to extend the
GitHub approach for GitLab and perhaps other services.

We only shorten basic types of GitHub links (issue, PR, commit) that
fit a set of simple common patterns; the default behaviour of Autolink
is kept for everything else.

Logic added in frontend and backend Markdown Processor is identical.
This makes easy to extend the logic for other services like GitLab.

Fixes #11895.
2021-03-25 00:39:44 -07:00
Tim Abbott 7e77320953 custom profile fields: Rename s/choice/select/ in tests. 2021-03-24 12:57:55 -07:00
tushar912 b220d29fed custom profile fields: Rename "CHOICE" to "SELECT" in backend.
Rename the "CHOICE" field to "SELECT" in backend. This is
done to improve readability as a prep for the upcoming
"SELECT_MULTIPLE" field.
2021-03-24 12:54:51 -07:00
sahil839 6d419565b7 streams: Show error according to create_stream_policy on failure.
This commit changes the list_to_streams function to raise error
according to create_stream_policy value when a user cannot create
streams instead of same error for all cases.
2021-03-24 12:33:58 -07:00
sahil839 13e412d553 tests: Modify invite_to_stream_policy tests to check all cases.
This commit modifies test_user_settings_for_subscribing_other_users
to check all the possible cases including the cases when a user
can successfully subscribe other users along with the already
tested failure cases. This commit also adds checks for guest users
which was not present before.
2021-03-24 12:33:58 -07:00
sahil839 b45d6ef0d8 tests: Use do_change_user_role and do_set_realm_property.
This commit replaces the code which directly changes user.role,
realm.create_stream_policy and realm.waiting_period_threshold
with do_change_user_role and do_set_realm_property functions
in test_can_create_streams. This makes the code similar to the
other tests.
2021-03-24 12:33:58 -07:00
sahil839 683b6f7d65 tests: Refactor test_can_create_streams and test_can_subscribe_other_users.
We refactor test_can_create_streams and test_can_subscribe_other_users
in test_subs.py. We want to follow a specific order in such tests
which is just set the policy value one by one and then checking
that the role in policy returns true and role just below that returns
false. This approach is explained in detail below.

Following hierarchy of roles is considered for these tests -
1. Realm admin
2. Full members
3. Members
4. Guests.

Then if the policy is set to admins only, we check that the having
role as admin returns true and the role just below that, i.e. full
member returns false. Similarly, if the policy is set to members
only, we check that a member should return true and role below it
which is guest should return false. We basically follow these as
we can assume that if a user with particular role cannot do the
required task, then user with role below in the hierarchy would
be not allowed to do the task too.

This commit refactors the above mentioned two tests to have above
explained workflow.
2021-03-24 12:33:58 -07:00
sahil839 bc72d2dbc4 tests: Remove unnecessary use of do_change_user_role in test_subs.
This commit removes the unnecessary do_change_user_role function
in test_can_subcribe_other_users. This was added in 1aebf3cab
which replaced the multiple functions like do_change_is_admin
and do_change_is_guest with do_change_user_role.

Previously two functions do_change_is_admin and do_change_is_guest
were used because there were two flags is_realm_admin and is_guest
which were used to determine the role of a user. But then we added
a single field role to UserProfile and removed the multiple flags
and thus also replaced the different functions with a single
do_change_user_role. With addition of a new field role, two
different do_change_* functions were not needed as we only have
a role field instead of different flags, but this was missed in
1aebf3cab and this commit fixes it.
2021-03-24 12:33:58 -07:00
Tim Abbott 11ff59aa5e test_home: Fix missing do_change_plan_type import. 2021-03-23 16:12:04 -07:00
Vishnu KS 7bf3d6f21f tests: Use do_change_plan_type in test_show_plans. 2021-03-23 15:51:50 -07:00
shanukun d68a2677d2 event_schema: Add checker for restart event.
Part of #17568.
2021-03-23 12:16:20 -07:00
shanukun cfe0fa3788 event_schema: Add schema check for realm/deactivated event.
This add the schema checker, openapi schema, and also a test for
realm/deactivated event.

With several block comments by tabbott explaining the logic behind our
behavior here.

Part of #17568.
2021-03-23 12:16:16 -07:00
m-e-l-u-h-a-n 830c4acedc markdown: Fix invalid mention bug for stream and stream topic mention.
Modifies `StreamPattern` and `StreamTopicPattern` to inherit
from InlineProcessor instead of Pattern. This change is done
because Pattern stopped checking for matching patterns as soon
as it found a match which was not a valid stream. Due to this
all the subsequent mention failed, even if they were valid.
This bug was only present in backend renderring due to
markdown.inlinepatterns.Pattern.

Due to above changes verbose_compile is no longer used for
precompiling STREAM_LINK_REGEX, STREAM_TOPIC_LINK_REGEX as
adds ^(.*?) and (.*?)$ which cause extra overhead of matching
pattern which is not required. With new InlineProcessor these
extra patterns at beggining and end are not required.
So, StreamPattern and StreamTopicPattern now define their own
__init__ method for precompiling the regex.

Fixes #17535.

These changes were tested locally in dev server and by adding
some new markdown tests to test these.
2021-03-23 01:28:30 -07:00
m-e-l-u-h-a-n dadbba0c25 markdown: Fix invalid mention bug for user group mention.
Modifies `UserGroupMentionPattern` to inherit from InlineProcessor
instead of Pattern. This change is done because Pattern
stopped checking for matching patterns as soon as it found
a match which was not a valid user group. Due to this all
the subsequent user group mention failed, even if they were
valid. This bug was only present in backend renderring due to
markdown.inlinepatterns.Pattern.

This was reported as issue #17535.

These changes were tested locally in dev server and by adding
some new markdown tests to test these.
2021-03-23 01:28:30 -07:00
m-e-l-u-h-a-n c8979a5100 markdown: Fix invalid mention bug for user mention.
Modifies `UserMentionPattern` to inherit from InlineProcessor
instead of Pattern. This change is done because Pattern
stopped checking for matching patterns as soon as it found
a match which was not a valid user. Due to this all the
subsequent user mention failed. This bug was only present in
backend renderring due to markdown.inlinepatterns.Pattern.

This was reported as issue #17535.

These changes were tested locally in dev server and by adding
some new markdown tests to test these.
2021-03-23 01:28:30 -07:00
Vishnu KS 5e0030927b models: Make get_human_admin_users optionally not return realm owners. 2021-03-22 18:10:03 -07:00
Vishnu KS 7a351edb60 models: Make get_admin_users_and_bots optionally not return owners. 2021-03-22 18:10:03 -07:00
Tim Abbott f121e40848 message: Record whether unread_msgs data is truncated.
This is preparatory work for investigating reports of missing unread
messages.

It's a little surprising that not test failed after adding the code
without API documentation.

Co-Author-By: Tushar Upadhyay (tushar912).
2021-03-21 19:48:13 -07:00
Steve Howell 7c9a766c29 tests: Test send_restart_events. 2021-03-21 10:46:55 -04:00
Siddharth Asthana ec28a7555c outgoing_webhook: modify outgoing_webhook's 407 error message.
The message from the bot which triggered the 407 error message notifies
the bot owner about the exceptions as well in the error message. This
commit handles it more gracefully and shows a generic message.
2021-03-18 14:39:26 -07:00
Siddharth Asthana 5ec0860a2f outgoing_webhook: Add bot name when a outgoing_webhook is triggered.
The messages from the bot which were triggered by the outgoing_webhooks
didn't have the bot name in them. This commit adds the bot name to it
and makes the corresponding changes in the tests.
2021-03-18 14:39:26 -07:00
Strifel 209c89be10 ldap: Add option to limit user access to certain realms.
This adds an option for restricting a ldap user
to only be allowed to login into certain realms.
This is done by configuring an attribute mapping of "org_membership"
to an ldap attribute that will contain the list of subdomains the ldap
user is allowed to access. This is analogous to how it's done in SAML.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2021-03-18 11:19:59 -07:00
Gaurav Pandey 44ff6da107 email-integration: Notify user on failure to send message via email.
On replying to an email notifcation from a stream where the user
does not come under the stream_post_policy will subsequently result
in a failure. In such a case, the user does not receive feedback
regarding the failure.

Notify the user via notification bot if their email
message failed to send.
Fixes #16642.
2021-03-17 14:56:17 -07:00
Abhijeet Prasad Bodas a1e75edb8a linkifiers: Remove outdated comment in tests.
After 34e39248fc, we do support
generic GitHub URLs. This test and the comment above
it were added in 043baa2af8.
2021-03-15 11:19:59 -07:00
Tim Abbott e42354c917 do_create_realm: Require passing kwargs by name. 2021-03-14 08:50:02 -07:00
Mateusz Mandera d91d3a05b9 tests: Use do_create_realm where possible.
Using do_create_realm should be preferred over manual creation where
possible, as it creates more realistic data.
2021-03-14 08:50:02 -07:00
Mateusz Mandera 20fd9b1dec tests: Eliminate use of get_user_profile_by_email in tests. 2021-03-11 20:30:57 -08:00
Mateusz Mandera 132ce525b1 test_realm: Fix test_do_deactivate_realm_clears_user_realm_cache.
self.example_user("hamlet") uses get_user_by_delivery_email, so it
doesn't actually cache anything. This should use a cached function, like
the test below: test_do_change_realm_subdomain_clears_user_realm_cache.
2021-03-11 20:30:57 -08:00
Mateusz Mandera faf314c8d3 test_retention: Delete redundant get_user_profile_by_email call.
This does absolutely nothing and must be in the code accidentally.
2021-03-11 20:30:57 -08:00
Tushar912 55de66f944 api: Support user_id in get_user_presence_endpoint.
This is part of our general process of replacing emails, which are not
static with time, with user_ids when referring to users in the API.

We still keep the `email` reference option, since it can be useful for
linking third-party applications to Zulip on an intranet that might
have a user's corporate email handy and not want to do the extra round
trip to lookup the user.

The name of the parameter, user_id_or_email, was chosen to to make it
clear that the default/preferred option is user_id.

Fixes #14304.
2021-03-11 20:14:04 -08:00
Aman Agrawal e587c029f6 display_settings: Allow user to set default_view.
TextField is used to allow users to set long stream + topic narrow
names in the urls.

We currently restrict users to only set "all_messages" and
"recent_topics" as narrows.

This commit achieves 3 things:
* Removes recent topics as the default view which loads when
  hash is empty.
* Loads default_view when hash is empty.
* Loads default_view on pressing escape key when it is unhandled by
  other present UI elements.

NOTE: After this commit loading zulip with an empty hash will
automatically set hash to default_view.  Ideally, we'd just display
the default view without a hash, but that involves extra complexity.

One exception is when user is trying to load an overlay directly,
i.e. zulip is loaded with an overlay hash. In this case,
we render recent topics is background irrespective of default_view.

We consider this last detail to be a bug not important enough to block
adding this setting.
2021-03-11 18:09:08 -08:00
Alex Vandiver 1926daa244 import/export: Fix black formatting. 2021-03-08 13:08:35 -08:00
Cyril Pletinckx ba7da6d5c0 import/export: Fix deprecated authentication method for Slack.
The query string parameter authentication method is now deprecated for
newly created Slack applications since the 24th of February[1].  This
causes Slack imports to fail, claiming that the token has none of the
required scopes.

Two methods can be used to solve this problem: either include the
authentication token in the header of an HTTP GET request, or include
it in the body of an HTTP POST request. The former is preferred, as
the code was already written to use HTTP GET requests.

Change the way the parameters are passed to the "requests.get" method
calls, to pass the token via the `Authorization` header.

[1] https://api.slack.com/changelog/2020-11-no-more-tokens-in-querystrings-for-newly-created-apps

Fixes: #17408.
2021-03-08 12:56:37 -08:00
Rex Ferrer d4c0578560 refactor: Integrate POSTRequestMock into HostRequestMock.
Minimized code duplication by integrating POSTRequestMock into
HostRequestMock and then updating the required files with
HostRequestMock.

Fixes part of #1211.
2021-03-03 21:52:05 -08:00
sahil839 4ca21a6982 users: Give moderators same permissions as that of full members.
This commit updates the stream creation, subscribing others to
stream, wildcard mention settings and stream post policy to allow
realm moderators even if they are new and the respective setting
is set to allow full members only.
2021-03-02 17:19:31 -08:00
sahil839 b4fd15d516 models: Rename is_new_member to is_provisional_member.
This commit renames the is_new_member property in models.py
to is_provisional_member which will return true for any user
who is not a full member. We will add a condition in further
commit such that this returns 'False' for a moderator as we
will initially give all the rights to moderator that a full
member has.
2021-03-02 17:19:31 -08:00
Mateusz Mandera 6f9f608225 test_home: Fix wrong bot references in test_people.
These are all referring to email_gateway_bot, when they're supposed to
refer to the notification and welcome bots, respectively. The values are
the same though, so the tests were passing anyway.
2021-02-28 17:02:37 -08:00
Sumanth V Rao 829f9272d2 hotspots: Extract INTRO_HOTSPOTS from ALL_HOTSPOTS.
Its likely that we would implement new hotspots that aren't
a part of the tutorial hotspots, in the future. For instance,
a hotspot to advertise new features. Hence, grouping them into
categories like INTRO_HOTSPOTS would be a good start. We also
have an aggregate of all types of hotspots we may add in the
future, under ALL_HOTSPOTS.
2021-02-26 15:02:48 -08:00
Mateusz Mandera 4b903c5dcd invites: Fix bug revoking user invites in other realms than intended.
Fixes #17238.
In process_new_human user, the queries were wrong, revoking all invites
sent to the email address, even in other realms than the one where the
new account just got created.
2021-02-26 08:26:43 -08:00
Mateusz Mandera b9c1fed18c invites: Delete old compat code in the invites queue worker.
1.7.* is old enough at this point that we can clean up this code.
2021-02-26 08:26:43 -08:00
shanukun 4b67946605 refactor: Make acting_user a mandatory kwarg for do_create_user. 2021-02-25 17:58:00 -08:00
Alex Vandiver e53be6d043 email: Set an envelope-from which may be different from the From: field.
The envelope-from is used by the MTA if the destination address is not
deliverable.  Route all such mail to the noreply address.
2021-02-24 17:32:28 -08:00
Mateusz Mandera 1d4badf6ad tests: Test internal_send_private_message can send to cross-realm bots. 2021-02-23 15:26:47 -08:00
Mateusz Mandera 51d7f24d20 actions: Remove realm argument to internal_send_stream_message.
The argument is redundant.
2021-02-23 15:26:47 -08:00
Mateusz Mandera 09fc79f911 actions: Remove realm argument to internal_send_private_message.
The argument is redundant.
2021-02-23 15:26:47 -08:00
Mateusz Mandera a652573169 tests: Fix tests causing internal_send_private_message with wrong realm.
test_signup: This test was wrong, because the inviter UserProfile was
from a different realm. Such a PreregistrationUser shouldn't be
considered valid.

test_tutorial: The direct call to internal_send_private_message was
using sender's realm as the realm argument which is not valid. It
doesn't lead to any error because the codepath seems to mostly not care
about the realm arg if the sender is a cross-realm bot. From my reading
of the code I think that wrong realm arg here would break user mentions,
because it makes its way to check_message() and then to
build_message_send_dict - but overall the message gets sent without
errors. Either way, this was a bug in the test and should be fixed.
2021-02-23 15:26:47 -08:00