Commit Graph

4916 Commits

Author SHA1 Message Date
Mohit Gupta 9a10929a6c tests: Verify warning log if multiple teams found in mattermost import.
Uses assertLogs to prevent spam in test-backend output.
2020-07-22 17:12:28 -07:00
Vishnu KS 67bacd6e31 billing: Don't allow guest users to upgrade. 2020-07-22 16:57:49 -07:00
Vishnu KS cb01a7f599 billing: Restrict access to billing page to realm owners and billing admins. 2020-07-22 16:57:49 -07:00
Steve Howell a6519e7b8f event_schema: Extract check_user_group_add. 2020-07-22 16:48:19 -07:00
Steve Howell 3f25e52667 event_schema: Extract check_user_status. 2020-07-22 16:48:19 -07:00
Steve Howell 631adc5677 event_schema: Extract check_alert_words. 2020-07-22 16:48:19 -07:00
Steve Howell 0a9a9d8258 event_schema: Extract check_custom_profile_fields. 2020-07-22 16:48:19 -07:00
Steve Howell 7176b90882 event_schema: Extract check_typing_start. 2020-07-22 16:48:19 -07:00
Steve Howell 5f3ea0a659 event_schema: Extract check_invites_changed. 2020-07-22 16:48:19 -07:00
Steve Howell ec17091521 event_schema: Extract check_submessage. 2020-07-22 16:48:19 -07:00
Steve Howell 92136d738a event_schema: Extract check_reaction. 2020-07-22 16:48:19 -07:00
Steve Howell 5209de0261 event_schema: Extract check_update_message_flags. 2020-07-22 16:48:19 -07:00
Steve Howell f2bc22e869 event_schema: Extract check_update_message*. 2020-07-22 16:48:19 -07:00
Steve Howell b81f3433d8 event_schema: Extract check_message. 2020-07-22 16:48:19 -07:00
Steve Howell 385050de20 event_schema: Extract check_realm_bot_(delete/remove).
It is strange that we have both of these events.
2020-07-22 16:48:19 -07:00
Steve Howell 96f5ab1c87 event_schema: Extract check_realm_bot_update. 2020-07-22 16:48:19 -07:00
Steve Howell f5c4ee4477 event_schema: Extract check_realm_bot_add.
Note that we use the actual integer bot_type
value now to determine how we validate
services.
2020-07-22 16:48:19 -07:00
Steve Howell 0a6ce36ac9 event_schema: Extract check_update_global_notifications. 2020-07-22 16:48:19 -07:00
Steve Howell 96b821684b event_schema: Extract check_update_display_settings. 2020-07-22 16:48:19 -07:00
Steve Howell dd5949274d event_schema: Extract check_subscription_peer_*. 2020-07-22 16:48:19 -07:00
Steve Howell 502f1b9fe2 event_schema: Extract check_subscription_remove. 2020-07-22 16:48:19 -07:00
Steve Howell 055f1a590d event_schema: Extract check_subscription_add. 2020-07-22 16:48:19 -07:00
Steve Howell b116f1e911 event_schema: Extract check_stream_update. 2020-07-22 16:48:19 -07:00
Steve Howell 14aa87a168 event_schema: Extract check_stream_create. 2020-07-22 16:48:19 -07:00
Steve Howell a6796e9e86 event_schema: Extract check_realm_update. 2020-07-22 16:48:19 -07:00
Steve Howell e49acfa637 event_schema: Extract event_schema.py.
Obviously, this file will soon grow--this
was the easiest way to start without introducing
noise into other commits.

It will soon be structurally similar
to frontend_tests/node_tests/lib/events.js--I
have some ideas there.  But this should also
help for things like API docs.
2020-07-22 16:48:19 -07:00
Steve Howell a908f5a693 test_events: Improve check_events_dict.
We add the ability to supply optional_keys,
and we don't mutate the list of required
keys that gets passed into us.

We also enforce that there is a "type"
field.

(We will use optional_keys soon.)
2020-07-22 16:48:19 -07:00
Vinit Singh 308cf8ac00 markdown: Inline Youtube previews instead of appending it to the end.
This change makes our handling of youtube-url previews consistent
with how we handle our inline images. This allows the previews to
render next to the paragraph that links to the youtube video.

Follow-up to PR #15773.
2020-07-22 16:11:17 -07:00
Mateusz Mandera 6a50911032 auth: Allow signing in to an existing account with noreply github email.
In particular importing gitter data leads to having accounts with these
noreply github emails. We generally only want users to have emails that
we can actually send messages to, so we'll keep the old behavior of
disallowing sign up with such an email address. However, if an account
of this type already exists, we should allow the user to have access to
it.
2020-07-22 15:50:43 -07:00
Emilio López 7b35234c7b
email_mirror: Fix exception handling unstructured headers.
This commit rewrites the way addresses are collected. If
the header with the address is not an AddressHeader (for instance,
Delivered-To and Envelope-To), we take its string representation.

Fixes: #15864 ("Error in email_mirror - _UnstructuredHeader has no attribute addresses").
2020-07-22 12:11:25 -07:00
Gittenburg 45e19dd6b9 emoji: Rename :slight_smile: to 😄.
Zulip converts :) to the 1F642 Unicode emoji and promotes the same emoji
in the popular section of the emoji picker.

Previously Zulip has labeled 1F642 as "slight smile". While that name
conforms to the Unicode standard (which describes the code point as
SLIGHTLY SMILING FACE), it didn't match our use case of the emoji.

If a user types :) or selects the first smile in the emoji picker they
probably mean to express a regular "smile" and not a "slight smile",
which raises the question why they are only smiling slightly.

This commit relabels 1F642 as 😄 and our previous 😄 263A as
:smiling_face:. Note that 263A looks different in our three supported
emoji sets, so it is not suited to be our "default smile".

This change does not require a migration since our emoji system stores
both unicode points and names and handles name changes transparently.
2020-07-21 16:49:54 -07:00
Mohit Gupta c65729511a tests: Mock print() for management command tests.
This avoids spam in test-backend output.
2020-07-21 16:22:36 -07:00
Mohit Gupta a7db38c6c9 tests: Verify error logs in mirror dummy user tests.
This commit verify that error logs are logging using assertLogs to avoid
spam in the output in ./tools/test-backend.
2020-07-21 16:22:26 -07:00
Tim Abbott 9efec1f929 auth: Clean up DevAuthBackend error messages.
We had a user confused by these error messages, which suggested they
needed to enable something rather than using a development
environment.
2020-07-21 12:55:11 -07:00
Mohit Gupta b2745f6e41 tests: Verify info logs logging in test_fix_unreads.
This commit verifies info logging in test_fix_unreads using assertLogs
so that the logging do not spam ./tools/test-backend output.
2020-07-21 12:22:21 -07:00
Mohit Gupta 363cd8c099 tests: Make errors stream for ERROR_BOT to report in test_report_error.
ERROR_BOT setting is not None during testing, so running
test_report_error without making errors stream was causing exception.
This commit make a stream name errors thus removes exception and error
log spam caused by it in ./tools/test-backend output.
2020-07-21 12:22:21 -07:00
Mohit Gupta 4c561653b3 tests: Verify info logs while testing send_to_email_mirror command.
This commit tests if info logs are logging using assertLogs so that the
log do not spam the test output.
2020-07-21 12:22:21 -07:00
Mohit Gupta c02e011be4 tests: Verify error logging of Internal Server Error in simulate_error.
This commits verify error logs while simulate_error simulates an error
using assertLogs so that the logs do not spam the test ouptut.
2020-07-21 12:22:21 -07:00
Mohit Gupta 40d59f7cf4 tests: Verify error logs while data export for zulip.
This commit verify that error logging while testing data export in
test_notify_realm_export_on_failure using assertLogs so that the logs
do not spam test output.
2020-07-21 12:22:21 -07:00
Mohit Gupta ef5ad080e2 tests: Verify logging on error in check_send_webhook_fixture_message.
This commit tests if error logs are logged when an error occurs during
testing of check_send_webhook_fixture_message using assertlogs. Using
assertlogs ensure logs are not printed as spam in test output.
2020-07-21 12:22:21 -07:00
Mohit Gupta c7a68216cd tests: Verify logs of incoming webhook profile api key validation.
This commit verify warning logs while testing validate_api_key and
profile is incoming webhook but is_webhook is not set to True.
Verification is done using assertLogs so that logs does not cause spam
by printing in the test output.
2020-07-21 12:22:21 -07:00
Mohit Gupta 60ee3ce68a tests: Verify error logs by log_and_report function.
This commit verify error logs printed during testing of log_and report
function using assertLogs without printing it in test output and hence
avoiding spam.
2020-07-21 12:22:21 -07:00
Mohit Gupta 4ef0d0b40c tests: Add assertLogs for test_auth_backends.
This commit tests logging of warning log using assertLogs instead of
printing it in test output hence avoiding spam in test output.
2020-07-21 12:22:07 -07:00
Anders Kaseorg aa16208fd8 dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-20 10:56:31 -07:00
Rohitt Vashishtha fb2946aaf6 Revert "markdown: Remove paragraphs that only contain a tweet link."
This reverts commit d3770153a6.

We do not show a link to the tweet in our preview, so we should revert
to our previous behavior for now.
2020-07-17 14:30:22 -07:00
Tim Abbott 6be3fca037 auth: Remove short_name from LDAP API.
As best I can tell, we fetched this field and then ignored it, so
unlike the last few commits, this is more a code cleanup than a
functional change.
2020-07-17 11:28:08 -07:00
Steve Howell c44500175d database: Remove short_name from UserProfile.
A few major themes here:

    - We remove short_name from UserProfile
      and add the appropriate migration.

    - We remove short_name from various
      cache-related lists of fields.

    - We allow import tools to continue to
      write short_name to their export files,
      and then we simply ignore the field
      at import time.

    - We change functions like do_create_user,
      create_user_profile, etc.

    - We keep short_name in the /json/bots
      API.  (It actually gets turned into
      an email.)

    - We don't modify our LDAP code much
      here.
2020-07-17 11:15:15 -07:00
Steve Howell c60f4236a9 api: Do not require short_name to create user.
When you post to /json/users, we no longer
require or look at the short_name parameter,
since we don't use it in any meaningful way.

An upcoming commit will eliminate it from the
database.
2020-07-17 11:15:15 -07:00
Steve Howell b375581f58 api: Remove (sender_)short_name from message APIs. 2020-07-17 11:15:14 -07:00
Steve Howell e6974d3013 reaction tests: Clean up optional parameters.
This fixes up some complex helpers that may
have had some value before f-strings come along,
but they mostly obscured the logic for
people reading the tests.

We still keep really simple helpers for the
common cases, but there are no optional
parameters for them.

One goal of this fix is to remove the
short_name concept, and we just explicitly
set senders everywhere we need them.

We also now have each test just explicitly set
its reaction_type.

For cases where we have custom message ids
or senders, we just inline the simple call
to api_post.
2020-07-17 11:04:54 -07:00
Steve Howell 3b2c881ce6 tests: Decouple test_retention and test_reactions.
We generally want to avoid having two sibling test
suites depend on each other, unless there's a real
compelling reason to share code.  (And if there is
code to share, we can usually promote it to either
test_helpers or ZulipTestCase, as I did here.)

This commit is also prep for the next commit, where
I try to simplify all of the helpers in EmojiReactionBase.

Especially now that we have f-strings, it is usually
better to just call api_post explicitly than to
obscure the mechanism with thin wrappers around
api_post.  Our url schemes are pretty stable, so it's
unlikely that the helpers are actually gonna prevent
future busywork.
2020-07-17 11:04:54 -07:00
Steve Howell f13327223d tests: Extract get_user_id helper for import tests.
The new helper sets us up to drop short_name.
2020-07-17 11:04:54 -07:00
Steve Howell 7d9f24c58c mypy: Fix Realm types in test_import_export.
It's not clear to me why these passed mypy
before, given this:

    def assert_realm_values(f: Callable[[Realm], Any], ...

But this is clearly more accurate.
2020-07-17 11:04:54 -07:00
Steve Howell 2a9cb4c8cf dead code: Remove legacy reaction actions. 2020-07-17 04:40:01 -04:00
Steve Howell 2374e25b94 import: Import AlertWord table. 2020-07-16 08:50:31 -07:00
Steve Howell 318c55e030 export: Export AlertWord table. 2020-07-16 08:50:31 -07:00
Anders Kaseorg c2f9db4602 logo: Update Zulip logo.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-16 01:37:08 -07:00
Tim Abbott 289819fb70 push_notifications: Reduce log level for weird warning.
This issue isn't something a system administrator needs to take action
on -- it's a likely minor logic bug around organization
administrators moving topics between streams.

As a result, it shouldn't send error emails to administrators.
2020-07-16 01:22:03 -07:00
Rohitt Vashishtha b64ba98e90 markdown: Use unicode ellipses for collapsing spoilers.
We had initially implemented this feature using `(...)` but `(…)` is the
better variation.
2020-07-15 23:30:28 -07:00
Rohitt Vashishtha 7d5d90b484 markdown: Format spoilers for emails.
This is a hacky fix to avoid spoiler content leaking in emails. The
general idea here is to tell people to open Zulip to view the actual
message in full.

We create a mini-markdown parser here that strips away the fence content
that has the 'spoiler' tag for the text emails.

Our handling of html emails is much better in comparison where we can
use lxml to parse the spoiler blocks.
2020-07-15 23:30:28 -07:00
Tim Abbott 2f66c825a2 push_notifications: Disable badge counts.
We include tests for the new implementation to avoid churning the
codebase too much so this can be easily reverted when we are able to
re-enable the feature.
2020-07-15 22:19:53 -07:00
Steve Howell 7c0fa3aefc populate_db: Add alert words for Zulip users. 2020-07-15 12:33:08 -07:00
Steve Howell 63b15b77a3 alert_word tests: Use hamlet for all tests.
The tests had a bunch of different ways to create
users; now we are consistent.  (This is a bit of
a prep step, too, to allow us to easily clean
Hamlet's existing words before each test.)
2020-07-15 12:33:08 -07:00
Rohitt Vashishtha 1a9a478e5d markdown: Assert we handle timestamps sensibly in push notifications.
We could certainly do better with the handling here, but using the raw
string that the user gave us is okayish for now.

Proper formatting of timestamps requires handling locales and timezones
of the receiver as well which is a larger project.
2020-07-15 11:18:32 -07:00
Rohitt Vashishtha 78c48935ca markdown: Format spoilers for push notifications.
We now do something sensible for spoilers in notifications. A message
like:

    ```spoiler Luke's father is
    Vader. Don't tell anyone else.
    ```

would be rendered as:

    Luke's father is (...)
2020-07-15 11:17:38 -07:00
Tim Abbott 7ade8fae78 test_message_send: Fix invalid UserProfile creation. 2020-07-15 10:19:41 -07:00
orientor 40f76bb030 openapi_py: Add function for validating requests.
Add a function for validating requests made during tests in `openapi.py`.
2020-07-14 14:23:58 -07:00
Aman Agrawal e22885a6bf push_notifications: Return if push_notify already active.
If the push_notification for the UserMessage is already active,
we don't send any push notification to the user. This may
happen due to race conditions.

Added and fixed test cases affected by this.
2020-07-14 00:35:29 -07:00
Rohitt Vashishtha 37c0577f8a markdown: Assert twitter previews render inside spoilers.
Previously, we rendered the twitter previews outside of a
spoiler block at the end of the message. The commit series
ending with this commit fixes that by inlining twitter
previews instead of appending them all at the end. As a
consequence of the inlining, we have fixed the issue here.
This commit just adds a test to assert that.

Fixes #15518.
2020-07-13 12:24:32 -07:00
Rohitt Vashishtha d3770153a6 markdown: Remove paragraphs that only contain a tweet link.
This is similar to our behavior with image previews, and helps
reduce clutter in the final rendered html.

We add the string 'Tweet: ' to our existing tests so those tests
remain the same.
2020-07-13 12:24:32 -07:00
Rohitt Vashishtha 87e01cd1fa markdown: Inline Twitter previews instead of appending at end.
This commit makes our handling of twitter previews consistent with
how we handle our inline images so that tweets render next to the
paragraph that links to the tweet.
2020-07-13 12:24:32 -07:00
Mohit Gupta 79243f7d52 tests: Move test_near_pm_message_url to test_message_fetch.py. 2020-07-08 14:00:05 -07:00
Mohit Gupta 16c3121bc8 tests: Move test_is_private_flag_not_leaked to test_message_flags.py. 2020-07-08 14:00:05 -07:00
Mohit Gupta b0b3cf86ce tests: Move test_auto_subbed_to_personals test to test_signup.py.
This is really a test about account creation.
2020-07-08 13:59:56 -07:00
Mohit Gupta 478956ff0b tests: Move MiscMessageTest to test_audit_log.py. 2020-07-08 13:53:56 -07:00
Mohit Gupta f03c5e35e4 tests: Move NoRecipientIDsTest to test_subs.py. 2020-07-08 13:51:32 -07:00
Mohit Gupta 086e9e281b models: Remove to_log_dict method of Message model as it's unused. 2020-07-07 17:14:10 -07:00
Mohit Gupta de81b92dee tests: Move MessageVisibilityTest to test_message_fetch.py. 2020-07-07 17:14:10 -07:00
Mohit Gupta 012622aa4a tests: Move CheckMessageTest to test_message_send.py. 2020-07-07 17:14:10 -07:00
Mohit Gupta 4ae6cc7328 tests: Move MessageHasKeywordsTest to test_message_fetch.py. 2020-07-07 17:14:10 -07:00
Mohit Gupta ce948e4a80 tests: Move MessageAccessTests to test_message_flags.py. 2020-07-07 17:14:10 -07:00
Mohit Gupta 096ed90ac2 tests: Rename test_unread.py to test_message_flags.py. 2020-07-07 17:14:10 -07:00
Mohit Gupta aa7dc6bf83 tests: Move SewMessageAndReactionTest to test_message_dict.py. 2020-07-07 17:14:10 -07:00
Mohit Gupta 228fb9f80d tests: Move TestAddressee to test_message_send.py. 2020-07-07 17:14:10 -07:00
Mohit Gupta 329265b1bf tests: Move TestCrossRealmPMs to test_message_send.py. 2020-07-07 17:14:10 -07:00
Mohit Gupta 308643cace tests: Rename test_narrow.py to test_message_fetch.py.
This commit renames test_narrow.py and also all its references to
test_message_fetch.py.
2020-07-07 17:14:10 -07:00
Rohitt Vashishtha 912e372c4e markdown: Remove !avatar() and !gravatar() syntax.
This particular commit has been a long time coming. For reference,
!avatar(email) was an undocumented syntax that simply rendered an
inline 50px avatar for a user in a message, essentially allowing
you to create a user pill like:

`!avatar(alice@example.com) Alice: hey!`

---

Reimplementation

If we decide to reimplement this or a similar feature in the future,
we could use something like `<avatar:userid>` syntax which is more
in line with creating links in markdown. Even then, it would not be
a good idea to add this instead of supporting inline images directly.

Since any usecases of such a syntax are in automation, we do not need
to make it userfriendly and something like the following is a better
implementation that doesn't need a custom syntax:

`![avatar for Alice](/avatar/1234?s=50) Alice: hey!`

---

History

We initially added this syntax back in 2012 and it was 'deprecated'
from the get go. Here's what the original commit had to say about
the new syntax:

> We'll use this internally for the commit bot.  We might eventually
> disable it for external users.

We eventually did start using this for our github integrations in 2013
but since then, those integrations have been neglected in favor of
our GitHub webhooks which do not use this syntax.

When we copied `!gravatar` to add the `!avatar` syntax, we also noted
that we want to deprecate the `!gravatar` syntax entirely - in 2013!

Since then, we haven't advertised either of these syntaxes anywhere
in our docs, and the only two places where this syntax remains is
our game bots that could easily do without these, and the git commit
integration that we have deprecated anyway.

We do not have any evidence of someone asking about this syntax on
chat.zulip.org when developing an integration and rightfully so- only
the people who work on Zulip (and specifically, markdown) are likely
to stumble upon it and try it out.

This is also the only peice of code due to which we had to look up
emails -> userid mapping in our backend markdown. By removing this,
we entirely remove the backend markdown's dependency on user emails
to render messages.

---

Relevant commits:

- Oct 2012, Initial commit        c31462c278
- Nov 2013, Update commit bot     968c393826
- Nov 2013, Add avatar syntax     761c0a0266
- Sep 2017, Avoid email use       c3032a7fe8
- Apr 2019, Remove from webhook   674fcfcce1
2020-07-07 10:39:44 -07:00
arpit551 a2428017e9 audit_log: Log RealmAuditLog in do_set_realm_authentication_methods.
Log RealmAuditLog in do_set_realm_authentication_methods and added
tests for it.
2020-07-06 17:38:20 -07:00
arpit551 ba92666dbd audit_log: Log RealmAuditLog in do_set_realm_property.
Log RealmAuditLog in do_set_realm_property and do_remove_realm_domain.
Tests for the changes are written in test_events because it will save
duplicate code for test_change_realm_property.
2020-07-06 17:38:19 -07:00
arpit551 6af337c07e audit_log: Log RealmAuditLog when stream is deactivated. 2020-07-06 17:32:11 -07:00
arpit551 e0c9b0fdff audit_log: Log RealmAuditLog when Stream is created.
Added new Event Type in AbstractRealmAuditLog STREAM_CREATED.
Since we finally create streams in create_stream_if_needed function
in zerver/lib/streams.py so logged realm_audit there.
Passed acting_user when create_stream_if_needed or ensure_stream
function  is called.
Added tests in test_audit_log.
2020-07-06 17:32:11 -07:00
arpit551 8807cddc90 audit_log: Correctly add acting_user in bulk_remove_subscriptions.
Earlier we were not adding acting_user while creating RealmAuditLog
objects in bulk_remove_subscriptions. Improved tests_subscriptions
to check it.
2020-07-06 17:32:11 -07:00
arpit551 87aaa84b42 audit_log: Log acting_user in do_change_user_role. 2020-07-06 17:32:11 -07:00
arpit551 01f12b9fc2 audit_log: Log acting_user in user creation and user activation. 2020-07-06 17:32:09 -07:00
arpit551 653928bdfe audit_log: Log acting_user in do_change_avatar_fields. 2020-07-06 17:24:18 -07:00
arpit551 2279fef316 audit_log: Log acting_user in do_change_logo_source. 2020-07-06 17:24:18 -07:00
Mohit Gupta 98cff4ecbf tests: Move SoftDeactivationMessageTest to test_soft_deactivation.py.
This commit moves out the SoftDeactivationMessageTest out of
test_messages.py (which at the moment have mixed category of tests) into
a more logical file, test_soft_deactivation.py.
2020-07-06 16:36:36 -07:00
Mohit Gupta 146b21a34a tests: Move display_recipient fetching test to test_message_dict.py.
This commit moves TestMessageForIdsDisplayRecipientFetching class which
have tests regarding display_recipient filled in by MessageDict to
test_message_dict.py.
2020-07-06 16:36:36 -07:00
Mohit Gupta b21f59e2f3 tests: Move MessageHydrationTest to test_message_dict.py.
MessageHydrationTest tests hydration done by MessageDict, so moving it
out of test_messages.py to test_message_dict.py.
2020-07-06 16:36:36 -07:00
Mohit Gupta eee5318327 tests: Move InternalPrepTest to test_message_send.py.
This commit moves InternalPrepTest test class to test_message_send.py
because it tests internal_send_* and internal_prep_* functions which are
used for internal message sending in zulip.
2020-07-06 16:36:36 -07:00