Commit Graph

225 Commits

Author SHA1 Message Date
Steve Howell e660428c21 Rename missed_message_userids to presence_idle_userids. 2017-10-02 15:19:28 -07:00
Steve Howell 1b518f1983 Return mentioned users in get_user_info_for_message_updates().
The dictionary result for get_user_info_for_message_updates()
now has a `mention_user_ids` field that is a set of user ids
who were mentioned in a message.
2017-09-27 16:01:50 -07:00
Steve Howell 646abb57b7 refactor: Extract get_user_info_for_message_updates.
We'll want to expand this to get users that were mentioned in
the prior message, but this commit is just a refactoring.
2017-09-27 16:01:50 -07:00
rht 1e87a4b68c zerver/tests: Remove absolute_import. 2017-09-27 10:00:39 -07:00
Tim Abbott c62befe655 test_messages: Fix invalid subdomain in mirrored message test. 2017-09-25 07:20:58 -07:00
Steve Howell 0966bf1a48 Simplify get_stream_cache_key().
Before this commit, we could pass in either a Realm object
or a realm_id to get_stream_cache_key().  Now we consistently
pass it a realm_id.
2017-09-20 10:31:33 -07:00
Tim Abbott a1ddd934b4 bots: Fix calculation of bot domain with REALMS_HAVE_SUBDOMAINS.
Previously, the bot domain was calculated correctly in most
circumstances, but if you were using the root domain, it would be
e.g. ".chat.zulip.org", not "chat.zulip.org".  We fix this, with
perhaps more use of setting REALMS_HAVE_SUBDOMAINS than would be ideal
if we weren't about to set that True unconditionally.
2017-09-16 02:49:15 -07:00
Steve Howell 7159f34bed Avoid getting realm emoji unless there is emoji syntax. 2017-09-15 01:09:08 -07:00
Steve Howell 8a941d03ae Avoid querying streams unless there are stream links. 2017-09-15 01:09:08 -07:00
Steve Howell c3032a7fe8 Avoid looking up emails when rendering messages.
We now fetch email -> id mappings for messages ONLY if
it potentially uses the !avatar(foo@example.com) syntax.
2017-09-15 01:09:08 -07:00
Sarah c3a8138f74 user_settings: Add push notifications for all stream messages.
Add setting to enable push notifications for all stream messages.
2017-09-14 05:41:37 -07:00
Steve Howell 82b2bd8b65 Take user_ids in get_userids_for_missed_messages().
This helps us phase out the need for getting lots of UserProfile
objects.
2017-09-12 04:22:55 -07:00
Steve Howell 262abe41ab Add a performance test for do_send_messages(). 2017-09-12 04:22:55 -07:00
Steve Howell f5edeb01ae Calculate idle users more efficiently when sending messages.
Usually a small minority of users are eligible to receive missed
message emails or mobile notifications.

We now filter users first before hitting UserPresence to find idle
users.  We also simply check for the existence of recent activity
rather than borrowing the more complicated data structures that we
use for the buddy list.
2017-09-07 06:59:44 -07:00
Tim Abbott 1c8c5cc36f test_messages: Fix deactivation tests for new /me behavior. 2017-08-27 09:58:02 -07:00
Tim Abbott d18f389bb5 test_messages: Set subdomain explicitly in most tests. 2017-08-26 13:45:27 -07:00
Tim Abbott 5fa7c3e0b2 test_classes: Clean up API for unsubscribing.
The old API required test authors to unnecessarily repeat themselves.
2017-08-24 21:37:57 -07:00
Tim Abbott eb720485c5 tests: Add and use new self.subscribe.
This new method cleans up the API for subscribing to something from a
test case.
2017-08-24 21:37:57 -07:00
Tim Abbott 80458945c7 tests: Remove unnecessary use of get_realm_by_email_domain.
That function was primarily intended for some limited registration
code paths, not the generic usage it was getting.
2017-08-24 20:22:56 -07:00
Steve Howell 81e3f489f2 Use sender realm in user_profiles_from_unvalidated_emails.
This change is mostly based on a similar commit from hackerkid
in a feature branch.  It borrows both code and ideas.  Some of
it's my own stuff, as I was working on a newer branch.

We now call get_user_including_cross_realm_email() inside of
user_profiles_from_unvalidated_emails(), instead of using
get_user_profile_by_email.

This requires a few of our callers to pass down sender into us.

One consequence of this change is that we change the symptoms
for trying to send to emails outside of your realm.  In some
cases, we simply raise an error that an email is invalid to us
instead of getting into the deeper validate_recipient_user_profiles
check.
2017-08-22 10:42:15 -07:00
Steve Howell 30d37d1270 Add Addressee class.
This class simplifies the calling sequence to methods like
check_message and _internal_prep_message, and it's also more
type safe.

Checking for message types is encapsulated with calls to is_stream()
and is_private().  There are also shortcut constructors when you
know that the type of the address (stream vs. private), which is often.
2017-08-22 10:42:15 -07:00
Aditya Bansal 7a2667ce53 soft_deactivation: Fix flaky tests by seeding message for users.
In this we basically seed a single message for the user which will
be soft deactivated by sending a stream message / group PM to
ensure that is has at least one UserMessage row, since in real
world every human user will always have at least one User Message
row.
2017-08-18 07:02:11 -04:00
Umair Khan 7bb9ac0014 result.json: Upgrade test_messages. 2017-08-17 09:03:35 -07:00
Umair Khan 70996de415 test_messages: Upgrade to HttpResponse.json(). 2017-08-16 07:06:58 -07:00
Aditya Bansal 0cb909b978 events: Fill in missing messages for a returing soft_deactivated user. 2017-08-15 08:33:16 -07:00
Aditya Bansal 658a14d0bb soft_deactivation: Stop creating UserMessage rows when soft deactivated. 2017-08-15 08:33:16 -07:00
Aditya Bansal 57858596ab soft_deactivation: Add new function 'add_missing_messages'.
This function will help us in creating undisturbed experience for
returning soft deactivated users.

Tweaked by tabbott to fix minor performance and clarity issues.
2017-08-15 08:29:51 -07:00
neiljp (Neil Pilgrim) 334348cbca mypy: Use namedtuples for request objects in test_messages.py. 2017-08-05 17:53:18 -07:00
Tim Abbott 7778a66171 test_messages: Add a test for unauthorized mirroring of PMs. 2017-07-28 15:55:34 -07:00
Tim Abbott 8d0b743a6f test_messages: Add a test for PMs to deactivated users. 2017-07-28 15:53:33 -07:00
Tim Abbott a9e987f31d test_messages: Add a test for private messages copying self. 2017-07-28 15:49:27 -07:00
Steve Howell 69fd967c5d topics history: Simplify data in /topics endpoint.
In anticipation of have all unread message ids available to the
web app in page_params (via a separate effort), we are simplifying
the /topics endpoint to no longer return unread counts.

Instead we have a list of tiny dictionaries with these fields:

    name - name of the topic
    max_id - max message id for the topic (aka most recent)

The items in the list are order by most-recent-topic-first.
2017-07-27 14:26:22 -07:00
Durga Akhil Mundroy 146dfa6f0b org-permissions: Add allow_edit_history organiztion setting.
This new setting controls whether or not users are allowed to see the
edit history in a Zulip organization.  It controls access through 2
key mechanisms:

* For long-ago edited messages, get_messages removes the edit history
  content from messages it sends to clients.

* For newly edited messages, clients are responsible for checking the
  setting and not saving the edit history data.  Since the webapp was
  the only client displaying it before this change, this just required
  some changes in message_events.js.

Significantly modified by tabbott to fix some logic bugs and add a
test.
2017-07-16 10:10:06 -07:00
Aditya Bansal ce70ded488 pep8: Add compliance with rule E261 to tests/test_messages.py. 2017-07-11 15:17:32 -07:00
Jack Zhang af3b7d4c98 message-edit: Patch html tags after edits on new lines. 2017-06-19 16:43:11 -04:00
Jack Zhang 98a785bb25 message_edit: Replace highlighting replaced text with delete / insert.
Rationale: For the more off-to-the-side edit history view, changes
are easier to digest by highlighting deleted content in red followed
immediately by added and changed content in green.

TODO: Toggle for showing the edited messages without highlighting;
deleted content would not be shown in this view.
2017-06-07 11:06:15 -07:00
Tim Abbott 73f8653da6 test_messages: Increase time allowed for bulk send.
This fixes an occasional test flake we've been seeing recently.
2017-06-01 00:00:36 -07:00
Umair Khan 5d794d08dd test_bulk_message_fetching: Add debug code.
This test fails on self.assertTrue(delay < 0.001 * num_ids, error_msg)
randomly. This commit adds debug code to see what the real values of
paramters are.
2017-05-31 09:27:30 -07:00
Umair Khan 71f97b7bcb testing: Invalidate cache before counting queries.
To get accurate count of the queries, we should make sure that
caches don't come into play. If we count queries while caches are
filled, we will get a lower count. Caches are not supposed to be
persistent, so our test can also fail if cache is invalidated
during the course of the unit test.

This commit solves the problem with Stream cache. This cache comes
into play when we use `get_stream` function. If cache is valid,
we will not issue queries to Stream and Recipient table. I think
the problem was one of those rare occasions when the Stream cache
got invalidated during the course of the test, due to which query
count was increased by 2. After this commit, we intentially invalidate
the Stream cache.
2017-05-30 17:28:41 +05:00
K.Kanakhin 2434f2d96c messages: Add support for admins deleting messages.
This makes it possible for Zulip administrators to delete messages.
This is primarily intended for use in deleting early test messages,
but it can solve other problems as well.

Later we'll want to play with the permissions model for this, but for
now, the goal is just to integrate the feature.

Note that it saves the deleted messages for some time using the same
approach as Zulip's message retention policy feature.

Fixes #135.
2017-05-29 21:59:38 -07:00
vaibhav 9cf9837f12 webhooks: Add outgoing webhook bot user to development database. 2017-05-29 16:01:23 -07:00
Vishnu Ks bb98e35aa0 Replace othello@zulip.com with example_email('othello'). 2017-05-24 19:37:36 -07:00
Vishnu Ks c4db3b7d1c Replace cordelia@zulip.com with example_email('cordelia'). 2017-05-24 19:37:36 -07:00
Vishnu Ks 961b35d52e Replace iago@zulip.com with example_email('iago'). 2017-05-24 19:37:36 -07:00
Vishnu Ks 5230eaef1c Replace hamlet@zulip.com with example_email('hamlet'). 2017-05-24 19:37:36 -07:00
Rick Chern e53d1c3885 tests: Remove get_user_profile_by_email from most tests. 2017-05-24 13:05:19 -07:00
Vishnu Ks 4403d179df tests: Replace mit_user().email with mit_email(). 2017-05-24 12:44:43 -07:00
umkay ccc70445d6 mypy: Fix strict-optional errors for test files.
Fix mypy --strict-optional errors in zerver/tests
2017-05-24 12:43:28 -07:00
Tim Abbott a833fa39b8 test_messages: Fix tests failing due to error message change. 2017-05-23 15:27:21 -07:00
Vishnu Ks 820dc9dd9a Replace espuser@mit.edu with mit_user('espuser'). 2017-05-22 19:02:42 -07:00
Vishnu Ks 99fc0e9e62 Replace starnine@mit.edu with mit_user('starnine'). 2017-05-22 19:02:42 -07:00
Vishnu Ks c680c5f1e8 Replace sipbtest@mit.edu with mit_user('sipbtest'). 2017-05-22 19:02:42 -07:00
Steve Howell 7f9057ba99 tests: Use example_user() in more places. 2017-05-08 11:57:38 -07:00
Steve Howell 6bc8424c71 Use self.example_user() in more places.
This fixes most cases where we were assigning a user to
the var email and then calling get_user_profile_by_email with
that var.

(This was fixed mostly with a script.)
2017-05-08 11:57:38 -07:00
Steve Howell 942db9b6c5 tests: Added ZulipTestCase.example_user() function.
The example_user() function is specifically designed for
AARON, hamlet, cordelia, and friends, and it allows a concise
way of using their built-in user profiles. Eventually, the
widespread use of example_user() should help us with refactorings
such as moving the tests users out of the "zulip.com" realm
and deprecating get_user_profile_by_email.
2017-05-08 11:57:38 -07:00
hackerkid b2504084ab Replace timezone.now with timezone_now. 2017-04-16 12:28:56 -07:00
Tim Abbott a417fd3c0b MessageDictTest: Fix nondeterministic query counts.
This fixes an issue with a nondeterministic number of database queries
being used in fetching bulk messages from the database.  The source of
the problem was that we were fetching _all_ messages, not just the 600
that had been created by the test, and thus if the set of streams
present in messages in the test fixtures (which is random) changes,
the number of streams used (and thus number of queries) would change.
2017-04-14 09:55:27 -07:00
Umair Khan cc96cefe75 test_messages: Change assert_max_length to assert_length. 2017-04-04 17:07:06 -07:00
Harshit Bansal ac2172e233 models: Rename RealmAlias model to RealmDomain.
Includes a migration.
2017-04-04 15:48:03 -07:00
Tim Abbott 9a9f0f86e9 test_messages: Fix sometimes incomplete coverage in StarTests.
This test had nondeterministically incomplete test coverage.
2017-03-28 11:41:45 -07:00
Rishi Gupta 30024d0a8f models: Remove Realm.domain. 2017-03-25 19:55:48 -07:00
Rishi Gupta b416587aab Change sender_domain to sender_realm_str in message dict. 2017-03-25 19:50:24 -07:00
Tim Abbott 3617ebfd41 api: Rename get_old_messages to get_messages in the backend.
Fixes #1315.
2017-03-23 23:52:44 -07:00
Tim Abbott 767f57ef03 tests: Remove imports of deleted message_ids. 2017-03-18 20:40:34 -07:00
Susan Salituro a2d948f2e0 test_messages: Add test for invalid markdown format. 2017-03-18 14:35:53 -07:00
Rishi Gupta a2bfa7793a models: Use string_id instead of domain in Realm.__unicode__. 2017-03-13 09:23:07 -07:00
Tim Abbott 60296c022c test_messages: Attempt to fix flaky coverage again. 2017-03-09 21:42:22 -08:00
Rishi Gupta 3797fa657e Change bot domains to string_id.EXTERNAL_HOST.
Change applies to both subdomains and non-subdomains case, though we use
just the EXTERNAL_HOST in the non-subdomains case if there is only 1 realm.

Fixes #3903.
2017-03-08 16:26:13 -08:00
Tim Abbott ff7e3a2125 tests: Move ExtractedRecipientsTest test to test_messages.py. 2017-03-08 03:57:37 -08:00
Tim Abbott 6a8dfe1f3f test_messages: Fix nondeterministic coverage issue. 2017-03-07 20:44:55 -08:00
Raghav Jajodia a3a03bd6a5 mypy: Added Dict, List and Set imports.
Fixed mypy errors associated with the upgrade.
2017-03-04 14:33:44 -08:00
Rishi Gupta 1453a5bfda Change string_id of test zephyr realm from mit to zephyr.
Also changes Realm.is_zephyr_mirror_realm to use string_id=zephyr instead of
domain=mit.edu.
Part of a larger migration away from Realm.domain.
2017-03-04 12:18:01 -08:00
PhilSk 53f3d84af2 attachment: Add 'size' field tracking size of uploaded files.
This tracking will make it possible in the future to limit the total
size of uploads on a per-user or per-organization basis.

Fixes #3774.
2017-03-01 15:58:21 -08:00
Tim Abbott ff37524db6 messages: Add test coverage for IRC mirroring code path. 2017-02-22 20:51:25 -08:00
Tim Abbott 8875deff47 test_messages: Add a message edit permissions test. 2017-02-22 20:51:25 -08:00
Tim Abbott 98894eb759 edit_history: Reverse the order of edit history list. 2017-02-19 16:47:07 -08:00
Tim Abbott eadd6fb3c6 message_edit: Include an initial entry in view output.
This makes it super easy for frontend code using this view code to
produce a nice display of the history.

This also fixes an off-by-one error with the timestamps.
2017-02-19 16:46:47 -08:00
Tim Abbott 515340ed00 Add message edit history backend with tests.
Based on work by Kartik Maji in #1204.

This has a few significant changes from the original version:
* We correctly handle filling in data for topic edits
* Has a complete test suite verifying correctness of the logic
* Currently, it doesn't include a special "start" entry

Things we may want to further change include:
* Adding a special "start" entry.
* Reversing the order of the history data returned for clarity.
2017-02-19 16:13:35 -08:00
Tim Abbott 498f6782a2 EditMessageEdit: Add complete test of edit history. 2017-02-19 16:13:35 -08:00
Tim Abbott 6ba1cd797b Fix regression in whitespace stripping at the start of messages.
This fixes a regression in 4060a97656,
which incorrectly stripped whitespace at the start of messages as
well.

Fixes #3719.
2017-02-18 14:47:28 -08:00
Umair Khan 6db4879f9c testing: Clear cache in queries_captured.
This results in a more deterministic result when we count queries.
2017-02-13 14:24:48 -08:00
Tim Abbott 4060a97656 messages: Strip trailing whitespace in message contents.
I dug into why we never did this before, and it turns out we did, but
using `$.trim()` (which removes leading whitespace as well!).  When
removing the `$.trim()` usage.

Fixes #3294.
2017-02-11 23:01:22 -08:00
Raghav Jajodia c8451097be tests: Replace Stream.objects.get() with get_stream().
This should have a slight perf benefit from using caching, and also in
some cases cleans up the code a bit.

Fixes #3284.
2017-01-27 15:04:13 -08:00
Tim Abbott 4e171ce787 lint: Clean up E126 PEP-8 rule. 2017-01-23 22:06:13 -08:00
Tim Abbott d6e38e2a5c lint: Clean up E123 PEP-8 rule. 2017-01-23 21:34:26 -08:00
Rishi Gupta 494c1a2b55 Remove unnecessary uses of Realm.domain in zerver/tests. 2017-01-09 11:26:08 -08:00
Rishi Gupta 89af775e03 test_classes: Remove domain from ZulipTestCase.register. 2017-01-06 19:45:08 -08:00
Rafid Aslam 88ec999cf3 tests: Bump up max length queries in test_bulk_message_fetching().
Bump up max length queries in `test_bulk_message_fetching()` to 11
in `zerver/tests/test_messages.py` to avoid test failing when run
this test alone.

Fixes #3087.
2017-01-05 17:30:47 -08:00
Steve Howell 99b5c00ec1 Add stream id to message dictionary.
We now pass the stream id in the messages we send to
our clients.
2017-01-05 15:32:45 -08:00
Rishi Gupta 2b0a7fd0ba Rename models.get_realm_by_string_id to get_realm.
Finishes the refactoring started in c1bbd8d. The goal of the refactoring is
to change the argument to get_realm from a Realm.domain to a
Realm.string_id. The steps were

* Add a new function, get_realm_by_string_id.

* Change all calls to get_realm to use get_realm_by_string_id instead.

* Remove get_realm.

* (This commit) Rename get_realm_by_string_id to get_realm.

Part of a larger migration to remove the Realm.domain field entirely.
2017-01-04 17:12:23 -08:00
Rishi Gupta 130f981901 messages.send_message_backend: Take a realm_str instead of a domain. 2017-01-03 16:46:15 -08:00
Rishi Gupta 50d67d9a13 test_messages: Add warning about running test_bulk_message_fetching alone.
Running this test fails if it is not run as a part of the larger
test_messages suite. This commit adds a comment to that effect.
2017-01-03 16:25:20 -08:00
Rishi Gupta 717afcb408 Remove calls to get_realm in preparation for its deprecation.
Also removes two calls to email_to_domain.
2016-12-26 17:53:32 -08:00
Juanvulcano 2fd1316d70 Removed json/update_message
Update test_alert_words.py

Update test_messages.py

Update messages.py
2016-12-24 10:08:22 -08:00
Rafid Aslam bfe84a10f0 API: Migrate POST `json/fetch_raw_message` to GET `json/messages/<id>` 2016-12-21 13:08:40 -08:00
Tim Abbott 66bce06a5a lint: Fix E127 vilations due to recent assertEquals migration. 2016-12-15 20:51:27 -08:00
Tim Abbott bf80873d4f tests: s/assertRaisesRegexp/assertRaisesRegex/ due to deprecation. 2016-12-15 17:11:42 -08:00
Tim Abbott a116c86f62 tests: s/assertEquals/assertEqual/ due to deprecation.
Fixes #2730.
2016-12-15 17:02:03 -08:00
Tim Abbott 0b33be50f3 lint: fix some whitespace issues in new reactions code. 2016-12-14 20:37:13 -08:00
Kracekumar R 61d2297c17 Add reactions in the /json/messages endpoint. 2016-12-14 19:21:04 -08:00