Commit Graph

246 Commits

Author SHA1 Message Date
Eeshan Garg 057ff9c91e models: Add Stream.history_public_to_subscribers.
This commit adds a new field history_public_to_subscribers to the
Stream model, which serves a similar function to the old
settings.PRIVATE_STREAM_HISTORY_FOR_SUBSCRIBERS; we still use that
setting as the default value for new streams to avoid breaking
backwards-compatibility for those users before we are ready with an
actual UI for users to choose directly.

This also comes with a migration to set the value of the new field for
existing streams with an algorithm matching that used at runtime.

With significant changes by Tim Abbott.

This is an initial part of our efforts on #9232.
2018-04-28 22:54:04 -07:00
Tim Abbott 127ac0df54 auth: Remove unnecessary case from validate_email_for_realm.
The removed code path was only needed due to buggy setup code in the
test_cross_realm_scenarios test.  We address that with a less buggy
workaround, and which lets us remove unnecessary complexity from this
important validation function.

Thanks for Umair Waheed for some preliminary work on this.

Fixes #7561.
2018-04-28 11:03:03 -07:00
Tim Abbott a3fc7d1371 message_edit: Fix spelling of "has passed". 2018-04-27 19:24:14 -07:00
YJDave 6bef44a9fa org setting: Add time limits for message deleting.
Add realm setting to set time limit for message deleitng.
Set default value of message_content_delete_limit_seconds
to 600 seconds(10 min).

Thanks to Shubham Dhama for rebasing and reworking this.  Some final
edits also done by Tim Abbott.

Fixes #7344.
2018-04-27 19:22:28 -07:00
Vishnu Ks cc93ac34a8 coverage: Add coverage to estimate_recent_messages.
With this message.py is fully covered and can be
removed from not_yet_fully_covered in test-backend.
2018-04-17 11:01:20 -07:00
Ben Reeves fdfbd45208 soft_deactivation: Change `<` to `<=` in add_missing_messages.
We should still short-circuit the iteration in
`add_missing_messages` if the unsubscription was the last
thing to happen to the user before unsubscription and
soft deactivation.
2018-04-16 11:28:08 -07:00
Alyssa Wagenmaker d4e5777296 tests: Test user unsubscribing before soft deactivation.
Brings lib/soft_deactivation.py up to 100% test coverage.

Improves: #7089.
2018-04-16 11:28:08 -07:00
Shubham Dhama b650b6b38c markdown: Add @stream as an alias for @all.
Fixes: #8930.
2018-04-09 16:35:14 -07:00
Tim Abbott 53e47e6991 messages: Modify access_message for is_history_public_to_subscribers.
This completes the Message side of #2745.
2018-04-04 16:18:47 -07:00
Greg Price e792fc6c07 spelling: Correctly write "cannot".
None of these errors were user-facing; mainly in comments, plus
one bit of internal docs and a developer tool.
2018-04-02 15:36:31 -07:00
Sarah 91197fa4f1 org settings: Add logic for applying allow_community_topic_editing.
Applies the logic to allow community members to edit topics
of others' messages if this setting is True. Otherwise,
only administrators can update the topic of others' messages.

This logic includes a 24-hour time limit for community topic editing.
2018-03-22 16:13:36 -07:00
Nikhil Kumar Mishra 1579f8cb4b messages: Add unit test for get_raw_unread_data.
Add test with 2 messages to the same group PM thread.
2018-03-19 10:36:26 -07:00
Tim Abbott ef92fcbe2b topic history: Fix fetching topic history of public streams.
Apparently, we did essentially all the work to support showing full
topic history to newly subscribed users from a data flow perspective,
but didn't actually enable this feature by having the topic history
endpoint grant access to historical topics.  This fixes that gap.

I'm not altogether happy with how the code and tests read for this
feature; the code itself has more duplication than I'd like, and the
tests do too, but it works.
2018-03-11 20:59:20 -07:00
Tim Abbott c47403b024 i18n: Fix message-send error messages using 'realm'.
We also do some small quality improvements to these strings, while
we're at it.
2018-03-07 17:15:29 -08:00
Shubham Dhama dbce49ac53 refactor: Move topic name checks to Addressee class.
In order to get test coverage on topic name checks, we
do them in Addressee, so that we don't hit an assertion
first. The assertion in question is in Addressee.topic(),
and it was added partly to appease mypy.
2018-01-25 06:21:01 -05:00
Vishnu Ks a11b742984 messages: Calculate value of first visible message ID using cron job.
[greg: Fixed buggy time conversion in estimate_recent_messages.]
2018-01-24 17:15:08 -08:00
Vishnu Ks 036dc53d20 messages: Rename last_visible_message_id to first_visible_message_id. 2018-01-22 19:53:44 -08:00
Aditya Bansal f272ea9087 scheduledmessages: Start using/expecting delivery_type as a param. 2018-01-19 11:33:11 -05:00
Aditya Bansal f46d098558 schedulemessages: Add handle_deferred_message() to handle requests.
This is responsible for:
1.) Handling all the incoming requests at the
messages endpoint which have defer param set. This is similar to
send_message_backend apart from the fact that instead of really
sending a message it schedules one to be sent later on.
2.) Does some preliminary checks such as validating timestamp for
scheduling a message, prevent scheduling a message in past, ensure
correct format of message to be scheduled.
3.) Extracts time of scheduled delivery from message.
4.) Add tests for the newly introduced function.
5.) timezone: Add get_timezone() to obtain tz object from string.
This helps in obtaining a timezone (tz) object from a timezone
specified as a string. This string needs to be a pytz lib defined
timezone string which we use to specify local timezones of the
users.
2018-01-19 11:33:11 -05:00
Xavier Cooney 8c5e74afe9 test_messages.py: Remove unecessary imports. 2018-01-16 08:16:43 -05:00
Vishnu Ks e6d3f8895f messages: Create function to calculate first visible message id. 2018-01-04 08:52:40 -05:00
Alena Volkova 26eb23afc7 tests: Replace api_auth usages with auth methods in zerver/tests. 2017-12-18 09:24:09 -05:00
Rhea Parekh 8ed7f73199 IRC mirror: No longer require IRC stream names to start with #. 2017-12-12 10:04:31 -06:00
Greg Price 453585abb1 test_messages: Clean up some error spew by updating a test.
This test would produce a bunch of log messages with tracebacks,
complaining that `welcome-bot@zulip.com` tried to send cross-realm PMs
and can't.  The issue is that the test overrides
`settings.CROSS_REALM_BOT_EMAILS`, and hasn't kept up with additions
to the normal value for that setting.  Update it so that welcome-bot
is permitted as usual.
2017-11-28 16:23:10 -08:00
Vishnu Ks b9bc1c2b33 Eliminate get_user_profile_by_email from test_classes. 2017-11-26 15:47:56 -08:00
rht 0260ba82ca zerver/tests: Use python 3 syntax for typing (part 4). 2017-11-21 22:01:19 -08:00
rht 6c28c7e879 zerver/tests: Use python 3 syntax for typing. 2017-11-18 15:35:57 -08:00
Tim Abbott 1ed50ee858 tests: Pass a realm option to login for non-zulip realms.
This better matches the model of how having multiple realms should
work: you need to specify which realm you're logging into.
2017-11-17 15:32:42 -08:00
Tim Abbott 8dc82f97c7 python: Wrap long def lines in test files.
We don't have our linter checking test files due to ultra-long strings
that are often present in test output that we verify.  But it's worth
at least cleaning out all the ultra-long def lines.
2017-11-16 22:00:53 -08:00
rht 4f5b1c0a5a zerver/tests: Use python 3 syntax for typing in most files. 2017-11-16 21:52:01 -08:00
Steve Howell 50e3285a0d tests: Remove force_text() in test_messages.py. 2017-11-09 10:27:30 -08:00
YJDave de67bf81fd org settings: Allow users to delete their message.
Fixes #7049.
2017-11-09 10:20:34 -08:00
derAnfaenger 19bc55aa45 Fix various typos.
The typos and their corrections were found with the
aid of https://github.com/lucasdemarchi/codespell.
2017-11-09 16:26:38 +01:00
Steve Howell ae0b27a7ed Extract messages_for_ids. 2017-11-07 17:48:27 -08:00
Steve Howell 88e1e284bb Restructure send-message code for gravatars.
This refactoring doesn't change behavior, but it sets us up
to more easily handle a register setting for `client_gravatar`,
which will allow clients to tell us they're going to compute
their own gravatar URLs.

The `client_gravatar` flag already exists in our code, but it
is only used for Django views (users/messages) but not for
Zulip events.

The main change is to move the call to `set_sender_avatar` into
`finalize_payload`, which adds the boolean `client_gravatar`
parameter to that function.  And then we update various callers
to supply that flag.

One small performance benefit of this change is that we now
lazily compute the client message payloads in
`event_queue.process_message_event` now, so this will improve
performance if all interested clients have the same value of
`apply_markdown`.  But the change here is really preparing us
for the additional boolean parameter, which will cause us to
have four variations of the payload.
2017-11-07 10:36:02 -08:00
Steve Howell 9767029211 Allow generic bots to be mentioned.
The original PR to allow generic bots to be mentioned had
some merge issues that we detected about a week after the
fact.  This commit restores the logic from the original PR.

The reason we didn't detect this bug earlier is that the
merge issues didn't break any existing behavior.  Instead,
they made it so that only UserMessage rows got written for
bots, but no events were being set.  The part of the commit
that got lost is restored here, so now events get sent as
well.

Thanks to @derAnfaenger for reporting this and being patient
as we tracked it down.

Fixes #7140
2017-11-03 07:45:18 -07:00
rht dcc831f767 refactor: Replace all __unicode__ method with __str__.
Close #6627.
2017-11-02 11:01:47 -07:00
Harshit Bansal c863bb83a0 html_diff: Migrate to use lxml.
We were using Google's diff-match-patch library to diff HTML. The
problem with that approach is that it is a text differ, not an HTML
differ and so it ends up messing up the HTML tags. `lxml` is a safer
option.

Fixes: #7219.
2017-10-30 14:10:35 -07:00
Steve Howell a28841e8aa Extract get_stream_recipient().
Do you call get_recipient(Recipient.STREAM, stream_id) or
get_recipient(stream_id, Recipient.STREAM)?  I could never
remember, and it was not very type safe, since both parameters
are integers.
2017-10-28 17:57:39 -07:00
Steve Howell d132c30c24 Remove `active` parameter from `do_create_user()`.
Almost all callers to do_create_user were trying to
create active users, except for one test.  The
active=False codepath was kind of broken (things
like sending welcome messages had sort of undefined
behavior there), so instead of trying to maintain it,
we just update the one test (`test_people`) to flip the
`is_active` flag manually.

Fixes #7197
2017-10-28 10:32:01 -07:00
Steve Howell 4b78f69a87 tests: Introduce send_huddle_message().
The tighter interface here makes for shorter lines (or
fewer multi-line calls) and it leads to stricter type
checking.
2017-10-27 11:00:23 -07:00
Steve Howell 9707767b21 tests: Introduce send_personal_message().
The tighter interface prevents the need to specify
Recipient.PERSONAL (which can often be inaccurate in the
huddle case, anyway), and it prevents tests from confusingly
specifying a "subject" field for PMs.
2017-10-27 11:00:23 -07:00
Steve Howell 25a6fbd05d tests: Introduce send_stream_message().
Having send_stream_message() avoids the need to supply
Recipient.STREAM as a parameter, and it also uses the more
modern name of `topic_name` for topics.  Under the hood, it
avoids some annoying steps for re-formatting the recipients,
since we just have a single stream name.
2017-10-27 11:00:23 -07:00
Steve Howell 9576d25f92 Use check_send_stream_message in test_not_too_many_queries.
When possible, we want to use direct APIs for sending
stream messages.

This changes the codepath slightly, by not using
forwarded_user_profile, but it doesn't impact the number
of queries, and it's a simple check.

We also remove a couple "subject" references here.
2017-10-27 11:00:23 -07:00
Steve Howell abf16a96d1 Rename subject -> topic_name in assert_stream_message(). 2017-10-27 11:00:23 -07:00
Steve Howell c4404ed078 tests: Change some test topic names to not use "subject". 2017-10-27 11:00:23 -07:00
Steve Howell a70ede6c75 Allow "default" bots to see mentions on all streams.
This change allows normal bots to get UserMessage rows when
they are mentioned on a stream, even if they are not actually
subscribed to the stream.

Fixes #7140.
2017-10-26 22:16:47 -07:00
Steve Howell 635675fe48 Reduce queries needed for sending messages.
In do_send_messages, we only produce one dictionary for
the event queues, instead of different flavors for text
vs. html.  This prevents two unnecessary queries to the
database.

It also means we only put one dictionary on the "message"
event queue instead of two, albeit a wider one that has
some values that won't be sent to the actual clients.

This wider dictionary from MessageDict.wide_dict is also
used for the `feedback_messages` queue and service bot
queues.  Since the extra fields are possibly useful down
the road, and they'll just be ignored for now, we don't
bother to remove them.  Also, those queue processors won't
have access to `content_type`, which they shouldn't need.

Fixes #6947
2017-10-26 16:35:28 -07:00
Steve Howell df93a99b50 Cache only one row per message.
Before this change, we populated two cache entries for each
message that we sent.  The entries were largely redundant,
with the only difference being whether we sent the content
as raw markdown or as the rendered HTML.

This commit makes it so we only have one cache entry per
message, and it includes both content and rendered_content.

One legacy source on confusion here is that `content`
changes meaning when you're on the front end.  Here is the
situation going forward:

    database:
        content = raw
        rendered_contented = rendered

    cache entry:
        content = raw
        rendered_contented = rendered

    payload for the frontend:
        content = raw (for apply_markdown=False)
        content = rendered (for apply_markdown=True)
2017-10-26 16:35:28 -07:00
Steve Howell 14d2d4e506 Fix bug in flush_user_profile().
Every time we updated a UserProfile object, we were calling
delete_display_recipient_cache(), which churns the cache and
does an extra database hop to find subscriptions.  This was
due to saying `updated_fields` instead of `update_fields`.

This made us prone to cache churn for fields like UserProfile.pointer
that are fairly volatile.

Now we use the helper function changed().  To prevent the
opposite problem, we use all the fields that could invalidate
the cache.
2017-10-25 11:30:56 -07:00