Commit Graph

239 Commits

Author SHA1 Message Date
YJDave 6bef44a9fa org setting: Add time limits for message deleting.
Add realm setting to set time limit for message deleitng.
Set default value of message_content_delete_limit_seconds
to 600 seconds(10 min).

Thanks to Shubham Dhama for rebasing and reworking this.  Some final
edits also done by Tim Abbott.

Fixes #7344.
2018-04-27 19:22:28 -07:00
Tim Abbott 79e8bff8fa views: Change use of typing.Text to str.
This is the first part of a general migration of our typing codebase
to use the simpler `str` for strings.
2018-04-23 18:51:23 -07:00
Tim Abbott ad861c5fae messages: Improve comment on need_messages. 2018-04-06 08:57:46 -07:00
Steve Howell 36844418e9 bug fix: Respect include_history for certain queries.
For certain queries where both include_history and
use_first_unread_anchor are set to True, we were excluding
historical rows.  Now we only use the use_first_unread_anchor
flag to filter rows that we use to find the anchor, without
having it filter the actual search results.

The bug went unreported for a long time, because it only
affected mobile users who had newly subscribed to streams.

Note that we make a small change to the test called
test_use_first_unread_anchor_with_muted_topics, which has
a very scary comment about being "arcane" and "be
absolutely sure you know what you're doing."  I think it's
fine.

Also, the new test code would fail before this fix, so it
should help prevent future regressions.

Fixes #8958
2018-04-05 17:16:41 -04:00
Steve Howell b64117d872 refactor: Build query in find_first_unread_anchor().
This is a bit more than a pure refactor, because we duplicate a
chunk of code to calculate a query inside of
find_first_unread_anchor(), so we're doing a bit more work
than before.

We need this refactoring to start decoupling find_first_unread_anchor
from get_messages_backend for the case where include_history is
True.  This will happen in a subsequent commit.

The only test that changes here is a direct test on
find_first_unread_anchor().  All other tests pass without
modification, and we have decent coverage on get_messages_backend.
2018-04-05 17:16:41 -04:00
Steve Howell 345d44b5f1 Extract get_base_query_for_search(). 2018-04-05 17:16:41 -04:00
Steve Howell 59a9b69c25 Simplify search code for keyword searches.
We use an array now to build up the list of search operands and
then consolidate the special search handling after the loop (which
means setting the flag, putting two more columns in the query, and
using ' '.join to build the string).
2018-04-05 13:27:31 -07:00
Steve Howell d521906fb6 search refactor: Extract add_narrow_conditions().
This code was basically pulled from two near-to-each-other
sections of get_messages_backend, and it does an early
return for narrow-is-None.
2018-04-05 13:27:31 -07:00
Steve Howell 3ac660d972 minor: Show `narrow` for UnicodeDecodeError.
We have a debugging statement for some obscure errors we get
when narrows have search terms.  We now show all the narrow
operators.  This isn't really to improve debugging; it's more
to make it easier in the next commit to extract a function
that would make search_term have to be passed back in a tuple.
But it shouldn't hurt debugging either.
2018-04-05 13:27:31 -07:00
Steve Howell 4cba679d38 Move code for find_first_unread_anchor().
This is a pure refactoring and just pulls the function out
to the top level of the module.  (The prior commit extracted
it inside a larger function to make a nicer diff.)
2018-04-05 09:55:54 -07:00
Steve Howell d8a95c6517 Extract find_first_unread_anchor().
This is a pure refactoring.
2018-04-05 09:55:54 -07:00
Tim Abbott 228f41e916 messages: Pass UserProfile to is_public_stream_by_name and rename.
The new name can_access_stream_history_by_name gets to the point of
what this function actually does.  And passing in a user object lets
us define what this does based on the user subscribed.
2018-04-04 15:13:11 -07:00
Tim Abbott 5e82d750c5 get_messages: Refactor ok_to_include_history to accept a UserProfile.
If we make history accessible to some stream subscribers of private
streams, we'll need the UserProfile to be available here.
2018-04-04 15:06:53 -07:00
Tim Abbott fcd265304c message editing: Fix buggy timestamp check.
This somehow got messed up in a final refactoring before merging
91197fa4f1.
2018-03-22 16:26:18 -07:00
Sarah 91197fa4f1 org settings: Add logic for applying allow_community_topic_editing.
Applies the logic to allow community members to edit topics
of others' messages if this setting is True. Otherwise,
only administrators can update the topic of others' messages.

This logic includes a 24-hour time limit for community topic editing.
2018-03-22 16:13:36 -07:00
Steve Howell a4a8527ec5 search: Return info flags in payload.
We now return these:
    anchor
    found_anchor
    found_oldest
    found_newest

Fixes #8639
2018-03-15 12:36:06 -07:00
Steve Howell c6839e07c0 search: Fix num_after/num_before semantics precisely.
We now post-process query results so that you never get
more than `num_after` rows with id < `anchor`, and likewise
for `num_before`.
2018-03-15 12:36:06 -07:00
Steve Howell 6f3ebf6c4c Add post_process_limited_query(). 2018-03-15 12:36:06 -07:00
Steve Howell bd95b37d67 search: Make `num_after`/`num_after` more consistent.
We now consistently set our query limits so that we get at
least `num_after` rows such that id > anchor.  (Obviously, the
caveat is that if there aren't enough rows that fulfill the
query, we'll return the full set of rows, but that may be less
than `num_after`.)  Likewise for `num_before`.

Before this change, we would sometimes return one too few rows
for narrow queries.

Now, we're still a bit broken, but in a more consistent way.  If
we have a query that does not match the anchor row (which could
be true even for a non-narrow query), but which does match lots
of rows after the anchor, we'll return `num_after + 1` rows
on the right hand side, whether or not the query has narrow
parameters.

The off-by-one semantics here have probably been moot all along,
since our windows are approximate to begin with.  If we set
num_after to 100, its just a rough performance optimization to
begin with, so it doesn't matter whether we return 99 or 101 rows,
as long as we set the anchor correctly on the subsequent query.

We will make the results more rigorous in a follow up commit.
2018-03-15 12:36:06 -07:00
Steve Howell ec5299b1f2 search refactor: Rename query_result to rows.
The name `query_result` seems to suggest something more complex.
2018-03-15 12:36:06 -07:00
Steve Howell 1ff653c986 search refactor: Extract `limit_query_to_range`.
This generic function isolates the before/after logic that really
is independent of Message and doesn't need to clutter up
`get_messages_backend`.  Also, introducing a new namespace
reduces some shadowing/mutation with variables like `query`.

It's a pure code move, with some very minor renaming (e.g.
inner_msg_id_col -> id_col).
2018-03-13 13:51:22 -07:00
Steve Howell 74a4a69333 search refactor: Move code that increments num_after/num_before.
We move the code that increments num_after/num_before to be closer
to where we do all the before/after logic.
2018-03-13 13:51:22 -07:00
Steve Howell 63c21707ee search refactor: Tighten up before/after logic.
If anchor is 0, there is no sense doing a before_query.

Likewise, if anchor is `LARGER_THAN_MAX_MESSAGE_ID`, there is
no sense doing an after_query.

We introduce variables called `need_before_query` and
`need_after_query` to enforce those conditions.

This also adds some comments explaining the fallthrough case
where neither query makes sense.
2018-03-13 13:51:22 -07:00
Steve Howell 129faa2c21 search refactor: Avoid `message_id >= 0` in queries.
We don't need things like `AND message_id >= 0` in our queries.

We can short circuit the syntax when `anchor` or
`first_message_visible` are zero.
2018-03-13 13:51:22 -07:00
Steve Howell e232a0bd57 search refactor: Streamline queries for caught-up users.
If use_first_unread_anchor is set and we don't have any unread
messages, then our anchor is effectively "positive infinity" and
we can streamline queries.

In the past we'd have clauses like `message_id <= 999999999999999`
in the query that were harmless but crufty.
2018-03-13 13:51:22 -07:00
Steve Howell e77350dc8e search refactor: Replace confusing `!= 0` idiom in search code.
We want to say `if num_after > 0` when we expect num_after to be
a positive integer.  We don't want any confusion that we will
execute the blocks for values of -7 or None.
2018-03-13 13:51:22 -07:00
Tim Abbott c47403b024 i18n: Fix message-send error messages using 'realm'.
We also do some small quality improvements to these strings, while
we're at it.
2018-03-07 17:15:29 -08:00
neiljp (Neil Pilgrim) 2176f4789b mypy: Rewrite conditional to clarify message-sending defer_until code.
This actually fixes a small bug.
2018-02-19 09:24:28 -08:00
Shubham Padia a4b686297a api: Return anchor in get_messages when use_first_unread_anchor=True.
This may be helpful for some API clients, since it avoids them needed
to do somewhat messy post-processing on the results (the data was
always available via scanning for the first unread message in the result).

Fixes #6244.
2018-02-16 10:06:20 -08:00
neiljp (Neil Pilgrim) 16b939a964 REQ: Ensure only initial parameter is positional & adjust views/messages.py. 2018-02-13 11:40:52 -08:00
rht fae8d23039 zerver/views: Remove u prefix from strings. 2018-02-05 12:11:33 -08:00
Aditya Bansal f272ea9087 scheduledmessages: Start using/expecting delivery_type as a param. 2018-01-19 11:33:11 -05:00
Aditya Bansal f46d098558 schedulemessages: Add handle_deferred_message() to handle requests.
This is responsible for:
1.) Handling all the incoming requests at the
messages endpoint which have defer param set. This is similar to
send_message_backend apart from the fact that instead of really
sending a message it schedules one to be sent later on.
2.) Does some preliminary checks such as validating timestamp for
scheduling a message, prevent scheduling a message in past, ensure
correct format of message to be scheduled.
3.) Extracts time of scheduled delivery from message.
4.) Add tests for the newly introduced function.
5.) timezone: Add get_timezone() to obtain tz object from string.
This helps in obtaining a timezone (tz) object from a timezone
specified as a string. This string needs to be a pytz lib defined
timezone string which we use to specify local timezones of the
users.
2018-01-19 11:33:11 -05:00
Vishnu Ks 277d463d23 messages: Add option to limit users message history. 2018-01-02 13:45:01 -05:00
Rhea Parekh 8ed7f73199 IRC mirror: No longer require IRC stream names to start with #. 2017-12-12 10:04:31 -06:00
picapi_ e2c14724d9 mypy: Use Python 3 type syntax in zerver/views/messages.py. 2017-12-03 13:48:18 -08:00
rht e538f4dd44 zerver/views: Use Python 3 syntax for typing.
Edited by tabbott to remove state.py and streams.py, because of
problems with the original PR's changes, and wrap some long lines.
2017-11-27 17:10:39 -08:00
Robert Hönig 0e0a8a2b14 queue processor tests: Call consume by default.
This significantly improves the API for queue_json_publish to not be
overly focused on what the behavior of this function should be in our
unit tests.
2017-11-26 11:45:34 -08:00
Sampriti Panda a696141a25 search: Add more extensive logging for Unicode errors in highlight_string.
Tweaked by tabbott to provide finer-grained logging.
2017-11-16 15:08:16 -08:00
rht a93b7c40a7 zerver: Text-wrap long lines exceeding 110. 2017-11-10 16:29:32 -08:00
Steve Howell c38996ddde Remove unused force_text import. 2017-11-09 10:32:14 -08:00
Steve Howell 786e90d774 Removed unused imports of force_bytes. 2017-11-09 10:32:14 -08:00
YJDave de67bf81fd org settings: Allow users to delete their message.
Fixes #7049.
2017-11-09 10:20:34 -08:00
Steve Howell 4108797218 Eliminate the parse_usermessage_flags interface.
This commit puts the guts of parse_usermessage_flags into
UserMessage.flags_list_for_flags, since it was slightly faster
than the old implementation and produced the same results.
(Both algorithms were super fast, actually.)

And then all callers use the model method now.
2017-11-07 17:48:27 -08:00
Steve Howell ae0b27a7ed Extract messages_for_ids. 2017-11-07 17:48:27 -08:00
Steve Howell 70f2cd2860 De-duplicate is_search logic in get_messages_backend().
The logic to set search_fields was essentially the same for both
sides of the include_history conditional.

Now we have just one code block that sets search_fields, and we
can quickly short-circuit the loop when is_search is False.
2017-11-07 17:48:27 -08:00
Steve Howell c62588c359 minor: Use dict comprehension in get_messages_backend(). 2017-11-07 17:48:27 -08:00
rht 80a8d4f9f3 refactor: Remove six.moves.map import. 2017-11-07 10:46:42 -08:00
rht 8990b1046d zerver: Remove inheritance from object. 2017-11-06 08:53:48 -08:00
neiljp (Neil Pilgrim) 186181c2a0 mypy: Specify type parameters in convert_term in narrow_parameter. 2017-11-04 19:47:45 -07:00