Commit Graph

225 Commits

Author SHA1 Message Date
Sarah 91197fa4f1 org settings: Add logic for applying allow_community_topic_editing.
Applies the logic to allow community members to edit topics
of others' messages if this setting is True. Otherwise,
only administrators can update the topic of others' messages.

This logic includes a 24-hour time limit for community topic editing.
2018-03-22 16:13:36 -07:00
Steve Howell a4a8527ec5 search: Return info flags in payload.
We now return these:
    anchor
    found_anchor
    found_oldest
    found_newest

Fixes #8639
2018-03-15 12:36:06 -07:00
Steve Howell c6839e07c0 search: Fix num_after/num_before semantics precisely.
We now post-process query results so that you never get
more than `num_after` rows with id < `anchor`, and likewise
for `num_before`.
2018-03-15 12:36:06 -07:00
Steve Howell 6f3ebf6c4c Add post_process_limited_query(). 2018-03-15 12:36:06 -07:00
Steve Howell bd95b37d67 search: Make `num_after`/`num_after` more consistent.
We now consistently set our query limits so that we get at
least `num_after` rows such that id > anchor.  (Obviously, the
caveat is that if there aren't enough rows that fulfill the
query, we'll return the full set of rows, but that may be less
than `num_after`.)  Likewise for `num_before`.

Before this change, we would sometimes return one too few rows
for narrow queries.

Now, we're still a bit broken, but in a more consistent way.  If
we have a query that does not match the anchor row (which could
be true even for a non-narrow query), but which does match lots
of rows after the anchor, we'll return `num_after + 1` rows
on the right hand side, whether or not the query has narrow
parameters.

The off-by-one semantics here have probably been moot all along,
since our windows are approximate to begin with.  If we set
num_after to 100, its just a rough performance optimization to
begin with, so it doesn't matter whether we return 99 or 101 rows,
as long as we set the anchor correctly on the subsequent query.

We will make the results more rigorous in a follow up commit.
2018-03-15 12:36:06 -07:00
Steve Howell ec5299b1f2 search refactor: Rename query_result to rows.
The name `query_result` seems to suggest something more complex.
2018-03-15 12:36:06 -07:00
Steve Howell 1ff653c986 search refactor: Extract `limit_query_to_range`.
This generic function isolates the before/after logic that really
is independent of Message and doesn't need to clutter up
`get_messages_backend`.  Also, introducing a new namespace
reduces some shadowing/mutation with variables like `query`.

It's a pure code move, with some very minor renaming (e.g.
inner_msg_id_col -> id_col).
2018-03-13 13:51:22 -07:00
Steve Howell 74a4a69333 search refactor: Move code that increments num_after/num_before.
We move the code that increments num_after/num_before to be closer
to where we do all the before/after logic.
2018-03-13 13:51:22 -07:00
Steve Howell 63c21707ee search refactor: Tighten up before/after logic.
If anchor is 0, there is no sense doing a before_query.

Likewise, if anchor is `LARGER_THAN_MAX_MESSAGE_ID`, there is
no sense doing an after_query.

We introduce variables called `need_before_query` and
`need_after_query` to enforce those conditions.

This also adds some comments explaining the fallthrough case
where neither query makes sense.
2018-03-13 13:51:22 -07:00
Steve Howell 129faa2c21 search refactor: Avoid `message_id >= 0` in queries.
We don't need things like `AND message_id >= 0` in our queries.

We can short circuit the syntax when `anchor` or
`first_message_visible` are zero.
2018-03-13 13:51:22 -07:00
Steve Howell e232a0bd57 search refactor: Streamline queries for caught-up users.
If use_first_unread_anchor is set and we don't have any unread
messages, then our anchor is effectively "positive infinity" and
we can streamline queries.

In the past we'd have clauses like `message_id <= 999999999999999`
in the query that were harmless but crufty.
2018-03-13 13:51:22 -07:00
Steve Howell e77350dc8e search refactor: Replace confusing `!= 0` idiom in search code.
We want to say `if num_after > 0` when we expect num_after to be
a positive integer.  We don't want any confusion that we will
execute the blocks for values of -7 or None.
2018-03-13 13:51:22 -07:00
Tim Abbott c47403b024 i18n: Fix message-send error messages using 'realm'.
We also do some small quality improvements to these strings, while
we're at it.
2018-03-07 17:15:29 -08:00
neiljp (Neil Pilgrim) 2176f4789b mypy: Rewrite conditional to clarify message-sending defer_until code.
This actually fixes a small bug.
2018-02-19 09:24:28 -08:00
Shubham Padia a4b686297a api: Return anchor in get_messages when use_first_unread_anchor=True.
This may be helpful for some API clients, since it avoids them needed
to do somewhat messy post-processing on the results (the data was
always available via scanning for the first unread message in the result).

Fixes #6244.
2018-02-16 10:06:20 -08:00
neiljp (Neil Pilgrim) 16b939a964 REQ: Ensure only initial parameter is positional & adjust views/messages.py. 2018-02-13 11:40:52 -08:00
rht fae8d23039 zerver/views: Remove u prefix from strings. 2018-02-05 12:11:33 -08:00
Aditya Bansal f272ea9087 scheduledmessages: Start using/expecting delivery_type as a param. 2018-01-19 11:33:11 -05:00
Aditya Bansal f46d098558 schedulemessages: Add handle_deferred_message() to handle requests.
This is responsible for:
1.) Handling all the incoming requests at the
messages endpoint which have defer param set. This is similar to
send_message_backend apart from the fact that instead of really
sending a message it schedules one to be sent later on.
2.) Does some preliminary checks such as validating timestamp for
scheduling a message, prevent scheduling a message in past, ensure
correct format of message to be scheduled.
3.) Extracts time of scheduled delivery from message.
4.) Add tests for the newly introduced function.
5.) timezone: Add get_timezone() to obtain tz object from string.
This helps in obtaining a timezone (tz) object from a timezone
specified as a string. This string needs to be a pytz lib defined
timezone string which we use to specify local timezones of the
users.
2018-01-19 11:33:11 -05:00
Vishnu Ks 277d463d23 messages: Add option to limit users message history. 2018-01-02 13:45:01 -05:00
Rhea Parekh 8ed7f73199 IRC mirror: No longer require IRC stream names to start with #. 2017-12-12 10:04:31 -06:00
picapi_ e2c14724d9 mypy: Use Python 3 type syntax in zerver/views/messages.py. 2017-12-03 13:48:18 -08:00
rht e538f4dd44 zerver/views: Use Python 3 syntax for typing.
Edited by tabbott to remove state.py and streams.py, because of
problems with the original PR's changes, and wrap some long lines.
2017-11-27 17:10:39 -08:00
Robert Hönig 0e0a8a2b14 queue processor tests: Call consume by default.
This significantly improves the API for queue_json_publish to not be
overly focused on what the behavior of this function should be in our
unit tests.
2017-11-26 11:45:34 -08:00
Sampriti Panda a696141a25 search: Add more extensive logging for Unicode errors in highlight_string.
Tweaked by tabbott to provide finer-grained logging.
2017-11-16 15:08:16 -08:00
rht a93b7c40a7 zerver: Text-wrap long lines exceeding 110. 2017-11-10 16:29:32 -08:00
Steve Howell c38996ddde Remove unused force_text import. 2017-11-09 10:32:14 -08:00
Steve Howell 786e90d774 Removed unused imports of force_bytes. 2017-11-09 10:32:14 -08:00
YJDave de67bf81fd org settings: Allow users to delete their message.
Fixes #7049.
2017-11-09 10:20:34 -08:00
Steve Howell 4108797218 Eliminate the parse_usermessage_flags interface.
This commit puts the guts of parse_usermessage_flags into
UserMessage.flags_list_for_flags, since it was slightly faster
than the old implementation and produced the same results.
(Both algorithms were super fast, actually.)

And then all callers use the model method now.
2017-11-07 17:48:27 -08:00
Steve Howell ae0b27a7ed Extract messages_for_ids. 2017-11-07 17:48:27 -08:00
Steve Howell 70f2cd2860 De-duplicate is_search logic in get_messages_backend().
The logic to set search_fields was essentially the same for both
sides of the include_history conditional.

Now we have just one code block that sets search_fields, and we
can quickly short-circuit the loop when is_search is False.
2017-11-07 17:48:27 -08:00
Steve Howell c62588c359 minor: Use dict comprehension in get_messages_backend(). 2017-11-07 17:48:27 -08:00
rht 80a8d4f9f3 refactor: Remove six.moves.map import. 2017-11-07 10:46:42 -08:00
rht 8990b1046d zerver: Remove inheritance from object. 2017-11-06 08:53:48 -08:00
neiljp (Neil Pilgrim) 186181c2a0 mypy: Specify type parameters in convert_term in narrow_parameter. 2017-11-04 19:47:45 -07:00
Greg Price 68b0a419ec decorator: Cut a bunch of dead imports of two view decorators.
Saw these when grepping for these two decorators; they're actually
more numerous than the surviving use sites are.  Cut out the noise.
2017-11-04 19:27:00 -07:00
Steve Howell 9689168bf6 Simplify highlight_string logic.
This change avoids some complicated byte arithmetic by
extracting some local variables and being more explicit
about using bytes vs. using strings.
2017-10-31 12:45:58 -07:00
Sampriti Panda ee61e1be2c search: Fix misaligned highlights due to unicode characters.
tsearch_extras returns search offsets in bytes but our highlight
function treated them as character offsets.  Added a check to subtract
extra bytes if the tsearch search backend is being used.

Fixes #4084.
Fixes #7021.
2017-10-31 09:06:46 -07:00
Steve Howell 8b012c6210 Extract get_personal_recipient(). 2017-10-28 17:57:39 -07:00
Steve Howell a28841e8aa Extract get_stream_recipient().
Do you call get_recipient(Recipient.STREAM, stream_id) or
get_recipient(stream_id, Recipient.STREAM)?  I could never
remember, and it was not very type safe, since both parameters
are integers.
2017-10-28 17:57:39 -07:00
Tim Abbott 5b5dffebe7 get_messages_backend: Tweak how search fields are added.
This is a no-op, but makes mypy happier, since previously it wanted to
know the type of the {} we were passing to `.update()`.
2017-10-28 09:58:51 -07:00
Steve Howell 655f37a34b Rename subject_name in send_message_backend(). 2017-10-27 10:48:11 -07:00
Steve Howell df93a99b50 Cache only one row per message.
Before this change, we populated two cache entries for each
message that we sent.  The entries were largely redundant,
with the only difference being whether we sent the content
as raw markdown or as the rendered HTML.

This commit makes it so we only have one cache entry per
message, and it includes both content and rendered_content.

One legacy source on confusion here is that `content`
changes meaning when you're on the front end.  Here is the
situation going forward:

    database:
        content = raw
        rendered_contented = rendered

    cache entry:
        content = raw
        rendered_contented = rendered

    payload for the frontend:
        content = raw (for apply_markdown=False)
        content = rendered (for apply_markdown=True)
2017-10-26 16:35:28 -07:00
derAnfaenger 1792dcbd09 tests: Call real consume method of queue processors.
This switches to more real tests for a first batch of
queue_json_publish() calls that don't cause trouble when
used with call_consume_tests=True.
2017-10-26 14:58:03 -07:00
Steve Howell a1d887c950 Use check_bool validator in get_messages_backend().
This is the more modern way in our codebase to validate
bool fields.
2017-10-20 15:51:57 -07:00
Steve Howell 0e106a2488 Add client_gratavar support to GET /messages.
Clients fetching messages can now specify that they are able
to compute their avatar, and if they set client_gratavar to
True in the request (w/our normal encoding scheme), then the
backend will not compute it, and the payload will be smaller.

The fix starts with get_messages_backend.  The flag gets
passed down through these functions:

    * MessageDict.post_process_dicts.
    * MessageDict.set_sender_avatar.

We also fix up the callers for post_process_dicts to explicitly
pass in the client_gravatar path, but for now they all just hard
code the value to False.
2017-10-20 15:49:21 -07:00
Steve Howell d909355dc2 refactor: Move methods from models.py -> lib/messages.py.
Message.get_raw_db_rows is moved to MessageDict, since its
implementation details are highly coupled to other methods
in MessageDict.

And then sew_messages_and_reactions comes along for the
ride.

We eventually want to move Reaction.get_raw_db_rows to there
as well.
2017-10-16 23:37:10 -07:00
Tim Abbott 339e206c90 highlight_html_differences: Improve logging output.
Now at least it will give the message ID, and thus be possible to
debug.
2017-10-11 23:38:29 -07:00
Steve Howell 3910448b1d Extract MessageDict.post_process_dicts().
Introduce MessageDict.post_process_dicts() will allow us
the ability to do the following:

    * use less memory in the cache for repeated data
    * prevent cache invalidation
    * format data according to different client needs

The first use of this function is pretty inconsequential, but
it sets us up for more consequential changes.

In this commit we defer the MessageDict.hydrate_recipient_info
step until after we pull data out of the cache.  This impacts
cache size as follows:

    * streams - negligibly bigger
    * PMs/huddles - slimmer due to not needing to repeat
                    sender data like email/full_name

Again, the main point of this change is to start setting up
the infrastructure to do post-processing.
2017-10-11 11:37:16 -07:00