Commit Graph

146 Commits

Author SHA1 Message Date
Durga Akhil Mundroy 146dfa6f0b org-permissions: Add allow_edit_history organiztion setting.
This new setting controls whether or not users are allowed to see the
edit history in a Zulip organization.  It controls access through 2
key mechanisms:

* For long-ago edited messages, get_messages removes the edit history
  content from messages it sends to clients.

* For newly edited messages, clients are responsible for checking the
  setting and not saving the edit history data.  Since the webapp was
  the only client displaying it before this change, this just required
  some changes in message_events.js.

Significantly modified by tabbott to fix some logic bugs and add a
test.
2017-07-16 10:10:06 -07:00
Steve Howell 475eb21a5e Revert commits related to client_message_id.
I pushed a bunch of commits that attempted to introduce
the concept of `client_message_id` into our server, as
part of cleaning up our codepaths related to messages you
sent (both for the locally echoed case and for the host
case).

When we deployed this, we had some strange failures involving
double-echoed messages and issues advancing the pointer that appeared
related to #5779.  We didn't get to the bottom of exactly why the PR
caused havoc, but I decided there was a cleaner approach, anyway.
2017-07-14 12:13:35 -07:00
Steve Howell 8fbb55df85 Introduce client_message_id on the server.
We are deprecating local_id/local_message_id on the Python server.
Instead of the server knowing about the client's implementation of
local id, with the message id = 9999.01 scheme, we just send the
server an opaque id to send back to us.

This commit changes the name from local_id -> client_message_id,
but it doesn't change the actual values passed yet.

The goal for client_key in future commits will be to:
    * Have it for all messages, not just locally rendered messages
    * Not have it overlap with server-side message ids.

The history behind local_id having numbers like 9999.01 is that
they are actually interim message ids and the numerical value is
used for rendering the message list when we do client-side rendering.
2017-07-13 23:42:27 -04:00
Greg Price 7b526b3946 narrow: Add comments explaining how we limit to legitimately-visible results.
Two wrinkles here:

* It's actually a little subtle why `ok_to_include_history` is
  correct; in particular, it's not true that a term `stream:foo` will
  always limit the query to the stream `foo`.  For this, add an
  explanatory comment backed up by an assert.

* The TODO comment in `messages_in_narrow_backend` about assuming this
  is a search, I'm pretty sure doesn't matter; it seems to only be
  saying that we return the set of fields we would for a search.
  They're harmless to send, and in any case it doesn't appear to be
  true anymore that the client only calls this for a search: the
  `can_apply_locally` function also causes narrows with `has:` to go
  to the server.  So just take that comment out.
2017-06-30 11:47:40 -04:00
Greg Price 7785a1046b narrow: Describe the key security invariant of NarrowBuilder.
(After writing the term "invariant" a few times in these comments and
now this commit message, my inner mathematician reminds me that this
property is properly termed a "monovariant" -- something does change,
but it changes only in one direction.  Pretty sure saying "invariant"
communicates better here, though.)
2017-06-30 11:47:40 -04:00
Cory Lynch 00ebb4f4d4 messages.py: Ignore own email in group PMs. 2017-06-20 21:22:45 -04:00
Cory Lynch b4419528f7 Update narrow.py, messages.py to support is:unread. 2017-06-19 06:51:13 -04:00
K.Kanakhin 2434f2d96c messages: Add support for admins deleting messages.
This makes it possible for Zulip administrators to delete messages.
This is primarily intended for use in deleting early test messages,
but it can solve other problems as well.

Later we'll want to play with the permissions model for this, but for
now, the goal is just to integrate the feature.

Note that it saves the deleted messages for some time using the same
approach as Zulip's message retention policy feature.

Fixes #135.
2017-05-29 21:59:38 -07:00
Tim Abbott 796cf8e5fd mypy: Fix a buggy annotation in create_mirrored_message_users. 2017-05-23 15:45:56 -07:00
Yago González 83f3959906 api: Remove unnecessary period for consistency. 2017-05-23 15:01:26 -07:00
Aditya Bansal 634c08c3dd pep8: Add compliance with rule E261 to zerver/views/messages.py. 2017-05-18 03:00:32 +05:30
sinwar 5c3cb79747 streams: Ban null character from stream name.
This is a better solution to the problem of how _pg_re_escape should
handle the null character.  There's really no good reason to have a
null character in a stream name.
2017-05-01 23:43:01 -07:00
Jacob Hurwitz 8343d80873 Fix mobile home view returning messages older than the pointer.
In cases where old unread messages in the home view might have been
leaked (either due to bugs or unusual muting interactions), it's
theoretically possible for the first unread message in the home view
to be far older than the pointer.

Since the Zulip mobile app is loading messages following the
use_first_unread logic, we need to plug this gap.

Probably a longer-term solution will involve changing how
update_message_flags works to automatically advance the pointer, but
this change should make it possible for the mobile apps to
consistently use the `use_first_unread` mechanism for fetching the
latest home view messages.

With tweaks to the tests by tabbott.

Fixes zulip/zulip-mobile#422.
2017-04-18 21:39:24 -07:00
Abhijeet Kaur 5e55fe992d backend: Add ability to search by group private message thread.
This doesn't yet contain the frontend or documentation for this
feature.

Modified by tabbott to rename the parameter and line-wrap the query
code.
2017-04-18 15:50:27 -07:00
Kouhei Sutou 2f09866364 message: Support highlight in link tag.
textsearch based full text search doesn't match text in link tag but
PGroonga based full text search can match text in link tag.

Without this change, highlighting text in link tag generates broken
HTML.
2017-04-18 13:15:48 -07:00
hackerkid b2504084ab Replace timezone.now with timezone_now. 2017-04-16 12:28:56 -07:00
Harshit Bansal ac2172e233 models: Rename RealmAlias model to RealmDomain.
Includes a migration.
2017-04-04 15:48:03 -07:00
Kouhei Sutou 7c43aa1372 message: Use pgroonga.match_positions_character.
We can remove byte version text highlight method with this change.

pgroonga.match_positions_character was added in PGroonga 1.1.1:
http://pgroonga.github.io/reference/functions/pgroonga-match-positions-character.html

PGroonga 1.1.1 was released at 2016-08-31. So we can use it.
2017-04-04 14:05:45 -07:00
Tim Abbott 3617ebfd41 api: Rename get_old_messages to get_messages in the backend.
Fixes #1315.
2017-03-23 23:52:44 -07:00
Akash Kothawale 4c2bfae83e get_stream: Throw DoesNotExist if stream is not found.
This makes get_stream match get_realm, get_user_profile_by_email,
etc., in interface, and is more convenient for mypy annotations
because `get_stream` now doesn't return an Optional[Stream].
2017-03-23 15:42:00 -07:00
Ayush Jain bddcfb1c96 Add realm-level settings to control inline image and url preview.
This gives users more control in case they don't want previews,
especially for the "previews of linked websites" feature.

Fixes: #2640.
2017-03-21 15:46:17 -07:00
Tim Abbott aa7e4fec35 messages: Require join with zerver_message if use_first_unread_anchor.
This fixes a 500 bug where the RN mobile app would simply fail to load
old messages if use_first_unread_anchor was not set.
2017-03-21 00:02:46 -07:00
Jacob Hurwitz 0a76a609ec Fix bugs with handling of the empty narrow.
An empty narrow (ie, the home view) can be represented in code as either
`None` or `[]` but we had incorrect handling that failed to fully
properly deal with either case.

(1) In `get_stream_name_from_narrow`, we failed to deal with `None` by
trying to always iterate over `narrow`.
(2) In several other places, we failed to deal with `[]` by explicitly
checking `if narrow is None` or `if narrow is not None`. Changing these
to truthiness checks should work for both the `None` and `[]` cases.
2017-03-19 11:47:03 -07:00
Jacob Hurwitz 8ab88f5aad Remove hack for old mobile clients
This FIXME was added in 50d229fe11.
Considering it's been more than 4 years, we can probably safely remove
it now.
2017-03-18 18:11:17 -07:00
Raghav Jajodia a3a03bd6a5 mypy: Added Dict, List and Set imports.
Fixed mypy errors associated with the upgrade.
2017-03-04 14:33:44 -08:00
Tim Abbott d90f7c72a3 zephyr: Fix broken postgres regular expression logic and add tests.
Like many rare-case code with new tests, it turns out that the logic
for handling null characters in our Zephyr postgres query escaping
never worked, in multiple ways.  First, it always changed the second
character in s, not the current one being inspected, and second, the
value it replaced it with was no the correct postgres escape of the
null byte.  We fix this and add tests.

This completes the effort to get zerver/views/messages.py to 100%
test coverage.

Fixes #1006.
2017-03-01 10:38:48 -08:00
Tim Abbott bc38870136 preview: Fix adding links in message editing.
When you edit a message to contain links, and URL previews are
enabled, previously we'd throw an exception, because the realm ID
wasn't included in the event.

Also adds a test so that we can have effective test coverage on this
codepath, though this history is actually that I found the bug through
writing this test :).
2017-03-01 10:38:47 -08:00
Rishi Gupta 15d60fa7ed Change now() to timezone.now() throughout codebase.
Change `from django.utils.timezone import now` to
`from django.utils import timezone`.

This is both because now() is ambiguous (could be datetime.datetime.now),
and more importantly to make it easier to write a lint rule against
datetime.datetime.now().
2017-02-28 16:03:37 -08:00
Tim Abbott 8dba310bee messages: Remove some unnecessary zephyr code paths.
The comments explain why this change is correct.  This change is
useful because it's better to not have dead code paths, both because
it makes our life easier for coverage analysis, and because the else
statement provided the illusion that it could actually happen.

If the analysis in that comment is wrong, we'd rather have a 500 error
so we fix the bug than things silently sorta working.
2017-02-22 20:51:25 -08:00
Tim Abbott 948e1bbd0a update_message_backend: Use access_message.
This continues our campaign of removing direct queries on the Message
and UserMessage tables that could not follow our security policy.
2017-02-22 20:51:25 -08:00
Tim Abbott 442066da12 messages: Remove unused json_update_message. 2017-02-22 20:51:25 -08:00
Tim Abbott 64434e04a1 messages: Fix empty condition for muted streams.
This fixes a sqlalchemy warning (that caused unnecessary complexity in
this query).
2017-02-22 20:51:23 -08:00
Tim Abbott a079bcdce1 get_old_messages: Add LARGER_THAN_MAX_MESSAGE_ID constant. 2017-02-22 20:50:15 -08:00
Tim Abbott ff65b6b842 get_old_messages: Remove unnecessary >= max_message_id query. 2017-02-22 20:47:48 -08:00
Tim Abbott 6f0410774c sqlalchemy: Fix most sqlalchemy deprecation warnings.
Fixes #2732.
2017-02-22 20:47:48 -08:00
Tim Abbott 66f016edbb zephyr: Fix zerver_message tables scans due to regex in queries.
This arguably regresses the Zephyr experience, in that we no longer
consider 'foo.d.d.d.d.d' to be something that gets narrowed in with
the rest, but that's a pretty rare use case anyway.

In practice, using that many '.d's anyway only happens a few times a
year.
2017-02-22 20:47:46 -08:00
Tim Abbott 98894eb759 edit_history: Reverse the order of edit history list. 2017-02-19 16:47:07 -08:00
Tim Abbott eadd6fb3c6 message_edit: Include an initial entry in view output.
This makes it super easy for frontend code using this view code to
produce a nice display of the history.

This also fixes an off-by-one error with the timestamps.
2017-02-19 16:46:47 -08:00
Tim Abbott 515340ed00 Add message edit history backend with tests.
Based on work by Kartik Maji in #1204.

This has a few significant changes from the original version:
* We correctly handle filling in data for topic edits
* Has a complete test suite verifying correctness of the logic
* Currently, it doesn't include a special "start" entry

Things we may want to further change include:
* Adding a special "start" entry.
* Reversing the order of the history data returned for clarity.
2017-02-19 16:13:35 -08:00
Tim Abbott eeca69cb4b mypy: Clean up more optional types. 2017-02-10 23:53:44 -08:00
Harshit Bansal 7d10cbc32b Add RealmAlias.allow_subdomains to model, frontend, and API.
Includes a database migration.

Fixes #1868.
2017-02-08 22:03:27 -08:00
Tim Abbott 22d1aa396b lint: Clean up W503 PEP-8 warning. 2017-01-23 20:50:04 -08:00
Tim Abbott 46ab7762e7 edit: Eliminate unnecessary json_update_message wrapper. 2017-01-23 17:22:42 -08:00
Tim Abbott 33b02a02dd update_message_backend: Fix memcached queries in loop.
This fixes a bug where update_message_backend would do one memcached
query per user receiving a given message.  Right now we just do a
single bulk database query, but in principle we could use
generic_bulk_cached_fetch to use the cache as well.
2017-01-23 17:22:42 -08:00
Tim Abbott 0f7f9dc0fb update_message_backend: Log number of changed messages.
This will be helpful for debugging potential performance problems.
2017-01-23 17:22:40 -08:00
Sampriti Panda 34a4a1378d bugdown: Use specified realm, not sender realm, for rendering.
This changes bugdown to use the realm passed in by the caller (if any)
for rendering, fixing a problem where bots such as the notification
bot would have their messages rendering using the admin realm's
settings, not the settings of the realm their messages are being sent
into.

Also adds a test for the notification bot case.

Fixes #3215.
2017-01-21 21:37:57 -08:00
Tim Abbott 01e0acb282 render_incoming_message: Require a realm object. 2017-01-21 21:37:57 -08:00
Bojidar Marinov df00ad3e84 search: Prevent crashes with multiple search operators.
Fixes #3071.
2017-01-16 19:42:49 -08:00
Rishi Gupta 2b0a7fd0ba Rename models.get_realm_by_string_id to get_realm.
Finishes the refactoring started in c1bbd8d. The goal of the refactoring is
to change the argument to get_realm from a Realm.domain to a
Realm.string_id. The steps were

* Add a new function, get_realm_by_string_id.

* Change all calls to get_realm to use get_realm_by_string_id instead.

* Remove get_realm.

* (This commit) Rename get_realm_by_string_id to get_realm.

Part of a larger migration to remove the Realm.domain field entirely.
2017-01-04 17:12:23 -08:00
Rishi Gupta cf762eaf84 Change X.realm.id to X.realm_id across codebase.
This makes it more clearly the pattern in the Zulip codebase, and thus
decreases the risk of accidentally doing database queries.
2017-01-03 16:46:26 -08:00