Commit Graph

118 Commits

Author SHA1 Message Date
Tim Abbott 8dba310bee messages: Remove some unnecessary zephyr code paths.
The comments explain why this change is correct.  This change is
useful because it's better to not have dead code paths, both because
it makes our life easier for coverage analysis, and because the else
statement provided the illusion that it could actually happen.

If the analysis in that comment is wrong, we'd rather have a 500 error
so we fix the bug than things silently sorta working.
2017-02-22 20:51:25 -08:00
Tim Abbott 948e1bbd0a update_message_backend: Use access_message.
This continues our campaign of removing direct queries on the Message
and UserMessage tables that could not follow our security policy.
2017-02-22 20:51:25 -08:00
Tim Abbott 442066da12 messages: Remove unused json_update_message. 2017-02-22 20:51:25 -08:00
Tim Abbott 64434e04a1 messages: Fix empty condition for muted streams.
This fixes a sqlalchemy warning (that caused unnecessary complexity in
this query).
2017-02-22 20:51:23 -08:00
Tim Abbott a079bcdce1 get_old_messages: Add LARGER_THAN_MAX_MESSAGE_ID constant. 2017-02-22 20:50:15 -08:00
Tim Abbott ff65b6b842 get_old_messages: Remove unnecessary >= max_message_id query. 2017-02-22 20:47:48 -08:00
Tim Abbott 6f0410774c sqlalchemy: Fix most sqlalchemy deprecation warnings.
Fixes #2732.
2017-02-22 20:47:48 -08:00
Tim Abbott 66f016edbb zephyr: Fix zerver_message tables scans due to regex in queries.
This arguably regresses the Zephyr experience, in that we no longer
consider 'foo.d.d.d.d.d' to be something that gets narrowed in with
the rest, but that's a pretty rare use case anyway.

In practice, using that many '.d's anyway only happens a few times a
year.
2017-02-22 20:47:46 -08:00
Tim Abbott 98894eb759 edit_history: Reverse the order of edit history list. 2017-02-19 16:47:07 -08:00
Tim Abbott eadd6fb3c6 message_edit: Include an initial entry in view output.
This makes it super easy for frontend code using this view code to
produce a nice display of the history.

This also fixes an off-by-one error with the timestamps.
2017-02-19 16:46:47 -08:00
Tim Abbott 515340ed00 Add message edit history backend with tests.
Based on work by Kartik Maji in #1204.

This has a few significant changes from the original version:
* We correctly handle filling in data for topic edits
* Has a complete test suite verifying correctness of the logic
* Currently, it doesn't include a special "start" entry

Things we may want to further change include:
* Adding a special "start" entry.
* Reversing the order of the history data returned for clarity.
2017-02-19 16:13:35 -08:00
Tim Abbott eeca69cb4b mypy: Clean up more optional types. 2017-02-10 23:53:44 -08:00
Harshit Bansal 7d10cbc32b Add RealmAlias.allow_subdomains to model, frontend, and API.
Includes a database migration.

Fixes #1868.
2017-02-08 22:03:27 -08:00
Tim Abbott 22d1aa396b lint: Clean up W503 PEP-8 warning. 2017-01-23 20:50:04 -08:00
Tim Abbott 46ab7762e7 edit: Eliminate unnecessary json_update_message wrapper. 2017-01-23 17:22:42 -08:00
Tim Abbott 33b02a02dd update_message_backend: Fix memcached queries in loop.
This fixes a bug where update_message_backend would do one memcached
query per user receiving a given message.  Right now we just do a
single bulk database query, but in principle we could use
generic_bulk_cached_fetch to use the cache as well.
2017-01-23 17:22:42 -08:00
Tim Abbott 0f7f9dc0fb update_message_backend: Log number of changed messages.
This will be helpful for debugging potential performance problems.
2017-01-23 17:22:40 -08:00
Sampriti Panda 34a4a1378d bugdown: Use specified realm, not sender realm, for rendering.
This changes bugdown to use the realm passed in by the caller (if any)
for rendering, fixing a problem where bots such as the notification
bot would have their messages rendering using the admin realm's
settings, not the settings of the realm their messages are being sent
into.

Also adds a test for the notification bot case.

Fixes #3215.
2017-01-21 21:37:57 -08:00
Tim Abbott 01e0acb282 render_incoming_message: Require a realm object. 2017-01-21 21:37:57 -08:00
Bojidar Marinov df00ad3e84 search: Prevent crashes with multiple search operators.
Fixes #3071.
2017-01-16 19:42:49 -08:00
Rishi Gupta 2b0a7fd0ba Rename models.get_realm_by_string_id to get_realm.
Finishes the refactoring started in c1bbd8d. The goal of the refactoring is
to change the argument to get_realm from a Realm.domain to a
Realm.string_id. The steps were

* Add a new function, get_realm_by_string_id.

* Change all calls to get_realm to use get_realm_by_string_id instead.

* Remove get_realm.

* (This commit) Rename get_realm_by_string_id to get_realm.

Part of a larger migration to remove the Realm.domain field entirely.
2017-01-04 17:12:23 -08:00
Rishi Gupta cf762eaf84 Change X.realm.id to X.realm_id across codebase.
This makes it more clearly the pattern in the Zulip codebase, and thus
decreases the risk of accidentally doing database queries.
2017-01-03 16:46:26 -08:00
Rishi Gupta 130f981901 messages.send_message_backend: Take a realm_str instead of a domain. 2017-01-03 16:46:15 -08:00
Rishi Gupta b206d6f251 message.py: Change domain to realm_id in render_markdown args. 2017-01-03 16:46:14 -08:00
Juan Verhook 535ce90272 mypy: Convert zerver/views to use typing.Text. 2016-12-26 13:43:09 -08:00
Juanvulcano 2fd1316d70 Removed json/update_message
Update test_alert_words.py

Update test_messages.py

Update messages.py
2016-12-24 10:08:22 -08:00
Rafid Aslam bfe84a10f0 API: Migrate POST `json/fetch_raw_message` to GET `json/messages/<id>` 2016-12-21 13:08:40 -08:00
Igor Tokarev c93f1d4eda Add oembed/Open Graph/Meta tags data retrieval from inline links.
This change adds support for displaying inline open graph previews for
links posted into Zulip.

It is designed to interact correctly with message editing.

This adds the new settings.INLINE_URL_EMBED_PREVIEW setting to control
whether this feature is enabled.

By default, this setting is currently disabled, so that we can burn it
in for a bit before it impacts users more broadly.

Eventually, we may want to make this manageable via a (set of?)
per-realm settings.  E.g. I can imagine a realm wanting to be able to
enable/disable it for certain URLs.
2016-12-07 17:40:18 -08:00
nikolay abc2ff4a06 pep8: Fix many rule E128 violations.
[Tweaked by tabbott to adjust some approaches used in wrapping]
2016-12-03 13:33:31 -08:00
bulat22101 a6f91064a2 pep8: Fix E129 violations 2016-12-03 10:56:36 -08:00
Rishi Gupta 5681ee90fa models.py: Rename split_email_to_domain to email_to_domain.
In order to better match models.email_to_username.
2016-11-11 15:26:51 -08:00
Rishi Gupta c1713c9659 Prevent code from using email domain to determine realm when subdomains.
Also removes the intermediate step of going through Realm.domain in the
non-subdomains case. Part of a larger project to remove Realm.domain
entirely.
2016-11-11 15:26:51 -08:00
Tim Abbott 48f671a314 same_realm_jabber_user: Remove old ist.mit.edu hack.
This is no longer in use.
2016-11-10 20:50:40 -08:00
Rishi Gupta 3c75ef2d79 Allow users to submit empty content when editing messages.
The closest we have to deleting a message. Was original behavior, but was
changed in 09754c9.
2016-11-10 16:45:18 -08:00
Mohsen Ibrahim e0351948ba render_message_backend: Pass realm data into bugdown.
Previously, the way that render_messages was calling bugdown meant
that the preview feature didn't have access to realm data like the
list of users or streams, resulting in previews for those elements
being wrong.

Now render_message_backend uses zerver.lib.render_markdown to render
messages correctly.

[Commit message tweaked and test added by tabbott]
2016-11-08 13:43:03 -08:00
Tim Abbott f1a399a4e1 message: Create new access_message library function.
With reactions and other upcoming features, we'll be adding several
places where we need to check whether a particular user can access a
particular message.  It's best to just have a single helper function
for this purpose that we can use everywhere.
2016-10-11 17:17:19 -07:00
Tim Abbott d2e41ff48e json_fetch_raw_message: Allow fetching messages you received.
This also modifies the tests to verify that the user in question has
access to the relevant message.
2016-10-11 17:17:19 -07:00
Steve Howell 583a6bbadd Extract zerver/lib/message.py.
This pulls message-related code from models.py into a new
module called message.py, and it starts to break some bugdown
dependencies.  All the methods here are basically related to
serializing Message objects as dictionaries for caches and
events.

    extract_message_dict
    stringify_message_dict
    message_to_dict
    message_to_dict_json
    MessageDict.to_dict_uncached
    MessageDict.to_dict_uncached_helper
    MessageDict.build_dict_from_raw_db_row
    MessageDict.build_message_dict

This fix also removes a circular dependency related
to get_avatar_url.

Also, there was kind of a latent bug in Message.need_to_render_content
where it was depending on other calls to Message to import bugdown
and set it globally in the namespace.  We really need to just
eliminate the function, since it's so small and used by code that
may be doing very sketchy things, but for now I just fix it.  (The
bug would possibly be exposed by moving build_message_dict out to the
library.)
2016-10-04 11:31:20 -07:00
Steve Howell ac994fdd51 Move three functions from models.py to lib/cache.py.
I move these three functions to lib/cache.py:

    to_dict_cache_key_id
    to_dict_cache_key
    flush_message

This will prepare us for a more significant refactoring that
eventually breaks down some circular dependencies with
Message and bugdown.
2016-10-04 11:31:20 -07:00
Tim Abbott 03bf8893e2 Remove legacy /api/v1/send_message endpoint.
This was the original way to send messages via the Zulip API in the
very early days of Zulip, but was replaced by the REST API back in
2013.

Fixes: #730.
2016-09-27 14:51:54 -07:00
Steve Howell a04a095738 Speed up alert word detection during message sends.
We no longer use all the alert words for all the users in the
entire realm when we look for alert words in a newly sent/edited
message.  Now we limit the search to only all the alert words
for all the users who will get UserMessage records.  This will
hopefully make a big difference for big realms where most messages
are only sent to a small subset of users.
2016-09-23 15:24:55 -07:00
Steve Howell cb0d75b23b rendering: Consolidate code to render new messages.
We now use render_incoming_message() to render all incoming
new messages (sends/edits), so that they will get the same treatment.

This change also establishes do_send_messages() as the code
path to get new messages rendered.  It removes some
logic from check_message() that only happened on certain code paths
for sending messages, and which would only detect failures by
expensively rendering messages, so it wasn't much of a guard.

This change also helps to phase out maybe_render_content(), which
deepens the call stack without providing much clarity to the reader,
since it's behavior is so variable.

Finally, this sets up to fix a flaw in the way we compute which
users have alert words in their messages (in a subsequent commit).
2016-09-23 15:21:17 -07:00
Tim Abbott e580ed579b Finish annotating zerver/views/messages.py.
These annotations aren't perfect because the sqlalchemy stubs in
typeshed are broken (e.g. a `Select` doesn't have the ability to do
`.where()`, but we've at least used some typevars to make it easy to
address that when the sqlalchemy stubs are less broken).
2016-09-12 08:47:52 -07:00
Kouhei Sutou 683f49aa99 Support full text search for all languages using pgroonga.
This adds support for using PGroonga to back the Zulip full-text
search feature.  Because built-in PostgreSQL full text search doesn't
support languages that don't put space between terms such as Japanese,
Chinese and so on. PGroonga supports all languages including Japanese
and Chinese.

Developers will need to re-provision when rebasing past this patch for
the tests to pass, since provision is what installs the PGroonga
package and extension.

PGroonga is enabled by default in development but not in production;
the hope is that after the PGroonga support is tested further, we can
enable it by default.

Fixes #615.

[docs and tests tweaked by tabbott]
2016-08-26 21:04:03 -07:00
Tim Abbott a90b470205 Fix regression in search interaction with wide unicode characters.
Apparently, we had incorrectly concluded that our highlight_string
search result highlighting offsets coming from tsearch_extras were
measured in bytes, whereas in fact it is measured in characters.
2016-08-25 09:04:29 -07:00
Tim Abbott 2da6b6bfd8 views.messages: Add some more annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott 375551aaa6 Clean up most hardcoding of mit.edu domain checks.
This moves all this code to be gated on a few virtual realm settings.
2016-07-26 20:30:12 -07:00
Steve Howell 3fa745ebe3 Remove untested code in is_public_stream().
We were calling valid_stream_name(), but more appropriate
validation checks for stream names happen elsewhere in the codepath.
2016-07-26 18:10:00 -07:00
Tim Abbott afaac85dc6 Move get_sqlalchemy_connection to its own file. 2016-07-19 15:28:41 -07:00
Taranjeet Singh 3031214718 zerver/views/messages.py: Fix line with length greater than 120. 2016-07-18 15:05:52 -07:00