Commit Graph

7605 Commits

Author SHA1 Message Date
Tim Abbott e6833b6427 cleanup: Remove the legacy Dropbox file upload integration.
This has been hard-disabled for years, we have no plans to re-enable
it, and it has some hacky code in it.
2018-04-11 11:39:48 -07:00
Aditya Bansal 6c1a50da76 csp_reports: Add endpoint to handle logging of reports sent by clients. 2018-04-11 23:01:13 +05:30
Vishwesh Jainkuniya 383c62fb03 dev_login: Identify each user's realm when listing them.
This is a mobile-specific endpoint used for logging into a dev server.
On mobile without this realm_uri it's impossible to send a login request
to the corresponding realm on the dev server and proceed further; we can
only guess, which doesn't work for using multiple realms.

Also rename the endpoint to reflect the additional data.

Testing Plan:
Sent a request to the endpoint, and inspected the result.

[greg: renamed function to match, squashed renames with data change,
 and adjusted commit message.]
2018-04-10 17:03:36 -07:00
Rhea Parekh f7398cbb09 slack import: Implement custom profile fields.
Add custom profile fields in the slack converted
data 'realm' file.
Added tests for the custom profile fields.

Fixes #8928
2018-04-10 13:28:53 -07:00
Rhea Parekh 852e8516b4 slack import: Add custom profile fields.
Build CustomProfileField and CustomProfileFieldValue
for every user and process the field type after getting an
entire list of the custom fields.
2018-04-10 13:28:53 -07:00
Rhea Parekh ccefaf7b26 scripts: Remove the depreciated script 'postgres-reset-sequences'. 2018-04-10 13:07:14 -07:00
rht a183186672 slack importer: User session.get to recycle previous connections. 2018-04-09 22:02:01 -07:00
Shubham Dhama b650b6b38c markdown: Add @stream as an alias for @all.
Fixes: #8930.
2018-04-09 16:35:14 -07:00
rht 7a8655cc50 Slack importer: Add test for Slack channel mention to Zulip stream mention. 2018-04-09 10:47:39 -07:00
rht 630adb406b Slack importer: Map Slack channel mentions to Zulip stream mentions. 2018-04-09 10:47:39 -07:00
Rhea Parekh 035c440ff3 import script: Support import custom profile fields.
Import of Custom profile fields is only supported for slack
import script for now.
2018-04-09 10:45:35 -07:00
Tim Abbott c41d7ee300 slack import: Write reasonable multi-line JSON.
This is a lot better for debugging.
2018-04-09 10:45:35 -07:00
Rhea Parekh ed7127c8b4 import script: Delete medium sized avatars if it exists.
Deletion of medium sized image is done if it exists before calling the
function 'ensure_medium_avatar_image', to avoid potentially confusing
problems with left-over medium-size avatar images from a previous run
being used when repeatedly importing the same realm in a development
environment..

Fixes #8949.
2018-04-08 07:04:24 -07:00
rht f6feac1316 Slack importer: Map Slack command for mentions to Zulip's all.
Fixes #9003.
2018-04-07 20:02:39 -07:00
Rhea Parekh e037c2f93e import script: Fix upload links.
Rendered content is None for Slack imports, hence it is replaced only
for Zulip->Zulip imports.

Fixes #8959.
2018-04-07 20:01:20 -07:00
Rhea Parekh b3f951d2cf import script: User profile ids should be allocated before allocating bot ids. 2018-04-07 13:28:33 +05:30
Vishnu Ks e92838a31f registration: Catch email validation error and show error message. 2018-04-06 15:18:32 -07:00
Shubham Dhama 0e6757af5c org settings: Change default realm description to empty string.
This fixes a traceback that users would get when editing the realm
description just after creating a new organization.
2018-04-06 15:15:47 -07:00
Rhea Parekh 2baa9bc16e Import: Add subdomain in the import script.
Also remove user input of subdomain in the slack data
conversion script.
2018-04-06 09:12:56 -07:00
Tim Abbott ad861c5fae messages: Improve comment on need_messages. 2018-04-06 08:57:46 -07:00
Eeshan Garg 42bbfea775 webhooks/splunk: Update docs to conform to style guide. 2018-04-05 23:28:27 -07:00
Eeshan Garg 7b1ce446cf webhook/opsgenie: Update docs to conform to style guide. 2018-04-05 22:47:22 -07:00
Eeshan Garg 2e700477e3 webhooks/groove: Update docs to conform to style guide. 2018-04-05 22:47:21 -07:00
Eeshan Garg 902ab01785 webhooks/trello: Update docs to conform to style guide.
This one is one of the most tedious to set up and get working.

We now also rely on the Trello scripts available as part of the
`python-zulip-api/zulip` API package to make the setup process
easier.
2018-04-05 14:48:37 -07:00
Tim Abbott b0b134cb4c help: Clean up settings_html and subscriptions_html.
After some thinking, I don't think there's any actual value to doing
the ../ style relative links here, whereas there is actual harm from
the links being slightly broken in the current model.  We fix this by
just using /#settings as the URL.

Fixes #8978.
2018-04-05 14:48:26 -07:00
Steve Howell 36844418e9 bug fix: Respect include_history for certain queries.
For certain queries where both include_history and
use_first_unread_anchor are set to True, we were excluding
historical rows.  Now we only use the use_first_unread_anchor
flag to filter rows that we use to find the anchor, without
having it filter the actual search results.

The bug went unreported for a long time, because it only
affected mobile users who had newly subscribed to streams.

Note that we make a small change to the test called
test_use_first_unread_anchor_with_muted_topics, which has
a very scary comment about being "arcane" and "be
absolutely sure you know what you're doing."  I think it's
fine.

Also, the new test code would fail before this fix, so it
should help prevent future regressions.

Fixes #8958
2018-04-05 17:16:41 -04:00
Steve Howell b64117d872 refactor: Build query in find_first_unread_anchor().
This is a bit more than a pure refactor, because we duplicate a
chunk of code to calculate a query inside of
find_first_unread_anchor(), so we're doing a bit more work
than before.

We need this refactoring to start decoupling find_first_unread_anchor
from get_messages_backend for the case where include_history is
True.  This will happen in a subsequent commit.

The only test that changes here is a direct test on
find_first_unread_anchor().  All other tests pass without
modification, and we have decent coverage on get_messages_backend.
2018-04-05 17:16:41 -04:00
Steve Howell 345d44b5f1 Extract get_base_query_for_search(). 2018-04-05 17:16:41 -04:00
Steve Howell 59a9b69c25 Simplify search code for keyword searches.
We use an array now to build up the list of search operands and
then consolidate the special search handling after the loop (which
means setting the flag, putting two more columns in the query, and
using ' '.join to build the string).
2018-04-05 13:27:31 -07:00
Steve Howell d521906fb6 search refactor: Extract add_narrow_conditions().
This code was basically pulled from two near-to-each-other
sections of get_messages_backend, and it does an early
return for narrow-is-None.
2018-04-05 13:27:31 -07:00
Steve Howell 3ac660d972 minor: Show `narrow` for UnicodeDecodeError.
We have a debugging statement for some obscure errors we get
when narrows have search terms.  We now show all the narrow
operators.  This isn't really to improve debugging; it's more
to make it easier in the next commit to extract a function
that would make search_term have to be passed back in a tuple.
But it shouldn't hurt debugging either.
2018-04-05 13:27:31 -07:00
Tim Abbott d7658bbec5 test_docs: Add an end-to-end test for HTML settings links.
This would have caught the issue fixed in the last few commits.
2018-04-05 12:22:41 -07:00
Tim Abbott 98889608a2 help: Fix structure of markdown context logic.
Refactoring in this file had resulted in the logic for
html_settings_link being duplicated and extra logic being needed to
ensure these variables were set where they were needed.

This fixes subscriptions_html not being rendered properly in the /help
and /api pages, in addition to removing duplicate code.
2018-04-05 12:22:41 -07:00
Nikhil Kumar Mishra 2cf32bda12 embed link: Add test for link_embed_data_from_cache. 2018-04-05 10:48:40 -07:00
Steve Howell a0aa8d4b11 Add test for find_first_unread_anchor(). 2018-04-05 09:55:54 -07:00
Steve Howell 4cba679d38 Move code for find_first_unread_anchor().
This is a pure refactoring and just pulls the function out
to the top level of the module.  (The prior commit extracted
it inside a larger function to make a nicer diff.)
2018-04-05 09:55:54 -07:00
Steve Howell d8a95c6517 Extract find_first_unread_anchor().
This is a pure refactoring.
2018-04-05 09:55:54 -07:00
Tim Abbott c06565d909 users: Improve testing for user_ids_to_users. 2018-04-04 16:31:30 -07:00
novokrest 4d2082ab14 actions.py: Obtain bot profiles by using users.user_ids_to_users().
Remove models.get_user_profiles_by_ids() and
obtain user's bots profiles in actions.get_service_dicts_for_bots() by
users.user_ids_to_users() instead of models.get_user_profiles_by_ids().
Fixes #8939
2018-04-04 16:24:55 -07:00
novokrest 807a6ccf2c users.py: Implement user_ids_to_users() by generic_bulk_cached_fetch().
Optimize users.user_ids_to_users() implementation
by using generic_bulk_cached_fetch() to obtain user profiles
2018-04-04 16:24:55 -07:00
Tim Abbott 53e47e6991 messages: Modify access_message for is_history_public_to_subscribers.
This completes the Message side of #2745.
2018-04-04 16:18:47 -07:00
Tim Abbott bec71d7a50 messages: Add a server-level setting to control private stream history.
We don't indend for this server-level setting to exist in the long
term; the purpose of this is just to make it easy to test this code
path for development purposes.

This implements much of the Message side part of #2745.
2018-04-04 16:18:46 -07:00
Tim Abbott 228f41e916 messages: Pass UserProfile to is_public_stream_by_name and rename.
The new name can_access_stream_history_by_name gets to the point of
what this function actually does.  And passing in a user object lets
us define what this does based on the user subscribed.
2018-04-04 15:13:11 -07:00
Tim Abbott 5e82d750c5 get_messages: Refactor ok_to_include_history to accept a UserProfile.
If we make history accessible to some stream subscribers of private
streams, we'll need the UserProfile to be available here.
2018-04-04 15:06:53 -07:00
Rhea Parekh f4ad464d82 import script: Fix broken links to attachments.
The comments explain this pretty well, but basically because we
rewrite the realm ID during the import process, we need to edit all
the message bodies that link to an attachment to instead link to the
post-processed URL where that file will be hosted on the new server.

Fixes #8926.
2018-04-04 10:05:15 -07:00
Rhea Parekh 5a9cea4134 import script: re map foreign key of UserProfile.last_active_message_id. 2018-04-04 08:53:09 -07:00
Rhea Parekh ed36314042 import script: Fix 're_map_foreign_keys' logging error. 2018-04-04 08:53:09 -07:00
Rhea Parekh 877c7760b7 import script: re_map Attachment foreign keys. 2018-04-04 08:53:09 -07:00
novokrest a613f3c0a1 get_owned_bot_dicts: Optimize admin's bot dicts loading from database.
Implement few optimizations for reading admin's bot dicts from database
for a constants number of requests:
- add models.get_user_profiles_by_ids() for reading bots profiles
  by single query from database
- add models.get_services_for_bots() for reading services for bots
  by single query from database
- add bot_config.get_bot_configs() for reading config data for bots
  by single query from database

Fixes #8838
2018-04-03 09:10:32 -07:00
Tim Abbott 758d7b9146 bots: Clean up editing bots impacting non-bot users.
This fixes a bug where the endpoint for editing bot users would allow
an organization administrator to edit the full name of a bot user.

A combination of this an another recently fixed bug made it possible
for this process to set a `bot_owner` for a non-bot user; so we also
include a migration to fix that for any users that might have had our
model invariants corrupted in that way.
2018-04-02 18:58:26 -07:00