Commit Graph

3186 Commits

Author SHA1 Message Date
Rhea Parekh f7398cbb09 slack import: Implement custom profile fields.
Add custom profile fields in the slack converted
data 'realm' file.
Added tests for the custom profile fields.

Fixes #8928
2018-04-10 13:28:53 -07:00
Rhea Parekh 852e8516b4 slack import: Add custom profile fields.
Build CustomProfileField and CustomProfileFieldValue
for every user and process the field type after getting an
entire list of the custom fields.
2018-04-10 13:28:53 -07:00
rht a183186672 slack importer: User session.get to recycle previous connections. 2018-04-09 22:02:01 -07:00
Shubham Dhama b650b6b38c markdown: Add @stream as an alias for @all.
Fixes: #8930.
2018-04-09 16:35:14 -07:00
rht 630adb406b Slack importer: Map Slack channel mentions to Zulip stream mentions. 2018-04-09 10:47:39 -07:00
Rhea Parekh 035c440ff3 import script: Support import custom profile fields.
Import of Custom profile fields is only supported for slack
import script for now.
2018-04-09 10:45:35 -07:00
Tim Abbott c41d7ee300 slack import: Write reasonable multi-line JSON.
This is a lot better for debugging.
2018-04-09 10:45:35 -07:00
Rhea Parekh ed7127c8b4 import script: Delete medium sized avatars if it exists.
Deletion of medium sized image is done if it exists before calling the
function 'ensure_medium_avatar_image', to avoid potentially confusing
problems with left-over medium-size avatar images from a previous run
being used when repeatedly importing the same realm in a development
environment..

Fixes #8949.
2018-04-08 07:04:24 -07:00
rht f6feac1316 Slack importer: Map Slack command for mentions to Zulip's all.
Fixes #9003.
2018-04-07 20:02:39 -07:00
Rhea Parekh e037c2f93e import script: Fix upload links.
Rendered content is None for Slack imports, hence it is replaced only
for Zulip->Zulip imports.

Fixes #8959.
2018-04-07 20:01:20 -07:00
Rhea Parekh b3f951d2cf import script: User profile ids should be allocated before allocating bot ids. 2018-04-07 13:28:33 +05:30
Rhea Parekh 2baa9bc16e Import: Add subdomain in the import script.
Also remove user input of subdomain in the slack data
conversion script.
2018-04-06 09:12:56 -07:00
Tim Abbott c06565d909 users: Improve testing for user_ids_to_users. 2018-04-04 16:31:30 -07:00
novokrest 4d2082ab14 actions.py: Obtain bot profiles by using users.user_ids_to_users().
Remove models.get_user_profiles_by_ids() and
obtain user's bots profiles in actions.get_service_dicts_for_bots() by
users.user_ids_to_users() instead of models.get_user_profiles_by_ids().
Fixes #8939
2018-04-04 16:24:55 -07:00
novokrest 807a6ccf2c users.py: Implement user_ids_to_users() by generic_bulk_cached_fetch().
Optimize users.user_ids_to_users() implementation
by using generic_bulk_cached_fetch() to obtain user profiles
2018-04-04 16:24:55 -07:00
Tim Abbott 53e47e6991 messages: Modify access_message for is_history_public_to_subscribers.
This completes the Message side of #2745.
2018-04-04 16:18:47 -07:00
Tim Abbott bec71d7a50 messages: Add a server-level setting to control private stream history.
We don't indend for this server-level setting to exist in the long
term; the purpose of this is just to make it easy to test this code
path for development purposes.

This implements much of the Message side part of #2745.
2018-04-04 16:18:46 -07:00
Tim Abbott 228f41e916 messages: Pass UserProfile to is_public_stream_by_name and rename.
The new name can_access_stream_history_by_name gets to the point of
what this function actually does.  And passing in a user object lets
us define what this does based on the user subscribed.
2018-04-04 15:13:11 -07:00
Rhea Parekh f4ad464d82 import script: Fix broken links to attachments.
The comments explain this pretty well, but basically because we
rewrite the realm ID during the import process, we need to edit all
the message bodies that link to an attachment to instead link to the
post-processed URL where that file will be hosted on the new server.

Fixes #8926.
2018-04-04 10:05:15 -07:00
Rhea Parekh 5a9cea4134 import script: re map foreign key of UserProfile.last_active_message_id. 2018-04-04 08:53:09 -07:00
Rhea Parekh ed36314042 import script: Fix 're_map_foreign_keys' logging error. 2018-04-04 08:53:09 -07:00
Rhea Parekh 877c7760b7 import script: re_map Attachment foreign keys. 2018-04-04 08:53:09 -07:00
novokrest a613f3c0a1 get_owned_bot_dicts: Optimize admin's bot dicts loading from database.
Implement few optimizations for reading admin's bot dicts from database
for a constants number of requests:
- add models.get_user_profiles_by_ids() for reading bots profiles
  by single query from database
- add models.get_services_for_bots() for reading services for bots
  by single query from database
- add bot_config.get_bot_configs() for reading config data for bots
  by single query from database

Fixes #8838
2018-04-03 09:10:32 -07:00
novokrest ad3ea775df actions.py: Rename get_service_dicts_for_bots() to get_service_dicts_for_bot() 2018-04-02 11:18:50 -07:00
Tim Abbott da27461491 bugdown: Rename rewrite_if_relative_link to more clear name.
The name was actually sorta the reverse of what that function did.
2018-04-02 10:37:18 -07:00
Tim Abbott 7d4234b167 bugdown: Only open #narrow hash links inside the app. 2018-04-02 10:34:48 -07:00
Tim Abbott 6744938b6d bugdown: Simplify implementation for target_blank.
The previous version not using startswith() was harder to read.
2018-04-02 10:30:50 -07:00
Tim Abbott 6055a7bceb bugdown: Clean up rewrite_if_relative_link.
This implementation is more obviously correct, and may fix a corner
case where a `/` is missing after the realm URI.
2018-04-02 10:29:32 -07:00
Umair Khan e48f8bdafd check_short_string: Use check_capped_string for impl. 2018-04-02 09:51:09 -07:00
Umair Khan 7fdf043564 profile: Set max length for long text to 500. 2018-04-02 09:51:09 -07:00
Umair Khan 1d4b74482c check_short_string: Change max_length to 50. 2018-04-02 09:51:09 -07:00
Rhea Parekh 1bba6cc4ce slack importer: Support custom emoji reactions. 2018-04-01 23:24:35 -07:00
Rhea Parekh 2c116e46ce slack importer: Get custom emojis. 2018-04-01 23:24:35 -07:00
Rhea Parekh 00c1f25b58 import script: Support custom emojis.
'processing_emojis' check is added in the 'import_uploads'
function, so that the emoji files present in the to be imported
data file can be uploaded.

The procedure of saving emoji files in slack importer is same as
saving attachments and avatars, and the import has the similar
procedure too.
2018-04-01 23:24:35 -07:00
Rhea Parekh c650b8fa3e slack importer: Add zerver_realmemoji. 2018-04-01 23:24:35 -07:00
Rhea Parekh b133d175a7 slack importer: Change 'get_user_data' function implementation.
Change 'get_user_data' function to a more general function
to get data from the slack api using legacy tokens.

Also, change the error handling as upon invalid token,
the response is 200, but the response has an error
field in it.

For eg. Go to the following link with invalid token:
https://slack.com/api/emoji.list?token=xoxp-249056023425
2018-04-01 23:24:35 -07:00
Rhea Parekh 220ad6a386 slack importer: Map standard reactions.
As mentioned in
https://get.slack.help/hc/en-us/articles/202931348-Use-emoji-and-emoticons,
slack supports the standard emoji codes
(https://www.webpagefx.com/tools/emoji-cheat-sheet/)
and majority of them are already supported in Zulip.
2018-04-01 23:24:35 -07:00
Rhea Parekh 6f867fee40 import script: Support import of reactions. 2018-04-01 23:24:33 -07:00
Rhea Parekh 8a028142d8 slack importer: Remove id allocation function and its implementation.
Remove allocation ID function from slack import script. All the IDs
count will start from 0. Hence the ID List returned
by the allocation function is of no use, and we remove its implementation.
(example: get_total_messages_and_attachments function is of no use anymore,
hence we remove it)
2018-04-01 23:10:55 -07:00
Rhea Parekh d147bd25d0 import script: Change file path of the upload in the import script.
In importing avatars, we use the implementation where the 'avatar_path'
is seperately calculated using realm and user ID and then the content
of the path provided in the avatar's 'records.json' are copied to this
'avatar_path'.

Similary, here for the uploads, 's3_file_name' is seperately calculated
using the realm ID and uploaded file name and then the content of the
path provided in upload's 'records.json' are copied to this 's3_file_name'.
2018-04-01 23:04:14 -07:00
Rhea Parekh ff34d07fa0 import script: Add function to update model ids after allocation.
Add function 'update_model_ids' to remove repetitive code.
2018-04-01 22:29:23 -07:00
Rhea Parekh a2ecdeb28d import script: re_map minor foreign keys. 2018-04-01 22:29:23 -07:00
Rhea Parekh 078453554e import script: re_map Message foreign keys. 2018-04-01 22:29:23 -07:00
Rhea Parekh 93aabcb81c import script: re_map Subscription foreign keys. 2018-04-01 22:29:23 -07:00
Rhea Parekh 9ef7870c5a import script: re_map Recipient foreign keys. 2018-04-01 22:29:23 -07:00
Rhea Parekh 4537223ba7 import script: re_map UserProfile foreign keys. 2018-04-01 22:29:23 -07:00
Rhea Parekh 1314e7d247 import script: re_map Stream foreign keys.
'recipient_field' is added as a bool variable in the function
'update_id_map' to update the recipient foreign keys.

Recipient Foreign Key is equal to the UserProfile ID, if the
type is 1, and the same is equal to Stream ID, if the type is 2.
Hence a check is added in the 'update_id_map' field for this.
2018-04-01 22:29:23 -07:00
Rhea Parekh 8624ba4132 import script: re_map Realm foreign keys.
All the objects with realm ID as the foreign keys need to
be remapped with updated with the allocated ID.
Also the ID of the realm object itself is updated with the allocated
ID.
2018-04-01 22:29:23 -07:00
Rhea Parekh 2b0ee472af import script: Refactor re_map_foreign_keys.
The 'id_field' bool variable is added to the function just to check
if the field is the ID of that object, and not the foreign key relation.
For foreign key field names, a "_id" has to be added after the field name,
however we don't need that for the ID field of the object.
2018-04-01 22:29:23 -07:00
Rhea Parekh cd0871bae4 Import script: Add id allocation functions. 2018-04-01 22:29:23 -07:00