Mirror
/
zulip
mirror of https://github.com/zulip/zulip.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
zulip/zerver/tests/test_hipchat_importer.py

77 lines
2.1 KiB
Python
Raw Normal View History

hipchat import: Start adding tests. These test are for the handling of HipChat sender info. The data formats are somewhat inconsistent and sometimes require us to generate "mirror" users, so this is potentially fragile code if we don't cover it well.
2018-10-23 18:24:06 +02:00
from typing import Any, Dict
python: Sort imports with isort. Fixes #2665. Regenerated by tabbott with `lint --fix` after a rebase and change in parameters. Note from tabbott: In a few cases, this converts technical debt in the form of unsorted imports into different technical debt in the form of our largest files having very long, ugly import sequences at the start. I expect this change will increase pressure for us to split those files, which isn't a bad thing. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 00:54:34 +02:00
from zerver.data_import.hipchat import get_hipchat_sender_id
from zerver.data_import.hipchat_user import UserHandler
from zerver.data_import.sequencer import IdMapper
from zerver.lib.test_classes import ZulipTestCase
hipchat import: Start adding tests. These test are for the handling of HipChat sender info. The data formats are somewhat inconsistent and sometimes require us to generate "mirror" users, so this is potentially fragile code if we don't cover it well.
2018-10-23 18:24:06 +02:00
class HipChatImporter(ZulipTestCase):
def test_sender_ids(self) -> None:
realm_id = 5
user_handler = UserHandler()
hipchat: Make conversion work with UUID ids from Stride. Normal hipchat exports use integer ids for their users and "rooms," which we just borrowed during conversion. Atlassian Stride uses stride UUIDs for these instead, but otherwise has the same export format. We now introduce IdMapper to handle external ids that aren't integer. The IdMapper will map UUID ids to ints and remember them. For ints it just leaves them alone. Fixes #10805.
2018-11-13 23:03:18 +01:00
user_id_mapper = IdMapper()
test_hipchat_importer: Fix a type: ignore issue. Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-01 01:18:36 +02:00
self.assertEqual(user_id_mapper.get(1), 1)
hipchat: Make conversion work with UUID ids from Stride. Normal hipchat exports use integer ids for their users and "rooms," which we just borrowed during conversion. Atlassian Stride uses stride UUIDs for these instead, but otherwise has the same export format. We now introduce IdMapper to handle external ids that aren't integer. The IdMapper will map UUID ids to ints and remember them. For ints it just leaves them alone. Fixes #10805.
2018-11-13 23:03:18 +01:00
hipchat import: Start adding tests. These test are for the handling of HipChat sender info. The data formats are somewhat inconsistent and sometimes require us to generate "mirror" users, so this is potentially fragile code if we don't cover it well.
2018-10-23 18:24:06 +02:00
# Simulate a "normal" user first.
user_with_id = dict(
id=1,
# other fields don't matter here
)
user_handler.add_user(user=user_with_id)
python: Convert assignment type annotations to Python 3.6 style. This commit was split by tabbott; this piece covers the vast majority of files in Zulip, but excludes scripts/, tools/, and puppet/ to help ensure we at least show the right error messages for Xenial systems. We can likely further refine the remaining pieces with some testing. Generated by com2ann, with whitespace fixes and various manual fixes for runtime issues: - invoiced_through: Optional[LicenseLedger] = models.ForeignKey( + invoiced_through: Optional["LicenseLedger"] = models.ForeignKey( -_apns_client: Optional[APNsClient] = None +_apns_client: Optional["APNsClient"] = None - notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE) - signup_notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE) + notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE) + signup_notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE) - author: Optional[UserProfile] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE) + author: Optional["UserProfile"] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE) - bot_owner: Optional[UserProfile] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL) + bot_owner: Optional["UserProfile"] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL) - default_sending_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE) - default_events_register_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE) + default_sending_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE) + default_events_register_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE) -descriptors_by_handler_id: Dict[int, ClientDescriptor] = {} +descriptors_by_handler_id: Dict[int, "ClientDescriptor"] = {} -worker_classes: Dict[str, Type[QueueProcessingWorker]] = {} -queues: Dict[str, Dict[str, Type[QueueProcessingWorker]]] = {} +worker_classes: Dict[str, Type["QueueProcessingWorker"]] = {} +queues: Dict[str, Dict[str, Type["QueueProcessingWorker"]]] = {} -AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional[LDAPSearch] = None +AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional["LDAPSearch"] = None Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-22 01:09:50 +02:00
normal_message: Dict[str, Any] = dict(
hipchat import: Start adding tests. These test are for the handling of HipChat sender info. The data formats are somewhat inconsistent and sometimes require us to generate "mirror" users, so this is potentially fragile code if we don't cover it well.
2018-10-23 18:24:06 +02:00
sender=dict(
id=1,
python: Use trailing commas consistently. Automatically generated by the following script, based on the output of lint with flake8-comma: import re import sys last_filename = None last_row = None lines = [] for msg in sys.stdin: m = re.match( r"\x1b\[35mflake8 \|\x1b\[0m \x1b\[1;31m(.+):(\d+):(\d+): (\w+)", msg ) if m: filename, row_str, col_str, err = m.groups() row, col = int(row_str), int(col_str) if filename == last_filename: assert last_row != row else: if last_filename is not None: with open(last_filename, "w") as f: f.writelines(lines) with open(filename) as f: lines = f.readlines() last_filename = filename last_row = row line = lines[row - 1] if err in ["C812", "C815"]: lines[row - 1] = line[: col - 1] + "," + line[col - 1 :] elif err in ["C819"]: assert line[col - 2] == "," lines[row - 1] = line[: col - 2] + line[col - 1 :].lstrip(" ") if last_filename is not None: with open(last_filename, "w") as f: f.writelines(lines) Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-10 05:23:40 +02:00
),
python: Convert assignment type annotations to Python 3.6 style. This commit was split by tabbott; this piece covers the vast majority of files in Zulip, but excludes scripts/, tools/, and puppet/ to help ensure we at least show the right error messages for Xenial systems. We can likely further refine the remaining pieces with some testing. Generated by com2ann, with whitespace fixes and various manual fixes for runtime issues: - invoiced_through: Optional[LicenseLedger] = models.ForeignKey( + invoiced_through: Optional["LicenseLedger"] = models.ForeignKey( -_apns_client: Optional[APNsClient] = None +_apns_client: Optional["APNsClient"] = None - notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE) - signup_notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE) + notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE) + signup_notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE) - author: Optional[UserProfile] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE) + author: Optional["UserProfile"] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE) - bot_owner: Optional[UserProfile] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL) + bot_owner: Optional["UserProfile"] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL) - default_sending_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE) - default_events_register_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE) + default_sending_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE) + default_events_register_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE) -descriptors_by_handler_id: Dict[int, ClientDescriptor] = {} +descriptors_by_handler_id: Dict[int, "ClientDescriptor"] = {} -worker_classes: Dict[str, Type[QueueProcessingWorker]] = {} -queues: Dict[str, Dict[str, Type[QueueProcessingWorker]]] = {} +worker_classes: Dict[str, Type["QueueProcessingWorker"]] = {} +queues: Dict[str, Dict[str, Type["QueueProcessingWorker"]]] = {} -AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional[LDAPSearch] = None +AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional["LDAPSearch"] = None Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-22 01:09:50 +02:00
)
hipchat import: Start adding tests. These test are for the handling of HipChat sender info. The data formats are somewhat inconsistent and sometimes require us to generate "mirror" users, so this is potentially fragile code if we don't cover it well.
2018-10-23 18:24:06 +02:00
sender_id = get_hipchat_sender_id(
realm_id=realm_id,
hipchat: Limit messages in slim mode. For messages with strange senders, we don't import messages. Basically, we only import a message if it has sender with an id that maps to a non-deleted user.
2018-11-21 19:46:00 +01:00
slim_mode=False,
hipchat import: Start adding tests. These test are for the handling of HipChat sender info. The data formats are somewhat inconsistent and sometimes require us to generate "mirror" users, so this is potentially fragile code if we don't cover it well.
2018-10-23 18:24:06 +02:00
message_dict=normal_message,
hipchat: Make conversion work with UUID ids from Stride. Normal hipchat exports use integer ids for their users and "rooms," which we just borrowed during conversion. Atlassian Stride uses stride UUIDs for these instead, but otherwise has the same export format. We now introduce IdMapper to handle external ids that aren't integer. The IdMapper will map UUID ids to ints and remember them. For ints it just leaves them alone. Fixes #10805.
2018-11-13 23:03:18 +01:00
user_id_mapper=user_id_mapper,
hipchat import: Start adding tests. These test are for the handling of HipChat sender info. The data formats are somewhat inconsistent and sometimes require us to generate "mirror" users, so this is potentially fragile code if we don't cover it well.
2018-10-23 18:24:06 +02:00
user_handler=user_handler,
)
self.assertEqual(sender_id, 1)
bot_message = dict(
sender='fred_bot',
)
# Every message from fred_bot should
# return the same sender_id.
fred_bot_sender_id = 2
for i in range(3):
sender_id = get_hipchat_sender_id(
realm_id=realm_id,
hipchat: Limit messages in slim mode. For messages with strange senders, we don't import messages. Basically, we only import a message if it has sender with an id that maps to a non-deleted user.
2018-11-21 19:46:00 +01:00
slim_mode=False,
hipchat import: Start adding tests. These test are for the handling of HipChat sender info. The data formats are somewhat inconsistent and sometimes require us to generate "mirror" users, so this is potentially fragile code if we don't cover it well.
2018-10-23 18:24:06 +02:00
message_dict=bot_message,
hipchat: Make conversion work with UUID ids from Stride. Normal hipchat exports use integer ids for their users and "rooms," which we just borrowed during conversion. Atlassian Stride uses stride UUIDs for these instead, but otherwise has the same export format. We now introduce IdMapper to handle external ids that aren't integer. The IdMapper will map UUID ids to ints and remember them. For ints it just leaves them alone. Fixes #10805.
2018-11-13 23:03:18 +01:00
user_id_mapper=user_id_mapper,
hipchat import: Start adding tests. These test are for the handling of HipChat sender info. The data formats are somewhat inconsistent and sometimes require us to generate "mirror" users, so this is potentially fragile code if we don't cover it well.
2018-10-23 18:24:06 +02:00
user_handler=user_handler,
)
self.assertEqual(sender_id, fred_bot_sender_id)
hipchat import: Support sender_id == 0 use case.
2018-10-23 15:22:26 +02:00
id_zero_message = dict(
sender=dict(
id=0,
name='hal_bot',
),
)
hal_bot_sender_id = 3
for i in range(3):
sender_id = get_hipchat_sender_id(
realm_id=realm_id,
hipchat: Limit messages in slim mode. For messages with strange senders, we don't import messages. Basically, we only import a message if it has sender with an id that maps to a non-deleted user.
2018-11-21 19:46:00 +01:00
slim_mode=False,
hipchat import: Support sender_id == 0 use case.
2018-10-23 15:22:26 +02:00
message_dict=id_zero_message,
hipchat: Make conversion work with UUID ids from Stride. Normal hipchat exports use integer ids for their users and "rooms," which we just borrowed during conversion. Atlassian Stride uses stride UUIDs for these instead, but otherwise has the same export format. We now introduce IdMapper to handle external ids that aren't integer. The IdMapper will map UUID ids to ints and remember them. For ints it just leaves them alone. Fixes #10805.
2018-11-13 23:03:18 +01:00
user_id_mapper=user_id_mapper,
hipchat import: Support sender_id == 0 use case.
2018-10-23 15:22:26 +02:00
user_handler=user_handler,
)
self.assertEqual(sender_id, hal_bot_sender_id)