Commit Graph

26 Commits

Author SHA1 Message Date
Anders Kaseorg 722842a0aa rocketchat: Remove unnecessary SHA-1 hashing of direct message groups.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-17 15:56:00 -07:00
Anders Kaseorg 27b0618704 data_import: Fix IdMapper typing.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-17 15:56:00 -07:00
Anders Kaseorg 48202389b8 ruff: Bump target-version from py38 to py310.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg e08a24e47f ruff: Fix UP006 Use `list` instead of `List` for type annotation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
roanster007 52692a6448 refactor: Rename `huddle` to `direct_message_group` in non API.
This commit performs a sweep on the first batch of non API
files to rename "huddle" to "direct_message_group`.

It also renames variables and methods of type -
"huddle_message" to "group_direct_message".

This is a part of #28640
2024-07-04 07:56:31 -07:00
Prakhar Pratyush 00474608c5 zulip_update: Send group DM for realm imported from other product.
When the export is NOT generated by another zulip server,
while importing:
* Set the 'zulip_update_announcements_level' to the latest level
as we don't want to send all the older update messages to them.

* Send a group DM to admins, suggesting them to configure the
stream in order to avoid missing future update messages.

Fixes #29041.
2024-05-08 17:05:59 -07:00
John Lu a5cf0ec526
refactor: Replace HUDDLE with DIRECT_MESSAGE_GROUP.
Replaced HUDDLE attribute with DIRECT_MESSAGE_GROUP using VS Code search,
part of a general renaming of the object class.

Fixes part of #28640.

Co-authored-by: JohnLu2004 <JohnLu10212004@gmail.com>
2024-03-21 16:39:33 -07:00
Anders Kaseorg cd96193768 models: Extract zerver.models.realms.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Anders Kaseorg 45bb8d2580 models: Extract zerver.models.users.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Alex Vandiver 710524465a messages: Switch limits from sender__realm to realm.
We now have a `realm_id` on Message; use it, rather than having to
check the sender's realm.  This is theoretically different for
cross-realm bots, but these changes are all in tests where that does
not apply.
2023-09-11 15:00:37 -07:00
Lauryn Menard d53b854a7c backend-tests: Update "private message" or "PM" to "direct message".
Updates comments and test strings/names with "private message" or
"PM" to use "direct message" instead.
2023-06-23 11:24:13 -07:00
Alex Vandiver fe654b76b7 data_import: Stop tar'ing up converted data.
`./manage.py import` does not take a tarball; it takes a directory.
Making a separate tarball is a waste of CPU time and disk, as it is
never used.

This was included in the commit of the initial Slack conversion code
in 5b37c5562b and propagated from there into every conversion tool.

Remove the unnecessary tarball creation.
2023-02-26 17:42:01 -08:00
Anders Kaseorg edab4ec997 rocketchat: Import timezone-aware datetimes.
The bson library creates naive datetime objects by default.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-12-27 10:34:30 -08:00
evykassirer 8a9e68e026 emoji: Finish script to generate emoji_names.py with CLDR data.
This script pulls from our previously custom-written emoji strings
and fills in the rest from CLDR. It also removes 4 custom emoji which
collide with some of the new CLDR names (they will now just be called
by their CLDR name).
2022-10-04 12:29:35 -07:00
Florian Pritz a276603766 rocketchat: Deduplicate and ignore huddle rooms with same users.
If there are more than 1 room with the same set of users, the import
will fail due to a unique constraint on the huddle_hash. Figuring out
why and which room is causing this database error is kinda difficult.

We deduplicate those cases here and simply merge the rooms together.
Note however, that the deduplication does not work as expected so we
simply ignore them all together for now and only raise an exception
along some logging output. At least this way, it is pretty clear what is
wrong and you do not have to wait to get a database error during the
actual import.

We also ignore empty huddle rooms since those are the duplicates that
caused problems for me and if they are empty, ignoring them is easier
than trying to get the merge to work.

Not sure where those channels come from since we discovered this with
production data.

Signed-off-by: Florian Pritz <bluewind@xinu.at>
2022-09-09 16:57:24 -07:00
Florian Pritz 26fe028534 rocketchat: Truncate long stream names.
These will lead to an error during import otherwise.

Signed-off-by: Florian Pritz <bluewind@xinu.at>
2022-09-09 16:57:24 -07:00
Mateusz Mandera ac9e927989 import_realm: Validate emails of users being imported.
Having users with malformed email addresses in our database can lead to
hard to debug issues and we really should try to prevent that.
2022-08-29 10:39:14 -07:00
Steve Howell 6ec5a5193a tests: Verify emoji codes. 2021-12-06 13:22:47 -08:00
Priyansh Garg 42f231c85c data_import: Ignore Rocket.Chat livechat streams/messages.
This resolves the issues reported in #20108, major chunk of which were
due to the incomplete support for importing the livechat streams/messages
in the tool. So, it's best not to import any livechat streams/messages for
now until a complete support for importing the same is developed.
2021-11-07 09:50:55 -08:00
Priyansh Garg 17409a78be data_import: Fix a few KeyError bugs in Rocket.Chat import tool.
This commit fixes a few bugs in Rocket.Chat import tool as reported on CZO.

Link: https://chat.zulip.org/#narrow/stream/9-issues/topic/Rocketchat.20Import
2021-11-03 16:50:56 -07:00
Priyansh Garg 0db9b7287b data_import: Import Rocket.Chat messages from direct discussions.
This commit adds functionality to import messages from the
Discussions having direct channels as their parent. As we don't
have topics in the PMs, the messages are imported in interleaved
form in the imported direct channels/PMs.

This was completely unsupported earlier and would have resulted in
an error.
2021-11-01 17:09:11 -07:00
Priyansh Garg 54452fef6c data_import: Fix channel mentions in Rocket.Chat import.
While the STREAM_LINK_REGEX and STREAM_TOPIC_LINK_REGEX
identifies the stream and topic mentions in the content
correctly (tested by printing out the matches), the
stream/topic mentions are still not linked to the
corresponding streams/topics for imported messages, as
a `zulip_message` instance is required for linking these
mentions to actual streams/topics (see `StreamPattern`
class in `markdown/__init__.py`) which is not provided
while processing the markdown for imported messages.
2021-08-09 06:38:26 -07:00
Priyansh Garg aed4e48da7 data_import: Import attachments from Rocket.Chat. 2021-08-09 06:38:26 -07:00
Priyansh Garg 65e28907cb data_import: Import custom emoji from Rocket.Chat. 2021-08-09 06:38:26 -07:00
Priyansh Garg 044fe547d3 data_import: Add huddle import support for Rocket.Chat. 2021-07-28 15:45:54 -07:00
Priyansh Garg 24dd0ff96c data_import: Add rocket chat import tool.
This commit allows to import the following from rocketchat:
* All users
* All public/private channels
* All teams and its public/private channels
* All discussion rooms as topics in their parent channel
* All the messages in all the channels
* All private conversations
* Reactions on messages (except for custom emojis)
* Mentions in messages (except @all, @here mentions)
2021-07-28 15:28:56 -07:00