Commit Graph

720 Commits

Author SHA1 Message Date
Mateusz Mandera 1a3297d122 email_mirror: Make send_to_email_mirror.py use mirror_email_message.
We change the send_to_email_mirror management command, to send messages
to the email mirror through the mirror_email_message function instead of
process_message - this makes the message follow a similar codepath as
emails sent into the mirror with the postfix configuration, which means
going through the MirrorWorker queue. The reason for this is to make
this command useful for testing the new email mirror rate limiter.
2019-03-18 11:16:58 -07:00
Puneeth Chaganti d75d2c9974 soft-deactivation: Run catch-up when "auto" deactivate is run.
When soft deactivation is run for in "auto" mode (no emails are
specified and all users inactive for specified number of days are
deactivated), catch-up is also run in the "auto" mode if
AUTO_CATCH_UP_SOFT_DEACTIVATED_USERS is True.

Automatically catching up soft-deactivated users periodically would
ensure a good user experience for returning users, but on some servers
we may want to turn off this option to save on some disk space.

Fixes #8858, at least for the default configuration, by eliminating
the situation where there are a very large number of messages to recover.
2019-03-14 11:53:15 -07:00
Puneeth Chaganti c61d3420e8 soft-deactivation: Extract getting users from emails to a function. 2019-03-14 20:57:00 +05:30
Puneeth Chaganti 7f06f6bd73 soft-deactivation: Raise exception with all missing emails at once. 2019-03-13 17:16:22 -07:00
Tim Abbott 76a06c8e54 send_test_email: Clean up output and provide advice.
Previously, while we sent emails using both noreply addresses, we
didn't make clear what was going on, leading to some potential
confusion.
2019-03-13 14:27:52 -07:00
Hemanth V. Alluri 99c3e2ecdc management: Support sending custom headers when testing a webhook.
this commit adds an option to specify custom headers when using the
`./manage.py send_webhook_fixture_message` tool.
2019-03-13 12:40:47 -07:00
Harshit Bansal 94649f58f2 tests: Refactor `query_ldap()` and add complete test coverage. 2019-03-09 22:12:51 -08:00
Harshit Bansal b519e6594e management: Move `query_ldap` function to `zproject/backends.py`.
This will make it simpler to organize and unit-test all of our
authentication backend code.
2019-03-09 22:12:36 -08:00
Harshit Bansal 4a9bd89f47 ldap: Continue syncing other fields even if a field is missing.
Earlier the behavior was to raise an exception thereby stopping the
whole sync. Now we log an error message and skip the field. Also
fixes the `query_ldap` command to report missing fields without
error.

Fixes: #11780.
2019-03-05 16:19:27 -08:00
Anders Kaseorg c6e60ebf0e backup: Set cwd while running pg_dump.
This avoids a spurious permission error inside the Postgres
`resolve_symlinks` function if we don’t have access to the current
working directory (e.g. we’re running with cwd /root inside `su
zulip`).

While we’re here, add a defensive `--` argument.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-28 14:40:28 -08:00
Anders Kaseorg 649235cfec python: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-22 16:54:36 -08:00
Anders Kaseorg ce01a4c5a3 backup: Add Zulip, OS, and PostgreSQL version to the tarball.
==> zulip-backup/os-version <==
    Ubuntu trusty

    ==> zulip-backup/postgres-version <==
    90324

    ==> zulip-backup/zulip-version <==
    2.0.0-rc1+git
    2.0.0-rc1-50-gbf169b8-dirty

Documentation added by tabbott.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-14 15:11:57 -08:00
Vishnu Ks 949d098e99 management: Add tooling for transfering local uploads to s3.
This solves a common migration problem for folks who cut corners when
first setting up Zulip.

Fixes #11294.
2019-02-14 14:31:46 -08:00
Anders Kaseorg e0a51948d9 script: Add ready-to-run tooling for doing backups.
Based on an initial version by Tim Abbott (#11204).

Fixes #552.
2019-02-11 17:30:37 -08:00
Tim Abbott 4826393ec4 hipchat: Fix help documentation for slim_mode. 2019-02-07 16:17:01 -08:00
Vishnu Ks 28769e040f invites: Add backend for multiuse admin invites. 2019-02-07 15:41:00 -08:00
Anders Kaseorg 2a2e744b27 management: Remove unused imports in management commands.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:12:36 -08:00
Vishnu Ks bec875a9af import realm: Use processes for resizing avatar images.
This should significantly improve the data import performance when
importing large open source realms from Slack.

Fixes #11009.
2019-01-25 12:37:12 -08:00
Anders Kaseorg c0792808e2 deliver_email, deliver_scheduled_messages: Remove lock file.
Tim believes it’s useless.

https://github.com/zulip/zulip/pull/11285#discussion_r248110926

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:56:49 -08:00
Anders Kaseorg 25d6107a4f docs: Avoid hardcoded /tmp paths in miscellaneous documentation.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:16:12 -08:00
Anders Kaseorg b37ed12df2 export: Let mkdtemp pick its favorite directory.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:08:48 -08:00
Aditya Bansal aae90d5d6b deliver_scheduled_messages: Make notification bot deliver reminders.
We do this since we are yet to figure out how the entire realm
internal bots scenerio should work and therefore for the timming
we will use notification bot to deliver the reminders.
2019-01-14 12:11:27 -08:00
Harshit Bansal 00ca98348a ldap: Shorten the name of logger of `sync_ldap_user_data` command. 2019-01-14 09:50:30 -08:00
Harshit Bansal 5037b2c5c4 management: Fix a bug in parser of `sync_ldap_user_data` command.
After recent changes for extending the `sync_ldap_user_data` to
update a single user, it was not running without specifying any
arguments.
2019-01-14 09:50:30 -08:00
Harshit Bansal 71761bc2da ldap: Add a setting to automatically deactivate non_matching users.
Fixes: #11151.
2019-01-13 19:04:09 -08:00
Harshit Bansal 348f370b79 management: Extract `sync_user_from_ldap()`. 2019-01-13 18:51:50 -08:00
Harshit Bansal e09ef69a8d management: Extend `sync_ldap_user_data` to allow update of a single user. 2019-01-13 18:51:50 -08:00
Tim Abbott 53436766c1 hipchat: Improve import of public room subscribers.
Now, if you pass an api_key, we'll initialize the public room
subscribers to be whatever they were at the time the import happened.

Also, document the situation on the caveats section.
2019-01-09 16:50:00 -08:00
Tim Abbott a3ac94fa26 management: Support multi-line usage in all management commands. 2019-01-09 10:39:29 -08:00
Mateusz Mandera 3e5f89f2fe email_mirror: Add send_to_email_mirror management command.
Closes #11195. We add a management command to allow us to send emails
to the email mirror directly. The command doesn't require any
configuring of email sending or receiving for the email mirror,
it passes the emails directly using the process_message function.
2019-01-09 10:36:16 -08:00
sahil839 4dcccf32f8 zerver: Add feature for notification at rename of a stream.
Feature of sending notification to the stream using notification bot
is added. user_profile is also passed to do_rename_stream for using
the name of user who renamed the stream in notification.
Notification is sent to the stream using
internal_send_stream_message in do_rename_stream.

Fixes #11034.
2019-01-07 19:13:01 -08:00
Tim Abbott 848b2f687c export: Add support for public-streams-only exports.
Previously, this wasn't an explicit feature of the export tool.

Note that the current version still includes metadata on private
streams and private message recipients, just not their messages.
2019-01-07 16:52:02 -08:00
Tim Abbott c6371bb3ef export: Add option to upload exports to S3.
This should make it more convenient to operationalize providing
exports from Zulip Cloud.

Fixes #11178.
2019-01-07 15:28:17 -08:00
Tim Abbott ff9714eb32 register_server: Prompt for ToS inside the command.
This way, we don't need to tell new sysadmins how to rerun the script.
2019-01-02 11:19:36 -08:00
Tim Abbott 22f39ff8e7 list_realms: Display '' as string_id for root domain realm. 2018-12-30 10:36:18 -08:00
Tim Abbott e48c9245ff list_realms: Fix the --all option.
I don't think anyone's used this in years, but it could be handy.
2018-12-30 10:36:18 -08:00
Hudda 50e14dbe97 list_realms: Print the full domains of the organizations.
This adds a column to display the full domains of realms.  Tweaked by
tabbott to use fewer columns.

Fixes part of #11015.
2018-12-30 10:36:00 -08:00
Rishi Gupta 0844c9a6ec emails: Update text for password_reset.
Note that a pretty common use case for this is a realm admin sending this to
everyone after an import from HipChat or Slack. So this adds the realm_name
to the title (so that there is something they might recognize) and kept the
wording generic enough to accommodate the user not having clicked anything
to get this email.

Also strengthens the tests a bit to better test the complicated template
logic.
2018-12-20 16:26:19 -08:00
Tim Abbott d005f4edd3 management: Fix incorrect create_user docsting on ToS. 2018-12-16 22:23:35 -08:00
Tim Abbott d9912cb23c management: Fix buggy create_user management command.
The create_user management command incorrectly was double-calling
notify_new_user.
2018-12-16 22:22:52 -08:00
Tim Abbott 67981725ec send_email: Fix migration code path for ScheduledEmail.
The previous migration code path was broken in two ways:

* ScheduledEmail objects generally contain a `None` value for
  whichever of `to_user_id` and `to_email` isn't in use; this could
  result in us sending a [None] to send_email(), which doesn't make
  sense.

* We were calling handle_send_email_format_changes in the wrong order
  with respect to the JSON loading process.

Thanks to Tom Daff for the report!
2018-12-14 12:46:31 -08:00
Tim Abbott 81271b0d20 sync_ldap_user_data: Process deactivated users as well.
Technically, we will only need to process deactivated users for the
purpose of reactivating them (and can ignore, e.g., name changes).
But it's simplest to just process them unconditionally.
2018-12-13 16:24:15 -08:00
Tim Abbott 5dd646f33f ldap: Add support for syncing avatar images from LDAP.
This should make life a lot more convenient for organizations that use
the LDAP integration and have their avatars in LDAP already.

This hasn't been end-to-end tested against LDAP yet, so there may be
some minor revisions, but fundamentally, it works, has automated
tests, and should be easy to maintain.

Fixes #286.
2018-12-13 13:39:22 -08:00
Tim Abbott ee901ac8b1 django: Remove the Django contrib Sites app to fix 2FA QR codes.
Apparently, Django's get_current_site function (used, e.g., in
django-two-factor to look up the domain to use in QR codes) first
tries to use the Sites framework, and if unavailable, does the right
thing (namely, using request.get_host()).

We don't use the Sites framework for anything in Zulip, so the correct
fix is to just remove it.

Fixes #11014.
2018-12-12 15:01:55 -08:00
Joshua Pan 441dca0eaf email: Support delivery email in email change system. 2018-12-06 17:43:26 -08:00
Tim Abbott e603237010 email: Convert accounts code to use delivery_email.
A key part of this is the new helper, get_user_by_delivery_email.  Its
verbose name is important for clarity; it should help avoid blind
copy-pasting of get_user (which we'll also want to rename).
Unfortunately, it requires detailed understanding of the context to
figure out which one to use; each is used in about half of call sites.

Another important note is that this PR doesn't migrate get_user calls
in the tests except where not doing so would cause the tests to fail.
This probably deserves a follow-up refactor to avoid bugs here.
2018-12-06 16:21:38 -08:00
Tim Abbott 02a79b677b send_email: Extract handle_email_format_changes and use.
Apparently, we have a second code path where we might try to call
send_email library functions on old data, namely in the
queue_processors codebase.  So we apply the same migration logic here.
2018-12-04 16:08:18 -08:00
Raymond Akornor 92dc3637df send_email: Add support for multiple recipients.
This adds a function that sends provided email to all administrators
of a realm, but in a single email. As a result, send_email now takes
arguments to_user_ids and to_emails instead of to_user_id and
to_email.

We adjust other APIs to match, but note that send_future_email does
not yet support the multiple recipients model for good reasons.

Tweaked by tabbott to modify `manage.py deliver_email` to handle
backwards-compatibily for any ScheduledEmail objects already in the
database.

Fixes #10896.
2018-12-03 15:12:11 -08:00
Tim Abbott 7a2d93ed6e show_admins: Add assertion to make mypy happy. 2018-11-29 11:51:39 -08:00
Tim Abbott 8a60ab7314 show_admins: Rewrite to use management library.
This makes this command more standardized, and helps avoid future bugs
like the one fixed in the last commit.
2018-11-29 11:40:17 -08:00