Commit Graph

712 Commits

Author SHA1 Message Date
Harshit Bansal 4a9bd89f47 ldap: Continue syncing other fields even if a field is missing.
Earlier the behavior was to raise an exception thereby stopping the
whole sync. Now we log an error message and skip the field. Also
fixes the `query_ldap` command to report missing fields without
error.

Fixes: #11780.
2019-03-05 16:19:27 -08:00
Anders Kaseorg c6e60ebf0e backup: Set cwd while running pg_dump.
This avoids a spurious permission error inside the Postgres
`resolve_symlinks` function if we don’t have access to the current
working directory (e.g. we’re running with cwd /root inside `su
zulip`).

While we’re here, add a defensive `--` argument.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-28 14:40:28 -08:00
Anders Kaseorg 649235cfec python: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-22 16:54:36 -08:00
Anders Kaseorg ce01a4c5a3 backup: Add Zulip, OS, and PostgreSQL version to the tarball.
==> zulip-backup/os-version <==
    Ubuntu trusty

    ==> zulip-backup/postgres-version <==
    90324

    ==> zulip-backup/zulip-version <==
    2.0.0-rc1+git
    2.0.0-rc1-50-gbf169b8-dirty

Documentation added by tabbott.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-14 15:11:57 -08:00
Vishnu Ks 949d098e99 management: Add tooling for transfering local uploads to s3.
This solves a common migration problem for folks who cut corners when
first setting up Zulip.

Fixes #11294.
2019-02-14 14:31:46 -08:00
Anders Kaseorg e0a51948d9 script: Add ready-to-run tooling for doing backups.
Based on an initial version by Tim Abbott (#11204).

Fixes #552.
2019-02-11 17:30:37 -08:00
Tim Abbott 4826393ec4 hipchat: Fix help documentation for slim_mode. 2019-02-07 16:17:01 -08:00
Vishnu Ks 28769e040f invites: Add backend for multiuse admin invites. 2019-02-07 15:41:00 -08:00
Anders Kaseorg 2a2e744b27 management: Remove unused imports in management commands.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:12:36 -08:00
Vishnu Ks bec875a9af import realm: Use processes for resizing avatar images.
This should significantly improve the data import performance when
importing large open source realms from Slack.

Fixes #11009.
2019-01-25 12:37:12 -08:00
Anders Kaseorg c0792808e2 deliver_email, deliver_scheduled_messages: Remove lock file.
Tim believes it’s useless.

https://github.com/zulip/zulip/pull/11285#discussion_r248110926

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:56:49 -08:00
Anders Kaseorg 25d6107a4f docs: Avoid hardcoded /tmp paths in miscellaneous documentation.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:16:12 -08:00
Anders Kaseorg b37ed12df2 export: Let mkdtemp pick its favorite directory.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:08:48 -08:00
Aditya Bansal aae90d5d6b deliver_scheduled_messages: Make notification bot deliver reminders.
We do this since we are yet to figure out how the entire realm
internal bots scenerio should work and therefore for the timming
we will use notification bot to deliver the reminders.
2019-01-14 12:11:27 -08:00
Harshit Bansal 00ca98348a ldap: Shorten the name of logger of `sync_ldap_user_data` command. 2019-01-14 09:50:30 -08:00
Harshit Bansal 5037b2c5c4 management: Fix a bug in parser of `sync_ldap_user_data` command.
After recent changes for extending the `sync_ldap_user_data` to
update a single user, it was not running without specifying any
arguments.
2019-01-14 09:50:30 -08:00
Harshit Bansal 71761bc2da ldap: Add a setting to automatically deactivate non_matching users.
Fixes: #11151.
2019-01-13 19:04:09 -08:00
Harshit Bansal 348f370b79 management: Extract `sync_user_from_ldap()`. 2019-01-13 18:51:50 -08:00
Harshit Bansal e09ef69a8d management: Extend `sync_ldap_user_data` to allow update of a single user. 2019-01-13 18:51:50 -08:00
Tim Abbott 53436766c1 hipchat: Improve import of public room subscribers.
Now, if you pass an api_key, we'll initialize the public room
subscribers to be whatever they were at the time the import happened.

Also, document the situation on the caveats section.
2019-01-09 16:50:00 -08:00
Tim Abbott a3ac94fa26 management: Support multi-line usage in all management commands. 2019-01-09 10:39:29 -08:00
Mateusz Mandera 3e5f89f2fe email_mirror: Add send_to_email_mirror management command.
Closes #11195. We add a management command to allow us to send emails
to the email mirror directly. The command doesn't require any
configuring of email sending or receiving for the email mirror,
it passes the emails directly using the process_message function.
2019-01-09 10:36:16 -08:00
sahil839 4dcccf32f8 zerver: Add feature for notification at rename of a stream.
Feature of sending notification to the stream using notification bot
is added. user_profile is also passed to do_rename_stream for using
the name of user who renamed the stream in notification.
Notification is sent to the stream using
internal_send_stream_message in do_rename_stream.

Fixes #11034.
2019-01-07 19:13:01 -08:00
Tim Abbott 848b2f687c export: Add support for public-streams-only exports.
Previously, this wasn't an explicit feature of the export tool.

Note that the current version still includes metadata on private
streams and private message recipients, just not their messages.
2019-01-07 16:52:02 -08:00
Tim Abbott c6371bb3ef export: Add option to upload exports to S3.
This should make it more convenient to operationalize providing
exports from Zulip Cloud.

Fixes #11178.
2019-01-07 15:28:17 -08:00
Tim Abbott ff9714eb32 register_server: Prompt for ToS inside the command.
This way, we don't need to tell new sysadmins how to rerun the script.
2019-01-02 11:19:36 -08:00
Tim Abbott 22f39ff8e7 list_realms: Display '' as string_id for root domain realm. 2018-12-30 10:36:18 -08:00
Tim Abbott e48c9245ff list_realms: Fix the --all option.
I don't think anyone's used this in years, but it could be handy.
2018-12-30 10:36:18 -08:00
Hudda 50e14dbe97 list_realms: Print the full domains of the organizations.
This adds a column to display the full domains of realms.  Tweaked by
tabbott to use fewer columns.

Fixes part of #11015.
2018-12-30 10:36:00 -08:00
Rishi Gupta 0844c9a6ec emails: Update text for password_reset.
Note that a pretty common use case for this is a realm admin sending this to
everyone after an import from HipChat or Slack. So this adds the realm_name
to the title (so that there is something they might recognize) and kept the
wording generic enough to accommodate the user not having clicked anything
to get this email.

Also strengthens the tests a bit to better test the complicated template
logic.
2018-12-20 16:26:19 -08:00
Tim Abbott d005f4edd3 management: Fix incorrect create_user docsting on ToS. 2018-12-16 22:23:35 -08:00
Tim Abbott d9912cb23c management: Fix buggy create_user management command.
The create_user management command incorrectly was double-calling
notify_new_user.
2018-12-16 22:22:52 -08:00
Tim Abbott 67981725ec send_email: Fix migration code path for ScheduledEmail.
The previous migration code path was broken in two ways:

* ScheduledEmail objects generally contain a `None` value for
  whichever of `to_user_id` and `to_email` isn't in use; this could
  result in us sending a [None] to send_email(), which doesn't make
  sense.

* We were calling handle_send_email_format_changes in the wrong order
  with respect to the JSON loading process.

Thanks to Tom Daff for the report!
2018-12-14 12:46:31 -08:00
Tim Abbott 81271b0d20 sync_ldap_user_data: Process deactivated users as well.
Technically, we will only need to process deactivated users for the
purpose of reactivating them (and can ignore, e.g., name changes).
But it's simplest to just process them unconditionally.
2018-12-13 16:24:15 -08:00
Tim Abbott 5dd646f33f ldap: Add support for syncing avatar images from LDAP.
This should make life a lot more convenient for organizations that use
the LDAP integration and have their avatars in LDAP already.

This hasn't been end-to-end tested against LDAP yet, so there may be
some minor revisions, but fundamentally, it works, has automated
tests, and should be easy to maintain.

Fixes #286.
2018-12-13 13:39:22 -08:00
Tim Abbott ee901ac8b1 django: Remove the Django contrib Sites app to fix 2FA QR codes.
Apparently, Django's get_current_site function (used, e.g., in
django-two-factor to look up the domain to use in QR codes) first
tries to use the Sites framework, and if unavailable, does the right
thing (namely, using request.get_host()).

We don't use the Sites framework for anything in Zulip, so the correct
fix is to just remove it.

Fixes #11014.
2018-12-12 15:01:55 -08:00
Joshua Pan 441dca0eaf email: Support delivery email in email change system. 2018-12-06 17:43:26 -08:00
Tim Abbott e603237010 email: Convert accounts code to use delivery_email.
A key part of this is the new helper, get_user_by_delivery_email.  Its
verbose name is important for clarity; it should help avoid blind
copy-pasting of get_user (which we'll also want to rename).
Unfortunately, it requires detailed understanding of the context to
figure out which one to use; each is used in about half of call sites.

Another important note is that this PR doesn't migrate get_user calls
in the tests except where not doing so would cause the tests to fail.
This probably deserves a follow-up refactor to avoid bugs here.
2018-12-06 16:21:38 -08:00
Tim Abbott 02a79b677b send_email: Extract handle_email_format_changes and use.
Apparently, we have a second code path where we might try to call
send_email library functions on old data, namely in the
queue_processors codebase.  So we apply the same migration logic here.
2018-12-04 16:08:18 -08:00
Raymond Akornor 92dc3637df send_email: Add support for multiple recipients.
This adds a function that sends provided email to all administrators
of a realm, but in a single email. As a result, send_email now takes
arguments to_user_ids and to_emails instead of to_user_id and
to_email.

We adjust other APIs to match, but note that send_future_email does
not yet support the multiple recipients model for good reasons.

Tweaked by tabbott to modify `manage.py deliver_email` to handle
backwards-compatibily for any ScheduledEmail objects already in the
database.

Fixes #10896.
2018-12-03 15:12:11 -08:00
Tim Abbott 7a2d93ed6e show_admins: Add assertion to make mypy happy. 2018-11-29 11:51:39 -08:00
Tim Abbott 8a60ab7314 show_admins: Rewrite to use management library.
This makes this command more standardized, and helps avoid future bugs
like the one fixed in the last commit.
2018-11-29 11:40:17 -08:00
Tim Abbott d828eba2f6 show_admins: Fix buggy realm parsing. 2018-11-29 11:40:17 -08:00
Tim Abbott adf27aae4c python: Remove now-unnecessary str_utils library.
This library was absolutely essential as part of our Python 2->3
migration process, but all of its calls should be either no-ops or
encode/decode operations.

Note also that the library has been wrong since the incorrect
refactoring in 1f9244e060.

Fixes #10807.
2018-11-27 11:57:54 -08:00
Raymond Akornor d00b889402 auth: Add an organization reactivation flow with admin confirmation.
This adds a web flow and management command for reactivating a Zulip
organization, with confirmation from one of the organization
administrators.

Further work is needed to make the emails nicer (ideally, we'd send
one email with all the admins on the `To` line, but the `send_email`
library doesn't support that).

Fixes #10783.

With significant tweaks to the email text by tabbott.
2018-11-27 10:58:34 -08:00
Tim Abbott 46acb608b1 tornado: Include port number in logging statements. 2018-11-20 18:45:22 -08:00
Tim Abbott 0cac7e1cd3 tornado: Extract functions for Tornado queue names.
This moves all control for what queue to use for which realm in our
Tornado system to just the sharding.py file; no actual sharding is
done yet.
2018-11-02 17:00:10 -07:00
Tim Abbott ec065e92ee tornado: Store port on SockJS connection object.
This will make it available for use inside our websockets code.
2018-11-02 16:55:33 -07:00
Tim Abbott 75e48459b5 tornado: Support using a port-aware file for dumping event queues.
This should make it possible for there to safely be multiple Tornado
processes running on different ports on the same system.

It may also fix a rare race bug in development, where previously, it
was possible for the Tornados processes for Casper and the main
development server to interfere; I haven't investigated whether this
was a real bug or not, but now those two services will use independent
Tornado files.

We still need to add something to direct traffic between the different
Tornado processes.
2018-11-02 16:47:39 -07:00
Tim Abbott 0976e3bd39 purge_queue: Use more robust Tornado patterns.
We're about to make the list of possible Tornado queue names broader.
2018-11-02 16:06:46 -07:00