Commit Graph

53 Commits

Author SHA1 Message Date
shanukun 4b00e5da72 refactor: Make acting_user a mandatory kwarg for do_deactivate_realm. 2021-04-02 14:44:41 -07:00
Anders Kaseorg 6e4c3e41dc python: Normalize quotes with Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 11741543da python: Reformat with Black, except quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 3a8cf869db python: Convert os.open(…, O_EXCL) to open(…, "x").
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-11-09 14:31:01 -08:00
Alex Vandiver f4eae83542 export: Only include real, active humans in the displayed count. 2020-10-28 18:31:06 -07:00
Anders Kaseorg 72d6ff3c3b docs: Fix more capitalization issues.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-23 11:46:55 -07:00
Anders Kaseorg fbfd4b399d python: Elide action="store" for argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg 1f2ac1962f python: Elide default=None for argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg a5dbab8fb0 python: Remove redundant dest for argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:04:10 -07:00
Alex Vandiver b23a55e669 export: Add a --deactivate flag which happens just prior to export.
This removes a manual step during export.
2020-08-30 15:26:50 -07:00
Alex Vandiver 15d7e7a6fd export: Prompt for confirmation if a consent message is involved.
We display the text of the consent message, and then continue with the
export, which will scroll the content off the screen.  Allow the
administrator time to examine the contents of the message, and decide
whether to proceed based on that and the fraction of users that have
responded so far.
2020-08-30 15:26:50 -07:00
Alex Vandiver e3ba79b9df export: Show total number of users in the realm as well.
Knowing 10 users opted in isn't useful unless one knows if the realm
has 10 or 100 total users.
2020-08-30 15:26:50 -07:00
Alex Vandiver ea65b75875 export: Move tarball and directory creation as late as possible.
This stops creating empty tarballs and directories if the option
validation fails for some reason.
2020-08-30 15:26:50 -07:00
Anders Kaseorg 6dea085187 docs: Correct “login” as a verb to “log in”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:25:53 -07:00
Tim Abbott 6130a61be0 export: Only print .s with percent_callback to console.
The S3 data export tool's upload code path uses this nice boto
callback feature for showing a progress bar, which is nice for the
management command.  It's spammy/broken in production and the backend
tests, so we change percent_callback to be a parameter passed in so
that it can only be used in the contexts where it makes sense.
2020-07-30 13:14:53 -07:00
Mateusz Mandera 60c4896c3a import: Fix import and export parallelism defaults.
Fixes #14960.
The default of 6 thread may not be appropriate in certain
configurations. Taking half of the numer of CPUs available to the
process will be more flexible.
2020-06-21 09:39:31 -07:00
Anders Kaseorg 5dc9b55c43 python: Manually convert more percent-formatting to f-strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
Anders Kaseorg 69730a78cc python: Use trailing commas consistently.
Automatically generated by the following script, based on the output
of lint with flake8-comma:

import re
import sys

last_filename = None
last_row = None
lines = []

for msg in sys.stdin:
    m = re.match(
        r"\x1b\[35mflake8    \|\x1b\[0m \x1b\[1;31m(.+):(\d+):(\d+): (\w+)", msg
    )
    if m:
        filename, row_str, col_str, err = m.groups()
        row, col = int(row_str), int(col_str)

        if filename == last_filename:
            assert last_row != row
        else:
            if last_filename is not None:
                with open(last_filename, "w") as f:
                    f.writelines(lines)

            with open(filename) as f:
                lines = f.readlines()
            last_filename = filename
        last_row = row

        line = lines[row - 1]
        if err in ["C812", "C815"]:
            lines[row - 1] = line[: col - 1] + "," + line[col - 1 :]
        elif err in ["C819"]:
            assert line[col - 2] == ","
            lines[row - 1] = line[: col - 2] + line[col - 1 :].lstrip(" ")

if last_filename is not None:
    with open(last_filename, "w") as f:
        f.writelines(lines)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 16:04:12 -07:00
Anders Kaseorg 67e7a3631d python: Convert percent formatting to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 15:02:09 -07:00
Anders Kaseorg 8dd83228e7 python: Convert "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, but with the
NamedTuple changes reverted (see commit
ba7906a3c6, #15132).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 15:31:20 -07:00
Tim Abbott 8e7ce7cc79 python: Sort migrations/management command imports with isort.
This is a preparatory commit for using isort for sorting all of our
imports, merging changes to files where we can easily review the
changes as something we're happy with.

These are also files with relatively little active development, which
means we don't expect much merge conflict risk from these changes.
2020-01-14 13:07:47 -08:00
Anders Kaseorg 78b018989e export: Refuse to overwrite an existing directory or tarball.
Previously, incorrectly passing an existing directory to the
`manage.py export --output` option would remove its contents without
warning.  Abort instead.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-12 16:04:55 -07:00
Anders Kaseorg becef760bf cleanup: Delete leading newlines.
Previous cleanups (mostly the removals of Python __future__ imports)
were done in a way that introduced leading newlines.  Delete leading
newlines from all files, except static/assets/zulip-emoji/NOTICE,
which is a verbatim copy of the Apache 2.0 license.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-06 23:29:11 -07:00
Tim Abbott 544f9c74ce export: Use outbox emoji for managing who is exported.
This is a little more unambiguous.
2019-06-17 16:10:28 -07:00
Wyatt Hoodes 4dd8c133a9 export: Rename `--upload-to-s3` to be `--upload`.
The upload option will no longer be limited to strictly S3 uploads. This
commit serves as a preliminary step for supporting LOCAL_UPLOADS_DIR as
part of the public only export feature.
2019-05-20 19:59:57 -07:00
Vishnu Ks 06983298ba export: Add support for exporting realm with member consent.
This lets us handle directly in our tooling the user experience that
we document for exporting a realm with member consent (before, it
required unpleasant manual work).
2019-05-15 12:35:32 -07:00
Wyatt Hoodes bafcf3c664 export.py: Add 'delete after upload' option for removing tarball.
This allows removal of the local tarball upon a succesful s3 upload.

A part of the public-only-realm-export webapp feature.
2019-04-12 10:50:06 -07:00
Wyatt Hoodes 0db7d6c31b export.py: Refactor './manage.py export' core logic.
This commit serves as the first step in supporting "public export" as a
webapp feature.  The refactoring was done as a means to allow calling
the export logic from elsewhere in the codebase.
2019-04-12 10:50:06 -07:00
Anders Kaseorg b37ed12df2 export: Let mkdtemp pick its favorite directory.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:08:48 -08:00
Tim Abbott a3ac94fa26 management: Support multi-line usage in all management commands. 2019-01-09 10:39:29 -08:00
Tim Abbott 848b2f687c export: Add support for public-streams-only exports.
Previously, this wasn't an explicit feature of the export tool.

Note that the current version still includes metadata on private
streams and private message recipients, just not their messages.
2019-01-07 16:52:02 -08:00
Tim Abbott c6371bb3ef export: Add option to upload exports to S3.
This should make it more convenient to operationalize providing
exports from Zulip Cloud.

Fixes #11178.
2019-01-07 15:28:17 -08:00
Tim Abbott d66967f851 import: Use os.path.expanduser when accessing paths.
This fixes an issue where passing a path like `~/exports/foo` would
result in a `~` directory being created and the export/import not
working correctly.
2018-08-13 13:32:53 -07:00
Tim Abbott f04d6ed19e python: Sort imports in management commands. 2017-11-15 15:43:47 -08:00
rht c4fcff7178 refactor: Replace super(.*self) with Python 3-specific super().
We change all the instances except for the `test_helpers.py`
TimeTrackingCursor monkey-patching, which actually needs to specify
the base class.
2017-10-30 14:30:25 -07:00
rht a311678190 zerver/management: Use python 3 syntax for typing. 2017-10-26 15:24:56 -07:00
Tim Abbott 15c0380a33 export: Fix exporting with an output file at a relative path.
Fixes #7051.
2017-10-17 20:47:29 -07:00
Tim Abbott 66a3f514d4 zilencer: Remove long-obsolete Deployment model.
This was a precursor to RemoteZulipServer, which is no longer used for
anything, and was cluttering the codebase.
2017-10-11 23:48:01 -07:00
rht 32650c8fdc zerver/management: Remove print_function. 2017-09-27 18:05:45 -07:00
rht e239e97351 zerver/management: Remove absolute_import. 2017-09-27 10:00:39 -07:00
neiljp (Neil Pilgrim) ab4cbf81f0 mypy: Add assertions of get_realm results when parser requires realm. 2017-09-26 13:49:01 -07:00
Vishnu Ks 19f8da4995 management: Use self.get_realm in export command. 2017-08-08 14:13:19 -07:00
Rishi Gupta aa845e7f60 models: Replace ScheduledJob with ScheduledEmail.
ScheduledJob was written for much more generality than it ended up being
used for. Currently it is used by send_future_email, and nothing
else. Tailoring the model to emails in particular will make it easier to do
things like selectively clear emails when people unsubscribe from particular
email types, or seamlessly handle using the same email on multiple realms.
2017-07-17 16:05:38 -07:00
Rishi Gupta b3654c7b7e confirmation: Remove vestiges of the MitUser table.
The MitUser table was removed in df525ad.

confirm_mituser.html could have been accessed through the last few lines of
confirmation/views.py:
    templates.insert(0, 'confirmation/confirm_%s.html'
                     % (obj._meta.model_name,))

The commit message on df525ad suggests there was another way
confirm_mituser.html could have been called, but I don't currently see
evidence for it in the code.
2017-07-07 18:53:00 -07:00
Vaida d5517bae36 Delete the old zulip.com "referrals" system.
This system hasn't been in active use for several years, and had some 
problems with it's design.  So it makes sense to just remove it to declutter
the codebase.

Fixes #5655.
2017-07-07 14:59:18 -07:00
Abhijeet Kaur 8f88b045a4 Rename "Administration" to "Organization" in the settings UI.
This better sets expectatations for the fact that in Zulip, the
Organization settings UI is available read-only to non-administrator
users.

Tweaked by tabbott to update some additional references.
2017-04-07 17:32:56 -07:00
Rishi Gupta 11346e50bb management commands: Use Realm.string_id instead of domain in print statements. 2017-03-13 09:44:32 -07:00
Rishi Gupta 2b0a7fd0ba Rename models.get_realm_by_string_id to get_realm.
Finishes the refactoring started in c1bbd8d. The goal of the refactoring is
to change the argument to get_realm from a Realm.domain to a
Realm.string_id. The steps were

* Add a new function, get_realm_by_string_id.

* Change all calls to get_realm to use get_realm_by_string_id instead.

* Remove get_realm.

* (This commit) Rename get_realm_by_string_id to get_realm.

Part of a larger migration to remove the Realm.domain field entirely.
2017-01-04 17:12:23 -08:00
reyha 82e32ad255 Access realm by `string_id` in management commands.
`Realm.string_id` replaces 'Realm.domain'
in the management commands.

Fixes #2325.
2016-12-14 10:38:03 -08:00
Steve Howell 64cb5d0778 Annotate commands/export.py. 2016-09-11 17:15:35 -07:00