Commit Graph

377 Commits

Author SHA1 Message Date
bulat22101 adebc75740 pep8: Fix E502 violations 2016-12-03 10:56:36 -08:00
Sidhant Bhavnani 8c0c12c1d9 pep8: Fix E303 violations. 2016-12-02 15:34:11 -08:00
Alex Huang c8ddea16c3 pep8: Fix E122. 2016-12-01 23:16:35 -08:00
Rafid Aslam c5316b4002 lint: Fix E127 pep8 violations.
Fix pep8: E127 continuation line over-indented for visual indent
style issue.
2016-12-01 10:23:55 -08:00
Rafid Aslam 41bd88d5ed pep8: Fix E301 pep8 violations.
Fix "E301: expected (1 or 2) blank line" pep8 violations.
2016-11-29 08:51:44 -08:00
Rafid Aslam 7a2282986a pep8: Fix E225 pep8 violations. 2016-11-28 15:21:15 -08:00
Tim Abbott 3d1bcb05e1 tornado: Move event_queue.py to zerver/tornado/.
Fixes #729.
2016-11-26 22:29:28 -08:00
Tim Abbott 282f74609c tornado: Move socket code to zerver/tornado/. 2016-11-26 22:29:27 -08:00
Tim Abbott b83361bcfc tornado: Extract create_tornado_application helper. 2016-11-26 21:43:13 -08:00
Tim Abbott 9e9066f77a tornado: Extract AsyncDjangoHandler to its own file. 2016-11-26 21:43:11 -08:00
Tim Abbott 8739f4d9f1 tornado: Move tornado_ioloop_logging to new zerver/tornado tree. 2016-11-26 21:24:05 -08:00
Umair Khan 671c45524b Django 1.10: Use new app discovery system.
'django.db.models.loading' is removed due to which both
'django.db.models.get_app' and 'django.db.models.get_models' are removed.

See https://docs.djangoproject.com/en/1.10/releases/1.9/#features-removed-in-1-9
2016-11-26 15:04:20 -08:00
Anders Kaseorg 207cf6302b Always start python via shebang lines.
This is preparation for supporting using Python 3 in production.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-26 14:46:37 -08:00
Vladislav Manchev d7e1e4a2c0 Add initial implementation of custom realm filters.
This PR was abandoned by Vladislav and then substantially modified by
Igor Tokarev and Tim Abbott to complete it and fix a number of bugs.

Fixes #544.
2016-11-17 17:11:25 -08:00
Steve Howell ca43cbc654 logging: Reducing logging in run-dev.py for queue workers.
In dev, we no longer log that individual queue workers were launched.

Instead, in dev (and prod as well), we log a message with the total
count.
2016-11-17 11:12:02 -08:00
Umair Khan 9b0f9bf326 Django 1.10: apply_response_fixes was removed.
Ref: https://code.djangoproject.com/ticket/26052?cversion=0&cnum_hist=2
2016-11-14 16:09:12 -08:00
Umair Khan 5cfd54994a Django 1.10: validate is replaced with check.
See
https://docs.djangoproject.com/en/dev/releases/1.7/#modeladmin-validators
for explanation.
2016-11-14 16:09:12 -08:00
Tim Abbott f6d3c63a01 realm_alias: Access realm alias objects in lower-case. 2016-11-09 16:31:48 -08:00
Umair Khan 682aa1f298 Django 1.10: Use add_argument for options in BaseCommand. 2016-11-04 10:20:23 -07:00
Rishi Gupta 655940acaf create_realm.py: Add basic checks to name and subdomain.
Don't allow the empty string for organization name, and don't pass
do_create_realm a subdomain that's already taken.
2016-11-03 13:59:11 -07:00
Rishi Gupta 1944ac844a RealmAlias: Drop uniqueness constraint when REALMS_HAVE_SUBDOMAINS.
Removes the uniqueness constraint on RealmAlias.domain, and adds a function
can_add_alias that checks for uniqueness conditional on
settings.REALMS_HAVE_SUBDOMAINS.
2016-11-03 13:53:01 -07:00
Rishi Gupta e4334f4735 do_create_realm: Remove domain argument from do_create_realm.
The domain argument was being used in do_create_realm to create a
RealmAlias. Callers are now responsible for creating a RealmAlias
when desired.
2016-11-03 13:53:01 -07:00
Rishi Gupta c0f1b5d9f6 Change Realm defaults to Community defaults.
Previously, we set restrict_to_domain and invite_required differently
depending on whether we were setting up a community or a corporate
realm. Setting restrict_to_domain requires validation on the domain of the
user's email, which is messy in the web realm creation flow, since we
validate the user's email before knowing whether the user intends to set up
a corporate or community realm. The simplest solution is to have the realm
creation flow impose as few restrictions as possible (community defaults),
and then worry about restrict_to_domain etc. after the user is already in.

We set the test suite to explictly use the old defaults, since several of
the tests depend on the old defaults.

This commit adds a database migration.
2016-11-03 13:53:01 -07:00
K.Kanakhin c89a2ffe2f Add raised exception output to development tornado running.
Replaces the default tornado ioloop tracback exception handler in
debug mode to add output for exceptions to the logging.

Fixes #2150.
2016-11-03 11:16:12 -07:00
Rishi Gupta db0e509422 do_create_realm: Replace domain argument with string_id.
Turns string_id into a required argument, and domain into an optional
argument.
2016-11-02 22:46:34 -07:00
Rishi Gupta 9ef8536cc6 models.Realm: Require Realm.string_id to be non-NULL.
Adds a database migration, adds a new string_id argument to the management
realm creation command, and adds a short name field to the web realm
creation form when REALMS_HAVE_SUBDOMAINS is False.
2016-11-02 22:46:34 -07:00
Tim Abbott 9b7a3f040c Remove now-unused /json/get_events endpoint. 2016-10-27 21:34:58 -07:00
Tim Abbott 86e933a4a1 process_queue: Suppress USING_RABBITMQ warnings in test suite. 2016-10-27 12:36:06 -07:00
Steve Howell cba8be6bbb Use bulk_remove_subscriptions() in remove_users_from_stream.
We are deprecating do_remove_subscription.
2016-10-20 22:05:19 -07:00
Steve Howell 98e4452e07 Replace do_add_subscription() in add_users_to_streams. 2016-10-20 22:05:19 -07:00
Tim Abbott 3b0b65c2d0 management: Fix recent argument parsing regressions.
Fixes a regression introduced in
fbdf539bf2.
2016-10-16 20:36:33 -07:00
Laura Hampton fbdf539bf2 Replace optparse with argparse in the several management commands.
Specifically:
* fill_memcached_caches.py
* initialize_voyager_db.py
* logout_all_users.py
2016-10-16 17:08:35 -07:00
Eklavya Sharma b741c7351d email_mirror: Use correct string types. 2016-10-11 11:27:35 -07:00
Umair Khan d7efdf050c Ignore docs directory while creating translation strings.
Fixes: #1941.
2016-10-10 08:23:56 -07:00
Umair Khan a9343a65d5 Don't localize javascript comments.
Fixes: #1934.
2016-10-10 08:21:02 -07:00
Brock Whittaker 3efe3601b8 Allow for templates/settings to be read and compiled.
This allows for the handlebars templates in the template/settings
subdirectory to be found and compiled.
2016-10-05 22:24:22 -07:00
Rishi Gupta 777fcaa6a0 Add new organization type field to Realm objects.
Adds a new field org_type to Realm.  Defaults for restricted_to_domain
and invite_required are now controlled by org_type at time of realm
creation (see zerver.lib.actions.do_create_realm), rather than at the
database level.  Note that the backend defaults are all
org_type=corporate, since that matches the current assumptions in the
codebase, whereas the frontend default is org_type=community, since if
a user isn't sure they probably want community.

Since we will likely in the future enable/disable various
administrative features based on whether an organization is corporate
or community, we discuss those issues in the realm creation form.
Before we actually implement any such features, we'll want to make
sure users understand what type of organization they are a member of.

Choice of org_type (via radio button) has been added to the realm
creation flow and the realm creation management command, and the
open-realm option removed.

The database defaults have not been changed, which allows our testing code
to work unchanged.

[includes some HTML/CSS work by Brock Whittaker to make it look nice]
2016-10-05 17:01:46 -07:00
Tomasz Kolek dbeab6aa6f Optimize checks of test database state by moving into Python.
Previously, the generate-fixtures shell script by called into Django
multiple times in order to check whether the database was in a
reasonable state.  Since there's a lot of overhead to starting up
Django, this resulted in `test-backend` and `test-js-with-casper`
being quite slow to run a single small test (2.8s or so) even on my
very fast laptop.

We fix this is by moving the checks into a new Python library, so that
we can avoid paying the Django startup overhead 3 times unnecessarily.
The result saves about 1.2s (~40%) from the time required to run a
single backend test.

Fixes #1221.
2016-10-05 10:40:19 -07:00
Tim Abbott 22fd7ba02a avatar: Move avatar hash computations to their own file. 2016-10-02 21:19:10 -07:00
Sahil Dua 058587da77 Remove extra new lines at the ends of Zulip authoried files.
Fixes #1627.

[tweaked by tabbott to avoid patching third-party modules, for now]
2016-09-26 21:05:24 -07:00
Umair Khan b4eb3a55be Annotate zerver/management/commands/realm_emoji.py. 2016-09-12 08:10:50 -07:00
Umair Khan f8c904e611 Annotate zerver/management/commands/deliver_email.py. 2016-09-12 08:10:50 -07:00
Umair Khan c096473e06 Annotate zerver/management/commands/check_redis.py. 2016-09-12 08:10:50 -07:00
Steve Howell 64cb5d0778 Annotate commands/export.py. 2016-09-11 17:15:35 -07:00
Steve Howell 52c6efa0a8 Annotate commands/export_single_user.py. 2016-09-11 17:15:35 -07:00
Steve Howell bb6fcaa4e7 Annotate commands/export_usermessage_batch.py. 2016-09-11 17:15:35 -07:00
Steve Howell 85e30a6d35 Annotate commands/import.py. 2016-09-11 17:15:35 -07:00
Tim Abbott 8fbbb3a471 Annotate send_password_reset_email.py. 2016-09-11 17:09:27 -07:00
Vishnu Ks a7ead9e99d settings: Eliminate ADMIN_DOMAIN for creating initial realm.
We now use `./manage.py generate_realm_creation_link` as the flow flow
for creating one's first realm.
2016-08-25 09:37:33 -07:00
Tim Abbott 3bdbc33e94 initialize_voyager_db: Fix zulip.com hardcoding. 2016-08-22 21:36:15 -07:00
Tim Abbott e1e384acc0 realm_emoji: Fix management command usage output. 2016-08-19 11:59:38 -07:00
Taranjeet Singh 57253c6ba3 makemessages.py: Fix line with length greater than 120.
This updates file zerver/management/commands/makemessages.py.
2016-08-19 11:56:43 -07:00
Tim Abbott 399f4f8870 Add management command to send password reset emails.
This is a convenient tool to have around.

We require an unusual argument value of "YES" to send to everyone on
the server, since that's something one should do with a great deal of
care.
2016-08-18 20:12:36 -07:00
Tim Abbott 3bae3cd54d settings: Add option to disable email deliverer. 2016-08-12 19:24:45 -07:00
Steve Howell 90494cfec2 export: Note user-owned bot limitations. 2016-08-12 15:10:49 -07:00
Tim Abbott 386c6c2a31 settings: Refactor SYSTEM_ONLY_REALMS configuration. 2016-08-12 12:12:53 -07:00
Steve Howell b0e6d20321 export: Write stats.txt for `./manage.py export <realm>`. 2016-08-12 09:06:10 -07:00
Steve Howell 9582a83d10 export: Fix usage in export_single_user command. 2016-08-11 15:21:30 -07:00
Steve Howell 1d1f36c0b8 export: Always use subprocesses to export UserMessage.
We no longer have an in-process code path to export
UserMessage rows.  We want to only maintain the
subprocess code, which we'll always use in production,
and which will work fine in dev.
2016-08-10 14:16:17 -07:00
Umair Khan e1090118c5 Percent translated should be an integer. 2016-08-09 16:19:07 -07:00
Tim Abbott 6264ff7039 Add new Zulip realm import/export tool.
The previous export tool would only work properly for small realms,
and was missing a number of important features:
* Export of avatars and uploads from S3
* Export of presence data, activity data, etc.
* Faithful export/import of timestamps
* Parallel export of messages
* Not OOM killing for large realms

The new tool runs as a pair of documented management commands, and
solves all of those problems.

Also we add a new management command for exporting the data of an
individual user.
2016-08-08 14:58:18 -07:00
Tim Abbott 1e54897ca7 process_queue: Add missing type annotation. 2016-08-04 15:57:03 -07:00
Tim Abbott 641aa08721 compilemessages: Add mypy annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott a787dee48b makemessages: Add missing annotations and fix string types. 2016-08-04 15:57:03 -07:00
Tim Abbott 92062b9526 process_queue: Add missing annotation. 2016-08-04 15:57:02 -07:00
Umair Khan 2edb79776e Calculate translation percentages.
Fixes: #1397
2016-07-29 11:58:41 -07:00
Eklavya Sharma 3b3b5c7c16 Rename management command email-mirror to email_mirror.
All other zulip management command names have underscores, so
rename email-mirror to email_mirror.

This will also make it possible to import this module, which will
help in writing tests for it.
2016-07-28 14:52:09 -07:00
Eklavya Sharma 6e9bc44123 Re-implement email-mirror using imaplib.
Switch from twisted to imaplib to gain python 3 compatibility and
make code easier to understand.
2016-07-28 14:52:09 -07:00
Tim Abbott 19b860ceec Rename local_settings.py symlink to prod_settings.py. 2016-07-19 20:59:59 -07:00
Tim Abbott 1f4c9eefe4 settings: Extract settings.ZILENCER_ENABLED. 2016-07-19 15:28:41 -07:00
Tim Abbott 3c591aa724 compilemessages: Don't try to include zh-CN and zh_CN in language options. 2016-07-18 15:02:43 -07:00
Tim Abbott 55a98a41d8 compilemessages: Improve error handling for unknown locale. 2016-07-18 15:02:43 -07:00
Taranjeet Singh 24ba060421 Fix line with length greater than 120 in deliver_email.py. 2016-07-18 14:34:34 -07:00
Umair Khan b546391f0b Move locale to static/locale 2016-07-18 13:59:07 -07:00
Umair Khan cdf2664030 `makemessages` command should not remove plurals. 2016-07-18 13:59:07 -07:00
Eklavya Sharma e10caf0b65 Gracefully give up when using email-mirror on python 3.
Running `./manage.py email-mirror` used to fail on python 3
because twisted.mail.imap4 is not python 3 compatible.
Display a message informing the user that email-mirror is not
available on python 3 instead of failing with a traceback.

Also add tools/test-management to py3-backend.
2016-07-18 09:57:28 -07:00
Eklavya Sharma da36947400 Change unbuffering strategy in runtornado.py.
runtornado unbuffers its output using
sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0).
This is not python 3 compatible since we can't specify
buffering on a text stream in python 3.  So use the '-u'
option of python when calling runtornado.py to make output
unbuffered.
2016-07-17 10:31:15 -07:00
Umair Khan a07a810a2e Fix regex for capturing frontned strings.
Fixes: #1155
2016-07-15 14:48:05 +05:00
Taranjeet a8a4caf2c0 zerver: Fix lines with length greater than 120. 2016-07-08 11:41:43 -07:00
Eklavya Sharma 83640ed0cd runtornado.py: Ignore due to incorrect stubs. 2016-07-07 10:09:35 -07:00
Eklavya Sharma 6fd8906358 rename_stream.py: Fix broken code.
* get_realm returns None if no matching realm is present, but
  create_stream.py assumed it raises Realm.DoesNotExist.
* encoded/decode strings properly.
2016-07-07 10:08:29 -07:00
Eklavya Sharma c679c180f5 enqueue_file.py: Add type hint for mypy. 2016-07-07 10:07:51 -07:00
Eklavya Sharma 896c18a57b email-mirror.py: Make it pass on mypy in python 3.
* Replace filter by list comprehension.

* Add '# type: ignore' to statements which use attributes from
  modeule `posix`, since stubs for posix are missing on python 3.
2016-07-07 10:07:28 -07:00
Eklavya Sharma 17cb6e00bd create_stream.py: Fix broken code.
* get_realm returns None if no matching realm is present, but
  create_stream.py assumed it raises Realm.DoesNotExist.
* encoded/decode strings properly.
2016-07-07 10:06:39 -07:00
Umair Khan 47fb293f8a i18n: Add English locale to language list if missing.
The English locale data is only present if the user has run `manage.py
makemigrations` on the system.
2016-07-06 11:34:28 -07:00
Umair Khan 035fceb814 Add dynamically loaded language dropdown. 2016-07-04 11:56:02 -07:00
Vishnu Ks 1cbd39b768 Unique link generator for realm creation. 2016-06-30 15:52:41 -07:00
Tim Abbott e72f41cdec Remove old prototype data import/export tool.
This prototype from Dropbox Hack Week turned out to be too inefficient
to be used for realms with any significant amount of history, so we're
removing it.

It will be replaced by https://github.com/zulip/zulip/pull/673.
2016-06-27 13:47:08 -07:00
rahuldeve a3745178e5 Use django.utils.autoreload to restart queue workers at code change.
Fixes #621, #1045.
2016-06-26 20:12:11 -07:00
Umair Khan 134b165b1a Fix frontend translation string bug.
The translation string which contain \n are not captured
within {{#tr}}{{/tr}} blocks. Fix is to escape the slash.
2016-06-20 11:31:28 -07:00
Vishnu Ks 8213ca135a Move default_stream list to settings. 2016-06-17 16:01:38 -07:00
Umair Khan 6fcfed8d9e Update regexes to capture translation strings.
Strings are captured from:
- i18n.t
- compose_error
- placeholder
2016-06-13 09:03:56 -07:00
Eklavya Sharma d169cc5376 zerver/lib/bulk_create.py: Fix string annotations.
Change string type from `str` to `text_type` where required.
Also fix clashing annotations.
2016-06-13 10:08:14 +05:30
Tim Abbott d9d0515d3b Add mypy bug number for check_redis type: ignore. 2016-06-04 16:11:18 -07:00
medullaskyline 7c2c7fb31c Annotate zerver/lib/bulk_create.py. 2016-06-04 15:51:05 -07:00
Tim Abbott a1a27b1789 Annotate most Zulip management commands. 2016-06-04 10:12:06 -07:00
Tim Abbott c2bea0fa08 zulip_finish: Remove useless return statement. 2016-06-04 10:06:31 -07:00
Tim Abbott ab2d325a08 Update production default streams to be less engineering-centric. 2016-05-31 07:38:25 -07:00
Eklavya Sharma 94e4b39112 Replace python2.7 by python everywhere. 2016-05-29 05:03:08 -07:00
Eklavya Sharma 149938d468 Change shebangs from python2.7 to python. 2016-05-29 05:03:08 -07:00