Commit Graph

581 Commits

Author SHA1 Message Date
Tim Abbott b01e6d750f management: Remove obsolete update_permissions management command.
This was part of the old Django/guardian-based permissions system.
2017-07-08 14:37:25 -07:00
Tim Abbott 3395f38bf7 management: Remove obsolete gravatar_to_user_avatar management command.
This was used for a 1-time migration in 2013, and definitely isn't
needed anymore.
2017-07-08 14:35:53 -07:00
Vishnu Ks 36972f611c add_users_to_streams: Remove unused imports. 2017-07-08 14:35:42 -07:00
Vishnu Ks 4340b94a2d create_user: Separate password help into multiple lines.
This line is too long and ./tools/lint --pep8 is failing because
of this.
2017-07-08 14:18:49 -07:00
rht 7a9de7766b Add password argument to `manage.py create_user`.
Tweaked by tabbott to provide clear --help output recommending against
setting passwords this way.
2017-07-08 10:59:02 -07:00
Rishi Gupta b3654c7b7e confirmation: Remove vestiges of the MitUser table.
The MitUser table was removed in df525ad.

confirm_mituser.html could have been accessed through the last few lines of
confirmation/views.py:
    templates.insert(0, 'confirmation/confirm_%s.html'
                     % (obj._meta.model_name,))

The commit message on df525ad suggests there was another way
confirm_mituser.html could have been called, but I don't currently see
evidence for it in the code.
2017-07-07 18:53:00 -07:00
Rishi Gupta 8fed9eeb75 confirmation: Make host a required argument in get_link_for_object.
Removes some lines of test from test_email_change.py. The relevant code path
was never utilized by the code itself, just by the tests.
2017-07-07 18:53:00 -07:00
Greg Price 207bf2e1fb i18n: Recognize a little more Handlebars syntax in makemessages.
Usually we write translation expressions as `{{t ... }}`, but `{{ t ... }}`
is equally valid as far as Handlebars is concerned, and it matches how we
usually write simple variable substitutions, as `{{ ... }}`.  So occasionally
someone writes `{{ t ... }}`; currently there are two examples of this
in the codebase, in `settings/bot-{settings,list-admin}.handlebars`.

Probably it'd be good to pick a style and enforce it uniformly, but
until we do, the other style shouldn't break translation.
2017-07-07 17:02:39 -07:00
Umair Khan 0e8231d0f1 process_queue: Recover gracefully after PostgreSQL restart.
- For threaded workers:
Django's autoreloader catches SIGQUIT(3) to reload the program. If
a process being watched by autoreloader exits with status code 3,
reloader will restart the process. To reload, we send SIGUSR1(10)
signal from consumers to a handler in process_queue which then
exits with status code 3.

- For single worker per process:
Catch the SIGUSR1 and quit; supervisorctl will restart the worker
automatically.

Fixes #5512
2017-07-07 16:33:15 -07:00
Vaida d5517bae36 Delete the old zulip.com "referrals" system.
This system hasn't been in active use for several years, and had some 
problems with it's design.  So it makes sense to just remove it to declutter
the codebase.

Fixes #5655.
2017-07-07 14:59:18 -07:00
Tim Abbott dbabe3fd37 add_users_to_streams: Fix check for all_users option. 2017-07-07 13:57:57 -07:00
Tim Abbott 982667acf5 management: Add new library for specifying realms.
This new library is intended to make it easy for management commands
to access a realm or a user in a realm without having to duplicate any
of the annoying parsing/extraction code.
2017-07-07 13:57:57 -07:00
Rishi Gupta a26703109e settings: Change all uses of ZULIP_ADMINISTRATOR to FromAddress.SUPPORT.
Make it less likely that further development will break compatibility with
ZULIP_ADMINISTRATORs of the form "name <email>".

Note that the suggested value for this setting has been
'zulip-admin@example.com' for a while, so hopefully this commit causes no
change for most installations.
2017-07-05 15:33:01 -07:00
Rishi Gupta 577e201df0 emails: Replace DEFAULT_FROM_EMAIL with ZULIP_ADMINISTRATOR.
We're getting rid of DEFAULT_FROM_EMAIL from prod_settings_template, since
we only really need one of {DEFAULT_FROM_EMAIL, ZULIP_ADMINISTRATOR}.
2017-06-29 17:54:33 -07:00
Rishi Gupta 36cd122905 models: Change default org_type to CORPORATE.
Once we implement org_type-specific features, it'll be easy to change a
corporate realm to a community realm, but hard to go the other way. The main
difference (the main thing that makes migrating from a community realm to a
corporate realm hard) is that you'd have to make everyone sign another terms
of service.
2017-06-29 15:14:58 -07:00
Rishi Gupta 78790ce825 send_password_reset_email: Remove spurious send_mail import.
Make grepping for and otherwise understanding our email pathways slightly
easier.
2017-06-29 10:51:12 -07:00
Rishi Gupta a56d703bdc i18n: Remove translation tags in management commands. 2017-06-25 11:28:28 -04:00
Umair Khan b74cc576ae Django 1.11: Make makemessages command compatible.
Now REs have moved to template module. This commit adds a condition
to use trans_real module if the Django version is less than 1.11 else
use template module.
2017-06-13 15:14:05 -07:00
Harshit Bansal e30e244b29 emoji: Moved `unified_reactions.json` file.
Moved `unified_reactions.json` file from `tools/setup/emoji` to
`zerver/management/data` so that it can be used in production
environment.
2017-06-09 16:38:58 -07:00
Rishi Gupta 769c5ab105 emails: Send password reset emails through zerver.lib.send_email.
Previously, the password reset email behaved differently from all the other
email Zulip sends.
2017-06-06 23:22:22 -07:00
rht 940cf9db3b Run queue processors multithreaded in production if system memory <3.5GB.
While running queue processors multithreaded will limit the
performance available to very small systems, it's easy to fix that by
adding more RAM, and previously, Zulip didn't work on such systems at
all, so this is unambiguously an improvement there.

Fixes #32.
Fixes #34.

(Commit message expanded significantly by tabbott.)
2017-06-03 12:19:58 -07:00
Aditya Bansal 15ea059aad pep8: Add compliance with rule E261 makemessages.py. 2017-05-31 17:07:15 -07:00
Aditya Bansal 717e5ae393 pep8: Add compliance with rule E261 commands/create_realm.py. 2017-05-31 17:07:15 -07:00
Maxim Averin 685fb16c39 Switch change_full_name to use RealmAuditLog.
This requires adding an `acting_user` parameter to the
`do_change_bot_owner` function.
2017-05-29 15:22:08 -07:00
Elliott Jin 5b4d2832fc bots: Add management command for making outgoing webhook bot. 2017-05-25 09:45:54 -07:00
Tim Abbott e9968a7a09 change_user_email: Use new get_user_for_mgmt function. 2017-05-24 15:29:59 -07:00
Matt Long 19363b2b77 notification_settings: Refactor notification preference settings.
Previously, all notification preference setting had a dedicated test
and setter. Now, all are handled through a modular function using the
property_types framework.
2017-05-23 14:47:46 -07:00
Aditya Bansal 8a6617e304 pep8: Add compliance with rule E261 to set_default_streams.py. 2017-05-07 23:21:50 -07:00
Aditya Bansal 7271fc8a7b pep8: Add compliance with rule E261 to import.py. 2017-05-07 23:21:50 -07:00
Aditya Bansal d8db632d21 pep8: Add compliance with rule E261 to enqueue_file.py. 2017-05-07 23:21:50 -07:00
Aditya Bansal 6a53c53588 pep8: Add compliance with rule E261 to email_mirror.py. 2017-05-07 23:21:50 -07:00
Aditya Bansal f1b2f10574 pep8: Add compliance with rule E261 to dump_messages.py. 2017-05-07 23:21:50 -07:00
Rishi Gupta 6b9f25a58a Remove duplicated code between send_future_email and send_email pathway.
Note that this change restricts the context for any template that uses
send_future_email to be jsonable.
2017-05-05 14:38:25 -07:00
Rishi Gupta 68c172192b email.py: Change recipients argument of send_future_email to to_email. 2017-05-05 14:38:25 -07:00
Rishi Gupta d70e09b41d notifications: Change sender arg of send_future_email to from_email.
This commit is a step towards the goal of replacing most of the
send_future_email pathway with a call to send_email.

Note that this commit changes the default value of sender from "Zulip
<NOREPLY_EMAIL_ADDRESS>" to "NOREPLY_EMAIL_ADDRESS". NOREPLY_EMAIL_ADDRESS
will soon be changed to have the Zulip in front.
2017-05-05 14:38:25 -07:00
Rishi Gupta e46cbaffa2 email: Remove Mandrill pathways and dependency.
Everything it was doing (send_future_email) can now be done using
ScheduledJob.
2017-05-05 14:38:23 -07:00
Rishi Gupta 925ee8c0f1 Add a send_email function that takes a template_prefix and context.
This commit replaces all uses of django.core.mail.send_mail with send_email,
other than in the password reset flow, since that code looks like it is just
a patch to Django's password reset code.

The send_email function is in a new file, since putting it in
zerver.lib.notifications would create an import loop with confirmation.models.

send_future_email will soon be moved into email.py as well.
2017-05-05 14:20:32 -07:00
Rishi Gupta 92dd76822f email: Move and rename password reset templates. 2017-05-03 23:26:14 -07:00
Tim Abbott 4a67ba241e realm_domains: Restructure library for getting realm domains.
* Remove duplicate list_of_domains_for_realm of get_realm_domains.
* Move get_realm_domains from actions.py.
2017-04-28 21:11:56 -07:00
K.Kanakhin e3e52e7284 email-mirror: Move postfix email mirror integration to separate script.
This fixes a performance problem where we were previously starting up
a full Django process (~0.7s even on a fast machine) every time a new
email came in, potentially allowing users to accidentally DoS a Zulip
server.  Now, we just post over HTTPS, allowing the existing thread
pool support to do its job.

- Add script wrapper to communicate postfix pipe with django web server
  over HTTP(S). It uses shared_secret authentication mode.
- Add django view to process messages from email mirror server.
- Clean management command `email-mirror`. Left just functional
  for cron email processing.
- Add routes for new tornado view.
- Change pipe script in master process postfix config template
  based on updated script.
- Add tests.

Tweaked by tabbott to adjust the directory and set better defaults.

Fixes #2421.
2017-04-24 21:24:23 -07:00
hackerkid b2504084ab Replace timezone.now with timezone_now. 2017-04-16 12:28:56 -07:00
hackerkid 55c3d12078 Replace timezone.utc with timezone_utc. 2017-04-16 12:28:56 -07:00
Yago González e710110a9e i18n: Automatically strip Handlebars strings.
Some Handlebars strings contained whitespaces characters at their ends.
With this, such characters are removed, as well as multiple spaces
(like the ones produced by code indentation).

This also includes a couple of fixes that removes spaces that were
intentionally placed before/after the string to translate.
2017-04-14 17:37:25 -07:00
Yago González 31c92cdcbc refactor: Keep sub replacements' format uniform. 2017-04-14 17:37:25 -07:00
Umair Khan 9dec5306ce makemessages.py: Handle unicode strings.
This commit applies to Python 2. It seems that sorted()
cannot handle non-ascii character unless the strings are
marked as Unicode.
2017-04-11 22:32:06 -07:00
Abhijeet Kaur 8f88b045a4 Rename "Administration" to "Organization" in the settings UI.
This better sets expectatations for the fact that in Zulip, the
Organization settings UI is available read-only to non-administrator
users.

Tweaked by tabbott to update some additional references.
2017-04-07 17:32:56 -07:00
Umair Khan 846e8686c4 makemessages: Exclude hidden files.
Fixes #3981.
2017-04-05 11:09:32 -07:00
Harshit Bansal 664b23c495 commands: Rename `realm_alias` command to `realm_domain`. 2017-04-04 15:48:03 -07:00
Harshit Bansal 362ee482c0 commands/realm_alias.py: Remove any occurrences of `alias`. 2017-04-04 15:48:03 -07:00
Harshit Bansal 983225612d actions.py: Rename `get_realm_aliases()` to `get_realm_domains()`. 2017-04-04 15:48:03 -07:00
Harshit Bansal 92c512d679 zerver/models.py: Rename can_add_alias() to can_add_realm_domain(). 2017-04-04 15:48:03 -07:00
Harshit Bansal 6f0b46d84f create_realm.py: Remove unnecessary imports. 2017-04-04 15:48:03 -07:00
Harshit Bansal ac2172e233 models: Rename RealmAlias model to RealmDomain.
Includes a migration.
2017-04-04 15:48:03 -07:00
Rishi Gupta 30024d0a8f models: Remove Realm.domain. 2017-03-25 19:55:48 -07:00
Rishi Gupta caef5332d5 mailchimp: Fix error in add_users_to_mailing_list.py.
Old behavior is a holdover from development testing.
2017-03-21 16:13:00 -07:00
Rishi Gupta 92dd767519 management commands: Change help text to say realm instead of domain. 2017-03-14 17:17:42 -07:00
Rishi Gupta 0c032adbde create_realm.py: Remove --domain option.
Realm domains can now be managed via Admin Settings in the web app.
2017-03-14 17:17:42 -07:00
Rishi Gupta 11346e50bb management commands: Use Realm.string_id instead of domain in print statements. 2017-03-13 09:44:32 -07:00
Tim Abbott ca50e10569 sessions: Move session deletion code to sessions.py.
This new organization feels more thematically appropriate, and of
course shrinks the amount of stuff unnecessarily in actions.py
2017-03-08 02:48:18 -08:00
Tim Abbott e90df029bf create_realm: Eliminate obsolete deployment code.
Since we're not using this model anymore, we certainly don't need it
when creating realms.
2017-03-06 00:07:45 -08:00
Tim Abbott 8815a598d8 digest: Remove obsolete deployment code.
This system was quite complicated, and never had great semantics.

Eventually, we'll want some other system for gating which server
should generate digest emails for which realm controlled via the
database.
2017-03-06 00:05:21 -08:00
sinwar 6f0564e9f4 python: Fix remaining bare excepts in codebase.
Fixes #2862.
2017-03-05 16:17:04 -08:00
Tim Abbott d95412425a generate_realm_creation_link: Fix line-wrapping of help text. 2017-03-04 19:02:01 -08:00
feorlen 3c954096f6 generate_realm_creation_link: Check for an uninitialized database.
This provides a nice error message to users who miss a step in the
instructions.

Fixes #3672.
2017-03-04 19:01:01 -08:00
Raghav Jajodia a3a03bd6a5 mypy: Added Dict, List and Set imports.
Fixed mypy errors associated with the upgrade.
2017-03-04 14:33:44 -08:00
adnrs96 9eb47f108c Refactor: Change upload_avatar_image to accept two user profiles.
In this commit we just change the upload_avatar_image function to accept
two user_profiles acting_user_profile and target_user_profile. Basically
email param is dropped for a target_user_profile so that avatar's could
be moved lateron to user id based storage.
2017-03-03 18:15:15 -08:00
Umair Khan 87c5ace24d i18n: Use deterministic order in translations.json.
Fixes: #3897
2017-03-03 12:32:17 +05:00
Rishi Gupta 3d07ac0c49 Change timezone-naive datetimes to use timezone.now() where safe to do so.
Change timezone-naive datetimes to use timezone.now() in cases where there
is no change in behavior.
2017-03-01 22:54:28 -08:00
Rishi Gupta c388858e53 Fix timezone errors in ScheduledJob and digest creation filters.
datetime.utcnow() is a timezone-naive datetime. The Django ORM interprets it
in the settings.TIME_ZONE timezone (e.g. 'America/New_York' in the
development server). We perhaps haven't noticed errors yet since with
'America/New_York' all it means is that emails are sent 5 hours early, or a
slightly different set of messages are included in the digest.
2017-03-01 22:54:28 -08:00
Rishi Gupta 562bc6429c Replace datetime.now() with timezone.now() in Django ORM queries.
When you pass a naive datetime to the Django ORM, it uses settings.TIME_ZONE
for the time zone. In the development environment, both settings.TIME_ZONE
and datetime.now() use 'America/New_York', so there is no change in behavior
there. (fromtimestamp with no tz argument uses the same timezone as
datetime.now)

We are soon going to change settings.TIME_ZONE to UTC, so need to remove
naive datetimes from queries to the ORM.
2017-03-01 22:54:28 -08:00
Rishi Gupta 2b2be8120f Change datetime.now(tz=X) to timezone.now().
datetime.now with a timezone set is equivalent to timezone.now() if it's
never being printed out, but the latter is cleaner and more idiomatic.
2017-03-01 22:54:28 -08:00
Tim Abbott cf96b1b873 generate_realm_creation_link: Clean up instructions. 2017-02-21 20:19:16 -08:00
Tim Abbott 32bfebeb7a mypy: Fix inconsistencies in use of *args/**kwargs. 2017-02-18 18:39:44 -08:00
Umair Khan 00f8239563 Clean code of send_test_email command.
* Derive from sendtestemail command of Django.
* Remove unwanted imported.
* Allow test email to admin and managers.
2017-02-10 16:01:43 -08:00
Umair Khan ef0d2a4bb5 logging: Use django.server to filter 200 and 304.
Previously, we were monkey patching the runserver command
in zerver/management/commands/rundjango.py for this.
2017-02-10 15:55:17 -08:00
Harshit Bansal 7d10cbc32b Add RealmAlias.allow_subdomains to model, frontend, and API.
Includes a database migration.

Fixes #1868.
2017-02-08 22:03:27 -08:00
Tim Abbott 12e7be585d i18n: Fix compilemessages processing extra files in production.
In a Zulip production environment, STATIC_ROOT points to the shared
directory that static assets are served from, and so the
compilemessages management command was trying to process every
historical version in there.
2017-02-03 17:17:23 -08:00
Tim Abbott a889346c9f Remove zh_CN old Django name for Chinese. 2017-02-03 14:29:03 -08:00
Tim Abbott 4d3e97f304 actions: Remove unnecessary do_create_stream function. 2017-01-29 22:07:14 -08:00
Tim Abbott 2dc90e8ebd rename_stream: Pass a stream object to do_rename_stream. 2017-01-29 20:27:00 -08:00
Harshit Bansal 06cc306d00 Add stricter domain validation and improve error messages. 2017-01-26 17:24:25 -08:00
Harshit Bansal 5b01694e00 actions.py: Rename do_get_realm_aliases to get_realm_aliases. 2017-01-26 17:24:25 -08:00
Tim Abbott 4e171ce787 lint: Clean up E126 PEP-8 rule. 2017-01-23 22:06:13 -08:00
Tim Abbott de99f48ce7 lint: Clean up E401 PEP-8 rule. 2017-01-23 21:36:39 -08:00
Tim Abbott 9cc83f87fc lint: Clean up E241 PEP-8 rule. 2017-01-23 21:21:14 -08:00
Tim Abbott e9158dd520 lint: Clean up E121 PEP-8 rule. 2017-01-23 21:02:39 -08:00
Tim Abbott bde2da7dfd lint: clean up PEP-8 W391 rule. 2017-01-23 20:39:02 -08:00
Rohith Asrk 560db2d482 send_password_reset_email.py: Migrate to send_mail from EmailMultiAlternatives. 2017-01-19 16:26:31 -08:00
Rohith Asrk 92ed9dd930 deliver_email.py: Migrate to send_mail from EmailMultiAlternatives. 2017-01-19 16:26:24 -08:00
JefftheBest1 a549ed6e65 Removed accommodate typos 2017-01-12 04:53:31 -08:00
Rishi Gupta 38c2dc8790 Change settings.SYSTEM_ONLY_REALMS to use Realm.string_id. 2017-01-06 19:45:08 -08:00
Harshit Bansal ceb636dbd9 Manage allowed domains from admin settings.
Fixes: #1867.
2017-01-06 12:03:31 -08:00
Rishi Gupta 2b0a7fd0ba Rename models.get_realm_by_string_id to get_realm.
Finishes the refactoring started in c1bbd8d. The goal of the refactoring is
to change the argument to get_realm from a Realm.domain to a
Realm.string_id. The steps were

* Add a new function, get_realm_by_string_id.

* Change all calls to get_realm to use get_realm_by_string_id instead.

* Remove get_realm.

* (This commit) Rename get_realm_by_string_id to get_realm.

Part of a larger migration to remove the Realm.domain field entirely.
2017-01-04 17:12:23 -08:00
Rishi Gupta c6e12e74be Change domain to realm_id in bugdown and realm filter dicts and caches. 2017-01-03 16:25:20 -08:00
Rishi Gupta 674f068b92 Script to add all current Zulip users to a mailchimp list. 2016-12-29 16:24:40 -08:00
Rishi Gupta fc188de8e0 bulk_create: Refactor bulk_create_users to take Realm instead of domains.
Previously bulk_create_users would figure out a user's realm from their
email domain. Now require that a realm be passed explicitly.
2016-12-15 13:22:17 -08:00
Tim Abbott 988a2e2d8e generate_invite_links: Use realm host properly. 2016-12-14 10:46:45 -08:00
Tim Abbott 639dc9108d set_default_streams: Fix multi-line help output. 2016-12-14 10:40:04 -08:00
reyha 82e32ad255 Access realm by `string_id` in management commands.
`Realm.string_id` replaces 'Realm.domain'
in the management commands.

Fixes #2325.
2016-12-14 10:38:03 -08:00
reallyly 94b28e0afb pep8: Fix E128 violations.
With some line-wrapping tweaks by tabbott.
2016-12-13 10:50:14 -08:00
Joy Chen bfb6ac5fdb streams: Add default stream description tests and functions.
This includes making the default stream description setting into a
dict.  That is an API change; we'll discuss it in the changelog but it
seems small enough to be OK.

With some small tweaks by tabbott to remove unnecessary backwards
compatibility code for the settings.

Fixes #2427.
2016-12-07 21:12:44 -08:00
anirudhjain75 beaa62cafa mypy: Convert several directories to use typing.Text.
Specifically, these directories are converted: [analytics/, scripts/,
tools/, zerver/management/, zilencer/, zproject/]
2016-12-07 20:51:05 -08:00
nikolay abc2ff4a06 pep8: Fix many rule E128 violations.
[Tweaked by tabbott to adjust some approaches used in wrapping]
2016-12-03 13:33:31 -08:00
bulat22101 a6f91064a2 pep8: Fix E129 violations 2016-12-03 10:56:36 -08:00
bulat22101 adebc75740 pep8: Fix E502 violations 2016-12-03 10:56:36 -08:00
Sidhant Bhavnani 8c0c12c1d9 pep8: Fix E303 violations. 2016-12-02 15:34:11 -08:00
Alex Huang c8ddea16c3 pep8: Fix E122. 2016-12-01 23:16:35 -08:00
Rafid Aslam c5316b4002 lint: Fix E127 pep8 violations.
Fix pep8: E127 continuation line over-indented for visual indent
style issue.
2016-12-01 10:23:55 -08:00
Rafid Aslam 41bd88d5ed pep8: Fix E301 pep8 violations.
Fix "E301: expected (1 or 2) blank line" pep8 violations.
2016-11-29 08:51:44 -08:00
Rafid Aslam 7a2282986a pep8: Fix E225 pep8 violations. 2016-11-28 15:21:15 -08:00
Tim Abbott 3d1bcb05e1 tornado: Move event_queue.py to zerver/tornado/.
Fixes #729.
2016-11-26 22:29:28 -08:00
Tim Abbott 282f74609c tornado: Move socket code to zerver/tornado/. 2016-11-26 22:29:27 -08:00
Tim Abbott b83361bcfc tornado: Extract create_tornado_application helper. 2016-11-26 21:43:13 -08:00
Tim Abbott 9e9066f77a tornado: Extract AsyncDjangoHandler to its own file. 2016-11-26 21:43:11 -08:00
Tim Abbott 8739f4d9f1 tornado: Move tornado_ioloop_logging to new zerver/tornado tree. 2016-11-26 21:24:05 -08:00
Umair Khan 671c45524b Django 1.10: Use new app discovery system.
'django.db.models.loading' is removed due to which both
'django.db.models.get_app' and 'django.db.models.get_models' are removed.

See https://docs.djangoproject.com/en/1.10/releases/1.9/#features-removed-in-1-9
2016-11-26 15:04:20 -08:00
Anders Kaseorg 207cf6302b Always start python via shebang lines.
This is preparation for supporting using Python 3 in production.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-26 14:46:37 -08:00
Vladislav Manchev d7e1e4a2c0 Add initial implementation of custom realm filters.
This PR was abandoned by Vladislav and then substantially modified by
Igor Tokarev and Tim Abbott to complete it and fix a number of bugs.

Fixes #544.
2016-11-17 17:11:25 -08:00
Steve Howell ca43cbc654 logging: Reducing logging in run-dev.py for queue workers.
In dev, we no longer log that individual queue workers were launched.

Instead, in dev (and prod as well), we log a message with the total
count.
2016-11-17 11:12:02 -08:00
Umair Khan 9b0f9bf326 Django 1.10: apply_response_fixes was removed.
Ref: https://code.djangoproject.com/ticket/26052?cversion=0&cnum_hist=2
2016-11-14 16:09:12 -08:00
Umair Khan 5cfd54994a Django 1.10: validate is replaced with check.
See
https://docs.djangoproject.com/en/dev/releases/1.7/#modeladmin-validators
for explanation.
2016-11-14 16:09:12 -08:00
Tim Abbott f6d3c63a01 realm_alias: Access realm alias objects in lower-case. 2016-11-09 16:31:48 -08:00
Umair Khan 682aa1f298 Django 1.10: Use add_argument for options in BaseCommand. 2016-11-04 10:20:23 -07:00
Rishi Gupta 655940acaf create_realm.py: Add basic checks to name and subdomain.
Don't allow the empty string for organization name, and don't pass
do_create_realm a subdomain that's already taken.
2016-11-03 13:59:11 -07:00
Rishi Gupta 1944ac844a RealmAlias: Drop uniqueness constraint when REALMS_HAVE_SUBDOMAINS.
Removes the uniqueness constraint on RealmAlias.domain, and adds a function
can_add_alias that checks for uniqueness conditional on
settings.REALMS_HAVE_SUBDOMAINS.
2016-11-03 13:53:01 -07:00
Rishi Gupta e4334f4735 do_create_realm: Remove domain argument from do_create_realm.
The domain argument was being used in do_create_realm to create a
RealmAlias. Callers are now responsible for creating a RealmAlias
when desired.
2016-11-03 13:53:01 -07:00
Rishi Gupta c0f1b5d9f6 Change Realm defaults to Community defaults.
Previously, we set restrict_to_domain and invite_required differently
depending on whether we were setting up a community or a corporate
realm. Setting restrict_to_domain requires validation on the domain of the
user's email, which is messy in the web realm creation flow, since we
validate the user's email before knowing whether the user intends to set up
a corporate or community realm. The simplest solution is to have the realm
creation flow impose as few restrictions as possible (community defaults),
and then worry about restrict_to_domain etc. after the user is already in.

We set the test suite to explictly use the old defaults, since several of
the tests depend on the old defaults.

This commit adds a database migration.
2016-11-03 13:53:01 -07:00
K.Kanakhin c89a2ffe2f Add raised exception output to development tornado running.
Replaces the default tornado ioloop tracback exception handler in
debug mode to add output for exceptions to the logging.

Fixes #2150.
2016-11-03 11:16:12 -07:00
Rishi Gupta db0e509422 do_create_realm: Replace domain argument with string_id.
Turns string_id into a required argument, and domain into an optional
argument.
2016-11-02 22:46:34 -07:00
Rishi Gupta 9ef8536cc6 models.Realm: Require Realm.string_id to be non-NULL.
Adds a database migration, adds a new string_id argument to the management
realm creation command, and adds a short name field to the web realm
creation form when REALMS_HAVE_SUBDOMAINS is False.
2016-11-02 22:46:34 -07:00
Tim Abbott 9b7a3f040c Remove now-unused /json/get_events endpoint. 2016-10-27 21:34:58 -07:00
Tim Abbott 86e933a4a1 process_queue: Suppress USING_RABBITMQ warnings in test suite. 2016-10-27 12:36:06 -07:00
Steve Howell cba8be6bbb Use bulk_remove_subscriptions() in remove_users_from_stream.
We are deprecating do_remove_subscription.
2016-10-20 22:05:19 -07:00
Steve Howell 98e4452e07 Replace do_add_subscription() in add_users_to_streams. 2016-10-20 22:05:19 -07:00
Tim Abbott 3b0b65c2d0 management: Fix recent argument parsing regressions.
Fixes a regression introduced in
fbdf539bf2.
2016-10-16 20:36:33 -07:00
Laura Hampton fbdf539bf2 Replace optparse with argparse in the several management commands.
Specifically:
* fill_memcached_caches.py
* initialize_voyager_db.py
* logout_all_users.py
2016-10-16 17:08:35 -07:00
Eklavya Sharma b741c7351d email_mirror: Use correct string types. 2016-10-11 11:27:35 -07:00
Umair Khan d7efdf050c Ignore docs directory while creating translation strings.
Fixes: #1941.
2016-10-10 08:23:56 -07:00
Umair Khan a9343a65d5 Don't localize javascript comments.
Fixes: #1934.
2016-10-10 08:21:02 -07:00
Brock Whittaker 3efe3601b8 Allow for templates/settings to be read and compiled.
This allows for the handlebars templates in the template/settings
subdirectory to be found and compiled.
2016-10-05 22:24:22 -07:00
Rishi Gupta 777fcaa6a0 Add new organization type field to Realm objects.
Adds a new field org_type to Realm.  Defaults for restricted_to_domain
and invite_required are now controlled by org_type at time of realm
creation (see zerver.lib.actions.do_create_realm), rather than at the
database level.  Note that the backend defaults are all
org_type=corporate, since that matches the current assumptions in the
codebase, whereas the frontend default is org_type=community, since if
a user isn't sure they probably want community.

Since we will likely in the future enable/disable various
administrative features based on whether an organization is corporate
or community, we discuss those issues in the realm creation form.
Before we actually implement any such features, we'll want to make
sure users understand what type of organization they are a member of.

Choice of org_type (via radio button) has been added to the realm
creation flow and the realm creation management command, and the
open-realm option removed.

The database defaults have not been changed, which allows our testing code
to work unchanged.

[includes some HTML/CSS work by Brock Whittaker to make it look nice]
2016-10-05 17:01:46 -07:00
Tomasz Kolek dbeab6aa6f Optimize checks of test database state by moving into Python.
Previously, the generate-fixtures shell script by called into Django
multiple times in order to check whether the database was in a
reasonable state.  Since there's a lot of overhead to starting up
Django, this resulted in `test-backend` and `test-js-with-casper`
being quite slow to run a single small test (2.8s or so) even on my
very fast laptop.

We fix this is by moving the checks into a new Python library, so that
we can avoid paying the Django startup overhead 3 times unnecessarily.
The result saves about 1.2s (~40%) from the time required to run a
single backend test.

Fixes #1221.
2016-10-05 10:40:19 -07:00
Tim Abbott 22fd7ba02a avatar: Move avatar hash computations to their own file. 2016-10-02 21:19:10 -07:00
Sahil Dua 058587da77 Remove extra new lines at the ends of Zulip authoried files.
Fixes #1627.

[tweaked by tabbott to avoid patching third-party modules, for now]
2016-09-26 21:05:24 -07:00
Umair Khan b4eb3a55be Annotate zerver/management/commands/realm_emoji.py. 2016-09-12 08:10:50 -07:00
Umair Khan f8c904e611 Annotate zerver/management/commands/deliver_email.py. 2016-09-12 08:10:50 -07:00
Umair Khan c096473e06 Annotate zerver/management/commands/check_redis.py. 2016-09-12 08:10:50 -07:00
Steve Howell 64cb5d0778 Annotate commands/export.py. 2016-09-11 17:15:35 -07:00
Steve Howell 52c6efa0a8 Annotate commands/export_single_user.py. 2016-09-11 17:15:35 -07:00
Steve Howell bb6fcaa4e7 Annotate commands/export_usermessage_batch.py. 2016-09-11 17:15:35 -07:00
Steve Howell 85e30a6d35 Annotate commands/import.py. 2016-09-11 17:15:35 -07:00
Tim Abbott 8fbbb3a471 Annotate send_password_reset_email.py. 2016-09-11 17:09:27 -07:00
Vishnu Ks a7ead9e99d settings: Eliminate ADMIN_DOMAIN for creating initial realm.
We now use `./manage.py generate_realm_creation_link` as the flow flow
for creating one's first realm.
2016-08-25 09:37:33 -07:00
Tim Abbott 3bdbc33e94 initialize_voyager_db: Fix zulip.com hardcoding. 2016-08-22 21:36:15 -07:00
Tim Abbott e1e384acc0 realm_emoji: Fix management command usage output. 2016-08-19 11:59:38 -07:00
Taranjeet Singh 57253c6ba3 makemessages.py: Fix line with length greater than 120.
This updates file zerver/management/commands/makemessages.py.
2016-08-19 11:56:43 -07:00
Tim Abbott 399f4f8870 Add management command to send password reset emails.
This is a convenient tool to have around.

We require an unusual argument value of "YES" to send to everyone on
the server, since that's something one should do with a great deal of
care.
2016-08-18 20:12:36 -07:00
Tim Abbott 3bae3cd54d settings: Add option to disable email deliverer. 2016-08-12 19:24:45 -07:00
Steve Howell 90494cfec2 export: Note user-owned bot limitations. 2016-08-12 15:10:49 -07:00
Tim Abbott 386c6c2a31 settings: Refactor SYSTEM_ONLY_REALMS configuration. 2016-08-12 12:12:53 -07:00
Steve Howell b0e6d20321 export: Write stats.txt for `./manage.py export <realm>`. 2016-08-12 09:06:10 -07:00
Steve Howell 9582a83d10 export: Fix usage in export_single_user command. 2016-08-11 15:21:30 -07:00
Steve Howell 1d1f36c0b8 export: Always use subprocesses to export UserMessage.
We no longer have an in-process code path to export
UserMessage rows.  We want to only maintain the
subprocess code, which we'll always use in production,
and which will work fine in dev.
2016-08-10 14:16:17 -07:00
Umair Khan e1090118c5 Percent translated should be an integer. 2016-08-09 16:19:07 -07:00
Tim Abbott 6264ff7039 Add new Zulip realm import/export tool.
The previous export tool would only work properly for small realms,
and was missing a number of important features:
* Export of avatars and uploads from S3
* Export of presence data, activity data, etc.
* Faithful export/import of timestamps
* Parallel export of messages
* Not OOM killing for large realms

The new tool runs as a pair of documented management commands, and
solves all of those problems.

Also we add a new management command for exporting the data of an
individual user.
2016-08-08 14:58:18 -07:00
Tim Abbott 1e54897ca7 process_queue: Add missing type annotation. 2016-08-04 15:57:03 -07:00
Tim Abbott 641aa08721 compilemessages: Add mypy annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott a787dee48b makemessages: Add missing annotations and fix string types. 2016-08-04 15:57:03 -07:00
Tim Abbott 92062b9526 process_queue: Add missing annotation. 2016-08-04 15:57:02 -07:00
Umair Khan 2edb79776e Calculate translation percentages.
Fixes: #1397
2016-07-29 11:58:41 -07:00
Eklavya Sharma 3b3b5c7c16 Rename management command email-mirror to email_mirror.
All other zulip management command names have underscores, so
rename email-mirror to email_mirror.

This will also make it possible to import this module, which will
help in writing tests for it.
2016-07-28 14:52:09 -07:00
Eklavya Sharma 6e9bc44123 Re-implement email-mirror using imaplib.
Switch from twisted to imaplib to gain python 3 compatibility and
make code easier to understand.
2016-07-28 14:52:09 -07:00
Tim Abbott 19b860ceec Rename local_settings.py symlink to prod_settings.py. 2016-07-19 20:59:59 -07:00
Tim Abbott 1f4c9eefe4 settings: Extract settings.ZILENCER_ENABLED. 2016-07-19 15:28:41 -07:00
Tim Abbott 3c591aa724 compilemessages: Don't try to include zh-CN and zh_CN in language options. 2016-07-18 15:02:43 -07:00
Tim Abbott 55a98a41d8 compilemessages: Improve error handling for unknown locale. 2016-07-18 15:02:43 -07:00
Taranjeet Singh 24ba060421 Fix line with length greater than 120 in deliver_email.py. 2016-07-18 14:34:34 -07:00
Umair Khan b546391f0b Move locale to static/locale 2016-07-18 13:59:07 -07:00
Umair Khan cdf2664030 `makemessages` command should not remove plurals. 2016-07-18 13:59:07 -07:00
Eklavya Sharma e10caf0b65 Gracefully give up when using email-mirror on python 3.
Running `./manage.py email-mirror` used to fail on python 3
because twisted.mail.imap4 is not python 3 compatible.
Display a message informing the user that email-mirror is not
available on python 3 instead of failing with a traceback.

Also add tools/test-management to py3-backend.
2016-07-18 09:57:28 -07:00
Eklavya Sharma da36947400 Change unbuffering strategy in runtornado.py.
runtornado unbuffers its output using
sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0).
This is not python 3 compatible since we can't specify
buffering on a text stream in python 3.  So use the '-u'
option of python when calling runtornado.py to make output
unbuffered.
2016-07-17 10:31:15 -07:00
Umair Khan a07a810a2e Fix regex for capturing frontned strings.
Fixes: #1155
2016-07-15 14:48:05 +05:00
Taranjeet a8a4caf2c0 zerver: Fix lines with length greater than 120. 2016-07-08 11:41:43 -07:00
Eklavya Sharma 83640ed0cd runtornado.py: Ignore due to incorrect stubs. 2016-07-07 10:09:35 -07:00
Eklavya Sharma 6fd8906358 rename_stream.py: Fix broken code.
* get_realm returns None if no matching realm is present, but
  create_stream.py assumed it raises Realm.DoesNotExist.
* encoded/decode strings properly.
2016-07-07 10:08:29 -07:00
Eklavya Sharma c679c180f5 enqueue_file.py: Add type hint for mypy. 2016-07-07 10:07:51 -07:00
Eklavya Sharma 896c18a57b email-mirror.py: Make it pass on mypy in python 3.
* Replace filter by list comprehension.

* Add '# type: ignore' to statements which use attributes from
  modeule `posix`, since stubs for posix are missing on python 3.
2016-07-07 10:07:28 -07:00
Eklavya Sharma 17cb6e00bd create_stream.py: Fix broken code.
* get_realm returns None if no matching realm is present, but
  create_stream.py assumed it raises Realm.DoesNotExist.
* encoded/decode strings properly.
2016-07-07 10:06:39 -07:00
Umair Khan 47fb293f8a i18n: Add English locale to language list if missing.
The English locale data is only present if the user has run `manage.py
makemigrations` on the system.
2016-07-06 11:34:28 -07:00
Umair Khan 035fceb814 Add dynamically loaded language dropdown. 2016-07-04 11:56:02 -07:00
Vishnu Ks 1cbd39b768 Unique link generator for realm creation. 2016-06-30 15:52:41 -07:00
Tim Abbott e72f41cdec Remove old prototype data import/export tool.
This prototype from Dropbox Hack Week turned out to be too inefficient
to be used for realms with any significant amount of history, so we're
removing it.

It will be replaced by https://github.com/zulip/zulip/pull/673.
2016-06-27 13:47:08 -07:00
rahuldeve a3745178e5 Use django.utils.autoreload to restart queue workers at code change.
Fixes #621, #1045.
2016-06-26 20:12:11 -07:00
Umair Khan 134b165b1a Fix frontend translation string bug.
The translation string which contain \n are not captured
within {{#tr}}{{/tr}} blocks. Fix is to escape the slash.
2016-06-20 11:31:28 -07:00
Vishnu Ks 8213ca135a Move default_stream list to settings. 2016-06-17 16:01:38 -07:00
Umair Khan 6fcfed8d9e Update regexes to capture translation strings.
Strings are captured from:
- i18n.t
- compose_error
- placeholder
2016-06-13 09:03:56 -07:00
Eklavya Sharma d169cc5376 zerver/lib/bulk_create.py: Fix string annotations.
Change string type from `str` to `text_type` where required.
Also fix clashing annotations.
2016-06-13 10:08:14 +05:30
Tim Abbott d9d0515d3b Add mypy bug number for check_redis type: ignore. 2016-06-04 16:11:18 -07:00
medullaskyline 7c2c7fb31c Annotate zerver/lib/bulk_create.py. 2016-06-04 15:51:05 -07:00
Tim Abbott a1a27b1789 Annotate most Zulip management commands. 2016-06-04 10:12:06 -07:00