Commit Graph

526 Commits

Author SHA1 Message Date
Vishnu Ks 3a2edc66f1 management: Use get_system_bot in initialize_voyager_db. 2017-08-08 14:13:19 -07:00
Vishnu Ks f45b80aafe management: Remove unused imports from initialize_voyager. 2017-08-08 14:13:19 -07:00
Vishnu Ks 05af206392 management: Remove unused imports from import command. 2017-08-08 14:13:19 -07:00
Vishnu Ks f58efdc3cb management: Remove unused imports from export_usermessage_batch. 2017-08-08 14:13:19 -07:00
Vishnu Ks 28e76c990a management: Remove unused imports from email_mirror command. 2017-08-08 14:13:19 -07:00
Vishnu Ks 6687b236bf management: Remove unused imports from deliver_email command. 2017-08-08 14:13:19 -07:00
Vishnu Ks b8dacf9bf6 management: Remove unused import from checkconfig command. 2017-08-08 14:13:19 -07:00
Vishnu Ks 5ce0d85ccf management: Remove unused import from check_redis command. 2017-08-08 14:13:19 -07:00
Vishnu Ks 19f8da4995 management: Use self.get_realm in export command. 2017-08-08 14:13:19 -07:00
Vishnu Ks 4e9dff09fd management: Use self.get_realm in dump_messages. 2017-08-08 14:13:19 -07:00
Vishnu Ks bc48c68c09 management: Use self.get_realm in reactivate_realm command. 2017-08-08 14:13:19 -07:00
Vishnu Ks 279af33ea2 management: Use self.get_realm in deactivate_realm. 2017-08-08 14:13:19 -07:00
Vishnu Ks 349d604ad5 management: Use self.get_realm in create_user. 2017-08-08 14:13:19 -07:00
Rishi Gupta ddbf8b9c3c management: Remove create_realm.py.
This no longer does the correct thing (in terms of onboarding emails,
default streams, etc), and is tempting for new server admins to use.

Once we remove it we'll also have the invariant that we can't have a realm
without a user, which will simplify accounts_register a bit.
2017-08-05 10:17:14 -07:00
Vishnu Ks e7f5da83ac management: Create list realms command.
Fixes #5917.
2017-08-04 14:25:03 -07:00
Steve Howell bc6543a236 Use do_mark_all_as_read() in bankrupt_users command. 2017-08-04 14:10:46 -07:00
Steve Howell c5e4ce5278 Optimize show_unreads --fix approach.
This now breaks the process of cleaning up unread counts for
non-active streams into a three step process.

This allows us to use our unread message flags index, at least
in testing on dev.  Here is the relevant excerpt from explain
analyze:

    Bitmap Index Scan on zerver_usermessage_unread_message_id
2017-08-04 11:04:43 -07:00
Greg Price cc30226af0 deliver_email: When disabled, sleep forever rather than exit.
This makes supervisor see the service as cheerfully running
and let it alone, rather than constantly retry starting it.
Because the crash/restart loop means repeatedly spending a
couple of seconds loading Django and the app, separated by
brief periods while supervisor notices the crash and acts
on it, it was actually consuming about 30-50% CPU on the
zulipchat.com staging server.
2017-08-03 16:28:19 -07:00
Steve Howell e994ff36b5 Add option to fix unread messages for inactive streams. 2017-08-03 13:17:04 -07:00
Steve Howell dad1c9326e Add show_unreads management command.
This should help us troubleshoot anomalies with old unread
messages.
2017-08-03 13:17:04 -07:00
neiljp (Neil Pilgrim) 1c7d0c932a mypy: Ensure type of trim_func is consistent in check_redis.py. 2017-08-03 11:03:14 -07:00
Umair Khan 9e33917d25 rate_limiter: Upgrade max_api_calls to generic API. 2017-08-02 18:01:39 -07:00
Umair Khan ca9ac1948e rate_limiter: Upgrade max_api_window to generic API. 2017-08-02 18:01:39 -07:00
Umair Khan 9c08b62448 rate_limiter: Upgrade unblock_user to generic API.
Change name to unblock_access as well.
2017-08-02 18:01:39 -07:00
Umair Khan 23be0875a1 rate_limiter: Upgrade block_user to generic API.
Change name to block_access as well.
2017-08-02 18:01:39 -07:00
rht ca960da58f create_user: Receive password input from a file instead of shell arg.
This is to be used for the case of container orchestration instead of
shell arg to prevent snooping by any user account on the server via `ps
-ef` or any superuser with read access to the user\'s bash history.
2017-07-27 18:08:52 -07:00
Umair Khan 758dbec9e2 tornado: Move setup_tornado_rabbitmq to application.py 2017-07-21 09:55:25 +05:00
Tim Abbott 240f37e8c4 management: Add script to merge two streams.
This is intended to be used occasionally when there's a really good
reason to do so.  While it does work, it hasn't been extensively tested.
2017-07-18 21:08:24 -07:00
Rishi Gupta 0f4b71b766 confirmation: Liberate get_link_for_object from ConfirmationManager. 2017-07-17 23:18:47 -07:00
Rishi Gupta aa845e7f60 models: Replace ScheduledJob with ScheduledEmail.
ScheduledJob was written for much more generality than it ended up being
used for. Currently it is used by send_future_email, and nothing
else. Tailoring the model to emails in particular will make it easier to do
things like selectively clear emails when people unsubscribe from particular
email types, or seamlessly handle using the same email on multiple realms.
2017-07-17 16:05:38 -07:00
Rishi Gupta 898269bbac email: Change send_email to raise exception on failure.
More in line with how we do error handling in the rest of Zulip.
2017-07-16 16:56:39 -07:00
Rishi Gupta 3d24d12ba1 emails: Change reset password emails to use to_user_id. 2017-07-16 16:56:39 -07:00
Rishi Gupta 154d37afd2 emails: Add to_user_id argument to send_email.
Both the queue processor and ScheduledJob emails need to sometimes pass a
to_user_id and sometimes pass a to_email, and it's more convenient to just
have one function that they can call that can handle either.

Also removes the now redundant send_email_to_user.
2017-07-16 16:56:39 -07:00
James Rowan 69f3ca7870 emails: Make password reset emails come from 'Zulip Account Security.' 2017-07-13 14:50:36 -07:00
Vishnu Ks 48617c5915 send_password_reset_email: Replace get_user_profile_by_email with get_user. 2017-07-11 11:48:21 -07:00
Vishnu Ks aca4d18f23 generate_invite_links: Replace get_user_profile_by_email with get_user.
The extra refactoring is because the realm is made a compulsary
argument.
2017-07-11 11:48:21 -07:00
Umair Khan c31beec9e6 makemessages: Support only Django version >= 1.11. 2017-07-10 11:32:10 -07:00
Vishnu Ks 8dd2268c3b set_message_flags: Replace get_user_profile_by_email with get_user. 2017-07-08 14:40:55 -07:00
Vishnu Ks 0e76d95b9f export_single_user: Replace get_user_profile_by_email with get_user. 2017-07-08 14:38:45 -07:00
Vishnu Ks afc41dfb6b rate_limit: Replace get_user_profile_by_email with get_user. 2017-07-08 14:38:45 -07:00
Vishnu Ks 1ce6b8db8c remove_users_from_stream: Replace get_user_profile_by_email with get_user. 2017-07-08 14:38:45 -07:00
Vishnu Ks 038f2d669a turn_off_digests: Replace get_user_profile_by_email with get_user. 2017-07-08 14:38:45 -07:00
Vishnu Ks fdf3df4154 bulk_change_user_name: Replace get_user_profile_by_email with get_user. 2017-07-08 14:38:45 -07:00
Vishnu Ks 3b6d5fd80e bankrupt_users: Replace get_user_profile_by_email with get_user. 2017-07-08 14:38:45 -07:00
Vishnu Ks a5d2576801 deactivate_users: Replace get_user_profile_by_email with get_user. 2017-07-08 14:38:45 -07:00
Tim Abbott b01e6d750f management: Remove obsolete update_permissions management command.
This was part of the old Django/guardian-based permissions system.
2017-07-08 14:37:25 -07:00
Tim Abbott 3395f38bf7 management: Remove obsolete gravatar_to_user_avatar management command.
This was used for a 1-time migration in 2013, and definitely isn't
needed anymore.
2017-07-08 14:35:53 -07:00
Vishnu Ks 36972f611c add_users_to_streams: Remove unused imports. 2017-07-08 14:35:42 -07:00
Vishnu Ks 4340b94a2d create_user: Separate password help into multiple lines.
This line is too long and ./tools/lint --pep8 is failing because
of this.
2017-07-08 14:18:49 -07:00
rht 7a9de7766b Add password argument to `manage.py create_user`.
Tweaked by tabbott to provide clear --help output recommending against
setting passwords this way.
2017-07-08 10:59:02 -07:00
Rishi Gupta b3654c7b7e confirmation: Remove vestiges of the MitUser table.
The MitUser table was removed in df525ad.

confirm_mituser.html could have been accessed through the last few lines of
confirmation/views.py:
    templates.insert(0, 'confirmation/confirm_%s.html'
                     % (obj._meta.model_name,))

The commit message on df525ad suggests there was another way
confirm_mituser.html could have been called, but I don't currently see
evidence for it in the code.
2017-07-07 18:53:00 -07:00
Rishi Gupta 8fed9eeb75 confirmation: Make host a required argument in get_link_for_object.
Removes some lines of test from test_email_change.py. The relevant code path
was never utilized by the code itself, just by the tests.
2017-07-07 18:53:00 -07:00
Greg Price 207bf2e1fb i18n: Recognize a little more Handlebars syntax in makemessages.
Usually we write translation expressions as `{{t ... }}`, but `{{ t ... }}`
is equally valid as far as Handlebars is concerned, and it matches how we
usually write simple variable substitutions, as `{{ ... }}`.  So occasionally
someone writes `{{ t ... }}`; currently there are two examples of this
in the codebase, in `settings/bot-{settings,list-admin}.handlebars`.

Probably it'd be good to pick a style and enforce it uniformly, but
until we do, the other style shouldn't break translation.
2017-07-07 17:02:39 -07:00
Umair Khan 0e8231d0f1 process_queue: Recover gracefully after PostgreSQL restart.
- For threaded workers:
Django's autoreloader catches SIGQUIT(3) to reload the program. If
a process being watched by autoreloader exits with status code 3,
reloader will restart the process. To reload, we send SIGUSR1(10)
signal from consumers to a handler in process_queue which then
exits with status code 3.

- For single worker per process:
Catch the SIGUSR1 and quit; supervisorctl will restart the worker
automatically.

Fixes #5512
2017-07-07 16:33:15 -07:00
Vaida d5517bae36 Delete the old zulip.com "referrals" system.
This system hasn't been in active use for several years, and had some 
problems with it's design.  So it makes sense to just remove it to declutter
the codebase.

Fixes #5655.
2017-07-07 14:59:18 -07:00
Tim Abbott dbabe3fd37 add_users_to_streams: Fix check for all_users option. 2017-07-07 13:57:57 -07:00
Tim Abbott 982667acf5 management: Add new library for specifying realms.
This new library is intended to make it easy for management commands
to access a realm or a user in a realm without having to duplicate any
of the annoying parsing/extraction code.
2017-07-07 13:57:57 -07:00
Rishi Gupta a26703109e settings: Change all uses of ZULIP_ADMINISTRATOR to FromAddress.SUPPORT.
Make it less likely that further development will break compatibility with
ZULIP_ADMINISTRATORs of the form "name <email>".

Note that the suggested value for this setting has been
'zulip-admin@example.com' for a while, so hopefully this commit causes no
change for most installations.
2017-07-05 15:33:01 -07:00
Rishi Gupta 577e201df0 emails: Replace DEFAULT_FROM_EMAIL with ZULIP_ADMINISTRATOR.
We're getting rid of DEFAULT_FROM_EMAIL from prod_settings_template, since
we only really need one of {DEFAULT_FROM_EMAIL, ZULIP_ADMINISTRATOR}.
2017-06-29 17:54:33 -07:00
Rishi Gupta 36cd122905 models: Change default org_type to CORPORATE.
Once we implement org_type-specific features, it'll be easy to change a
corporate realm to a community realm, but hard to go the other way. The main
difference (the main thing that makes migrating from a community realm to a
corporate realm hard) is that you'd have to make everyone sign another terms
of service.
2017-06-29 15:14:58 -07:00
Rishi Gupta 78790ce825 send_password_reset_email: Remove spurious send_mail import.
Make grepping for and otherwise understanding our email pathways slightly
easier.
2017-06-29 10:51:12 -07:00
Rishi Gupta a56d703bdc i18n: Remove translation tags in management commands. 2017-06-25 11:28:28 -04:00
Umair Khan b74cc576ae Django 1.11: Make makemessages command compatible.
Now REs have moved to template module. This commit adds a condition
to use trans_real module if the Django version is less than 1.11 else
use template module.
2017-06-13 15:14:05 -07:00
Harshit Bansal e30e244b29 emoji: Moved `unified_reactions.json` file.
Moved `unified_reactions.json` file from `tools/setup/emoji` to
`zerver/management/data` so that it can be used in production
environment.
2017-06-09 16:38:58 -07:00
Rishi Gupta 769c5ab105 emails: Send password reset emails through zerver.lib.send_email.
Previously, the password reset email behaved differently from all the other
email Zulip sends.
2017-06-06 23:22:22 -07:00
rht 940cf9db3b Run queue processors multithreaded in production if system memory <3.5GB.
While running queue processors multithreaded will limit the
performance available to very small systems, it's easy to fix that by
adding more RAM, and previously, Zulip didn't work on such systems at
all, so this is unambiguously an improvement there.

Fixes #32.
Fixes #34.

(Commit message expanded significantly by tabbott.)
2017-06-03 12:19:58 -07:00
Aditya Bansal 15ea059aad pep8: Add compliance with rule E261 makemessages.py. 2017-05-31 17:07:15 -07:00
Aditya Bansal 717e5ae393 pep8: Add compliance with rule E261 commands/create_realm.py. 2017-05-31 17:07:15 -07:00
Maxim Averin 685fb16c39 Switch change_full_name to use RealmAuditLog.
This requires adding an `acting_user` parameter to the
`do_change_bot_owner` function.
2017-05-29 15:22:08 -07:00
Elliott Jin 5b4d2832fc bots: Add management command for making outgoing webhook bot. 2017-05-25 09:45:54 -07:00
Tim Abbott e9968a7a09 change_user_email: Use new get_user_for_mgmt function. 2017-05-24 15:29:59 -07:00
Matt Long 19363b2b77 notification_settings: Refactor notification preference settings.
Previously, all notification preference setting had a dedicated test
and setter. Now, all are handled through a modular function using the
property_types framework.
2017-05-23 14:47:46 -07:00
Aditya Bansal 8a6617e304 pep8: Add compliance with rule E261 to set_default_streams.py. 2017-05-07 23:21:50 -07:00
Aditya Bansal 7271fc8a7b pep8: Add compliance with rule E261 to import.py. 2017-05-07 23:21:50 -07:00
Aditya Bansal d8db632d21 pep8: Add compliance with rule E261 to enqueue_file.py. 2017-05-07 23:21:50 -07:00
Aditya Bansal 6a53c53588 pep8: Add compliance with rule E261 to email_mirror.py. 2017-05-07 23:21:50 -07:00
Aditya Bansal f1b2f10574 pep8: Add compliance with rule E261 to dump_messages.py. 2017-05-07 23:21:50 -07:00
Rishi Gupta 6b9f25a58a Remove duplicated code between send_future_email and send_email pathway.
Note that this change restricts the context for any template that uses
send_future_email to be jsonable.
2017-05-05 14:38:25 -07:00
Rishi Gupta 68c172192b email.py: Change recipients argument of send_future_email to to_email. 2017-05-05 14:38:25 -07:00
Rishi Gupta d70e09b41d notifications: Change sender arg of send_future_email to from_email.
This commit is a step towards the goal of replacing most of the
send_future_email pathway with a call to send_email.

Note that this commit changes the default value of sender from "Zulip
<NOREPLY_EMAIL_ADDRESS>" to "NOREPLY_EMAIL_ADDRESS". NOREPLY_EMAIL_ADDRESS
will soon be changed to have the Zulip in front.
2017-05-05 14:38:25 -07:00
Rishi Gupta e46cbaffa2 email: Remove Mandrill pathways and dependency.
Everything it was doing (send_future_email) can now be done using
ScheduledJob.
2017-05-05 14:38:23 -07:00
Rishi Gupta 925ee8c0f1 Add a send_email function that takes a template_prefix and context.
This commit replaces all uses of django.core.mail.send_mail with send_email,
other than in the password reset flow, since that code looks like it is just
a patch to Django's password reset code.

The send_email function is in a new file, since putting it in
zerver.lib.notifications would create an import loop with confirmation.models.

send_future_email will soon be moved into email.py as well.
2017-05-05 14:20:32 -07:00
Rishi Gupta 92dd76822f email: Move and rename password reset templates. 2017-05-03 23:26:14 -07:00
Tim Abbott 4a67ba241e realm_domains: Restructure library for getting realm domains.
* Remove duplicate list_of_domains_for_realm of get_realm_domains.
* Move get_realm_domains from actions.py.
2017-04-28 21:11:56 -07:00
K.Kanakhin e3e52e7284 email-mirror: Move postfix email mirror integration to separate script.
This fixes a performance problem where we were previously starting up
a full Django process (~0.7s even on a fast machine) every time a new
email came in, potentially allowing users to accidentally DoS a Zulip
server.  Now, we just post over HTTPS, allowing the existing thread
pool support to do its job.

- Add script wrapper to communicate postfix pipe with django web server
  over HTTP(S). It uses shared_secret authentication mode.
- Add django view to process messages from email mirror server.
- Clean management command `email-mirror`. Left just functional
  for cron email processing.
- Add routes for new tornado view.
- Change pipe script in master process postfix config template
  based on updated script.
- Add tests.

Tweaked by tabbott to adjust the directory and set better defaults.

Fixes #2421.
2017-04-24 21:24:23 -07:00
hackerkid b2504084ab Replace timezone.now with timezone_now. 2017-04-16 12:28:56 -07:00
hackerkid 55c3d12078 Replace timezone.utc with timezone_utc. 2017-04-16 12:28:56 -07:00
Yago González e710110a9e i18n: Automatically strip Handlebars strings.
Some Handlebars strings contained whitespaces characters at their ends.
With this, such characters are removed, as well as multiple spaces
(like the ones produced by code indentation).

This also includes a couple of fixes that removes spaces that were
intentionally placed before/after the string to translate.
2017-04-14 17:37:25 -07:00
Yago González 31c92cdcbc refactor: Keep sub replacements' format uniform. 2017-04-14 17:37:25 -07:00
Umair Khan 9dec5306ce makemessages.py: Handle unicode strings.
This commit applies to Python 2. It seems that sorted()
cannot handle non-ascii character unless the strings are
marked as Unicode.
2017-04-11 22:32:06 -07:00
Abhijeet Kaur 8f88b045a4 Rename "Administration" to "Organization" in the settings UI.
This better sets expectatations for the fact that in Zulip, the
Organization settings UI is available read-only to non-administrator
users.

Tweaked by tabbott to update some additional references.
2017-04-07 17:32:56 -07:00
Umair Khan 846e8686c4 makemessages: Exclude hidden files.
Fixes #3981.
2017-04-05 11:09:32 -07:00
Harshit Bansal 664b23c495 commands: Rename `realm_alias` command to `realm_domain`. 2017-04-04 15:48:03 -07:00
Harshit Bansal 362ee482c0 commands/realm_alias.py: Remove any occurrences of `alias`. 2017-04-04 15:48:03 -07:00
Harshit Bansal 983225612d actions.py: Rename `get_realm_aliases()` to `get_realm_domains()`. 2017-04-04 15:48:03 -07:00
Harshit Bansal 92c512d679 zerver/models.py: Rename can_add_alias() to can_add_realm_domain(). 2017-04-04 15:48:03 -07:00
Harshit Bansal 6f0b46d84f create_realm.py: Remove unnecessary imports. 2017-04-04 15:48:03 -07:00
Harshit Bansal ac2172e233 models: Rename RealmAlias model to RealmDomain.
Includes a migration.
2017-04-04 15:48:03 -07:00
Rishi Gupta 30024d0a8f models: Remove Realm.domain. 2017-03-25 19:55:48 -07:00
Rishi Gupta caef5332d5 mailchimp: Fix error in add_users_to_mailing_list.py.
Old behavior is a holdover from development testing.
2017-03-21 16:13:00 -07:00