Commit Graph

503 Commits

Author SHA1 Message Date
Umair Khan 9c08b62448 rate_limiter: Upgrade unblock_user to generic API.
Change name to unblock_access as well.
2017-08-02 18:01:39 -07:00
Umair Khan 23be0875a1 rate_limiter: Upgrade block_user to generic API.
Change name to block_access as well.
2017-08-02 18:01:39 -07:00
rht ca960da58f create_user: Receive password input from a file instead of shell arg.
This is to be used for the case of container orchestration instead of
shell arg to prevent snooping by any user account on the server via `ps
-ef` or any superuser with read access to the user\'s bash history.
2017-07-27 18:08:52 -07:00
Umair Khan 758dbec9e2 tornado: Move setup_tornado_rabbitmq to application.py 2017-07-21 09:55:25 +05:00
Tim Abbott 240f37e8c4 management: Add script to merge two streams.
This is intended to be used occasionally when there's a really good
reason to do so.  While it does work, it hasn't been extensively tested.
2017-07-18 21:08:24 -07:00
Rishi Gupta 0f4b71b766 confirmation: Liberate get_link_for_object from ConfirmationManager. 2017-07-17 23:18:47 -07:00
Rishi Gupta aa845e7f60 models: Replace ScheduledJob with ScheduledEmail.
ScheduledJob was written for much more generality than it ended up being
used for. Currently it is used by send_future_email, and nothing
else. Tailoring the model to emails in particular will make it easier to do
things like selectively clear emails when people unsubscribe from particular
email types, or seamlessly handle using the same email on multiple realms.
2017-07-17 16:05:38 -07:00
Rishi Gupta 898269bbac email: Change send_email to raise exception on failure.
More in line with how we do error handling in the rest of Zulip.
2017-07-16 16:56:39 -07:00
Rishi Gupta 3d24d12ba1 emails: Change reset password emails to use to_user_id. 2017-07-16 16:56:39 -07:00
Rishi Gupta 154d37afd2 emails: Add to_user_id argument to send_email.
Both the queue processor and ScheduledJob emails need to sometimes pass a
to_user_id and sometimes pass a to_email, and it's more convenient to just
have one function that they can call that can handle either.

Also removes the now redundant send_email_to_user.
2017-07-16 16:56:39 -07:00
James Rowan 69f3ca7870 emails: Make password reset emails come from 'Zulip Account Security.' 2017-07-13 14:50:36 -07:00
Vishnu Ks 48617c5915 send_password_reset_email: Replace get_user_profile_by_email with get_user. 2017-07-11 11:48:21 -07:00
Vishnu Ks aca4d18f23 generate_invite_links: Replace get_user_profile_by_email with get_user.
The extra refactoring is because the realm is made a compulsary
argument.
2017-07-11 11:48:21 -07:00
Umair Khan c31beec9e6 makemessages: Support only Django version >= 1.11. 2017-07-10 11:32:10 -07:00
Vishnu Ks 8dd2268c3b set_message_flags: Replace get_user_profile_by_email with get_user. 2017-07-08 14:40:55 -07:00
Vishnu Ks 0e76d95b9f export_single_user: Replace get_user_profile_by_email with get_user. 2017-07-08 14:38:45 -07:00
Vishnu Ks afc41dfb6b rate_limit: Replace get_user_profile_by_email with get_user. 2017-07-08 14:38:45 -07:00
Vishnu Ks 1ce6b8db8c remove_users_from_stream: Replace get_user_profile_by_email with get_user. 2017-07-08 14:38:45 -07:00
Vishnu Ks 038f2d669a turn_off_digests: Replace get_user_profile_by_email with get_user. 2017-07-08 14:38:45 -07:00
Vishnu Ks fdf3df4154 bulk_change_user_name: Replace get_user_profile_by_email with get_user. 2017-07-08 14:38:45 -07:00
Vishnu Ks 3b6d5fd80e bankrupt_users: Replace get_user_profile_by_email with get_user. 2017-07-08 14:38:45 -07:00
Vishnu Ks a5d2576801 deactivate_users: Replace get_user_profile_by_email with get_user. 2017-07-08 14:38:45 -07:00
Tim Abbott b01e6d750f management: Remove obsolete update_permissions management command.
This was part of the old Django/guardian-based permissions system.
2017-07-08 14:37:25 -07:00
Tim Abbott 3395f38bf7 management: Remove obsolete gravatar_to_user_avatar management command.
This was used for a 1-time migration in 2013, and definitely isn't
needed anymore.
2017-07-08 14:35:53 -07:00
Vishnu Ks 36972f611c add_users_to_streams: Remove unused imports. 2017-07-08 14:35:42 -07:00
Vishnu Ks 4340b94a2d create_user: Separate password help into multiple lines.
This line is too long and ./tools/lint --pep8 is failing because
of this.
2017-07-08 14:18:49 -07:00
rht 7a9de7766b Add password argument to `manage.py create_user`.
Tweaked by tabbott to provide clear --help output recommending against
setting passwords this way.
2017-07-08 10:59:02 -07:00
Rishi Gupta b3654c7b7e confirmation: Remove vestiges of the MitUser table.
The MitUser table was removed in df525ad.

confirm_mituser.html could have been accessed through the last few lines of
confirmation/views.py:
    templates.insert(0, 'confirmation/confirm_%s.html'
                     % (obj._meta.model_name,))

The commit message on df525ad suggests there was another way
confirm_mituser.html could have been called, but I don't currently see
evidence for it in the code.
2017-07-07 18:53:00 -07:00
Rishi Gupta 8fed9eeb75 confirmation: Make host a required argument in get_link_for_object.
Removes some lines of test from test_email_change.py. The relevant code path
was never utilized by the code itself, just by the tests.
2017-07-07 18:53:00 -07:00
Greg Price 207bf2e1fb i18n: Recognize a little more Handlebars syntax in makemessages.
Usually we write translation expressions as `{{t ... }}`, but `{{ t ... }}`
is equally valid as far as Handlebars is concerned, and it matches how we
usually write simple variable substitutions, as `{{ ... }}`.  So occasionally
someone writes `{{ t ... }}`; currently there are two examples of this
in the codebase, in `settings/bot-{settings,list-admin}.handlebars`.

Probably it'd be good to pick a style and enforce it uniformly, but
until we do, the other style shouldn't break translation.
2017-07-07 17:02:39 -07:00
Umair Khan 0e8231d0f1 process_queue: Recover gracefully after PostgreSQL restart.
- For threaded workers:
Django's autoreloader catches SIGQUIT(3) to reload the program. If
a process being watched by autoreloader exits with status code 3,
reloader will restart the process. To reload, we send SIGUSR1(10)
signal from consumers to a handler in process_queue which then
exits with status code 3.

- For single worker per process:
Catch the SIGUSR1 and quit; supervisorctl will restart the worker
automatically.

Fixes #5512
2017-07-07 16:33:15 -07:00
Vaida d5517bae36 Delete the old zulip.com "referrals" system.
This system hasn't been in active use for several years, and had some 
problems with it's design.  So it makes sense to just remove it to declutter
the codebase.

Fixes #5655.
2017-07-07 14:59:18 -07:00
Tim Abbott dbabe3fd37 add_users_to_streams: Fix check for all_users option. 2017-07-07 13:57:57 -07:00
Tim Abbott 982667acf5 management: Add new library for specifying realms.
This new library is intended to make it easy for management commands
to access a realm or a user in a realm without having to duplicate any
of the annoying parsing/extraction code.
2017-07-07 13:57:57 -07:00
Rishi Gupta a26703109e settings: Change all uses of ZULIP_ADMINISTRATOR to FromAddress.SUPPORT.
Make it less likely that further development will break compatibility with
ZULIP_ADMINISTRATORs of the form "name <email>".

Note that the suggested value for this setting has been
'zulip-admin@example.com' for a while, so hopefully this commit causes no
change for most installations.
2017-07-05 15:33:01 -07:00
Rishi Gupta 577e201df0 emails: Replace DEFAULT_FROM_EMAIL with ZULIP_ADMINISTRATOR.
We're getting rid of DEFAULT_FROM_EMAIL from prod_settings_template, since
we only really need one of {DEFAULT_FROM_EMAIL, ZULIP_ADMINISTRATOR}.
2017-06-29 17:54:33 -07:00
Rishi Gupta 36cd122905 models: Change default org_type to CORPORATE.
Once we implement org_type-specific features, it'll be easy to change a
corporate realm to a community realm, but hard to go the other way. The main
difference (the main thing that makes migrating from a community realm to a
corporate realm hard) is that you'd have to make everyone sign another terms
of service.
2017-06-29 15:14:58 -07:00
Rishi Gupta 78790ce825 send_password_reset_email: Remove spurious send_mail import.
Make grepping for and otherwise understanding our email pathways slightly
easier.
2017-06-29 10:51:12 -07:00
Rishi Gupta a56d703bdc i18n: Remove translation tags in management commands. 2017-06-25 11:28:28 -04:00
Umair Khan b74cc576ae Django 1.11: Make makemessages command compatible.
Now REs have moved to template module. This commit adds a condition
to use trans_real module if the Django version is less than 1.11 else
use template module.
2017-06-13 15:14:05 -07:00
Harshit Bansal e30e244b29 emoji: Moved `unified_reactions.json` file.
Moved `unified_reactions.json` file from `tools/setup/emoji` to
`zerver/management/data` so that it can be used in production
environment.
2017-06-09 16:38:58 -07:00
Rishi Gupta 769c5ab105 emails: Send password reset emails through zerver.lib.send_email.
Previously, the password reset email behaved differently from all the other
email Zulip sends.
2017-06-06 23:22:22 -07:00
rht 940cf9db3b Run queue processors multithreaded in production if system memory <3.5GB.
While running queue processors multithreaded will limit the
performance available to very small systems, it's easy to fix that by
adding more RAM, and previously, Zulip didn't work on such systems at
all, so this is unambiguously an improvement there.

Fixes #32.
Fixes #34.

(Commit message expanded significantly by tabbott.)
2017-06-03 12:19:58 -07:00
Aditya Bansal 15ea059aad pep8: Add compliance with rule E261 makemessages.py. 2017-05-31 17:07:15 -07:00
Aditya Bansal 717e5ae393 pep8: Add compliance with rule E261 commands/create_realm.py. 2017-05-31 17:07:15 -07:00
Maxim Averin 685fb16c39 Switch change_full_name to use RealmAuditLog.
This requires adding an `acting_user` parameter to the
`do_change_bot_owner` function.
2017-05-29 15:22:08 -07:00
Elliott Jin 5b4d2832fc bots: Add management command for making outgoing webhook bot. 2017-05-25 09:45:54 -07:00
Tim Abbott e9968a7a09 change_user_email: Use new get_user_for_mgmt function. 2017-05-24 15:29:59 -07:00
Matt Long 19363b2b77 notification_settings: Refactor notification preference settings.
Previously, all notification preference setting had a dedicated test
and setter. Now, all are handled through a modular function using the
property_types framework.
2017-05-23 14:47:46 -07:00
Aditya Bansal 8a6617e304 pep8: Add compliance with rule E261 to set_default_streams.py. 2017-05-07 23:21:50 -07:00
Aditya Bansal 7271fc8a7b pep8: Add compliance with rule E261 to import.py. 2017-05-07 23:21:50 -07:00
Aditya Bansal d8db632d21 pep8: Add compliance with rule E261 to enqueue_file.py. 2017-05-07 23:21:50 -07:00
Aditya Bansal 6a53c53588 pep8: Add compliance with rule E261 to email_mirror.py. 2017-05-07 23:21:50 -07:00
Aditya Bansal f1b2f10574 pep8: Add compliance with rule E261 to dump_messages.py. 2017-05-07 23:21:50 -07:00
Rishi Gupta 6b9f25a58a Remove duplicated code between send_future_email and send_email pathway.
Note that this change restricts the context for any template that uses
send_future_email to be jsonable.
2017-05-05 14:38:25 -07:00
Rishi Gupta 68c172192b email.py: Change recipients argument of send_future_email to to_email. 2017-05-05 14:38:25 -07:00
Rishi Gupta d70e09b41d notifications: Change sender arg of send_future_email to from_email.
This commit is a step towards the goal of replacing most of the
send_future_email pathway with a call to send_email.

Note that this commit changes the default value of sender from "Zulip
<NOREPLY_EMAIL_ADDRESS>" to "NOREPLY_EMAIL_ADDRESS". NOREPLY_EMAIL_ADDRESS
will soon be changed to have the Zulip in front.
2017-05-05 14:38:25 -07:00
Rishi Gupta e46cbaffa2 email: Remove Mandrill pathways and dependency.
Everything it was doing (send_future_email) can now be done using
ScheduledJob.
2017-05-05 14:38:23 -07:00
Rishi Gupta 925ee8c0f1 Add a send_email function that takes a template_prefix and context.
This commit replaces all uses of django.core.mail.send_mail with send_email,
other than in the password reset flow, since that code looks like it is just
a patch to Django's password reset code.

The send_email function is in a new file, since putting it in
zerver.lib.notifications would create an import loop with confirmation.models.

send_future_email will soon be moved into email.py as well.
2017-05-05 14:20:32 -07:00
Rishi Gupta 92dd76822f email: Move and rename password reset templates. 2017-05-03 23:26:14 -07:00
Tim Abbott 4a67ba241e realm_domains: Restructure library for getting realm domains.
* Remove duplicate list_of_domains_for_realm of get_realm_domains.
* Move get_realm_domains from actions.py.
2017-04-28 21:11:56 -07:00
K.Kanakhin e3e52e7284 email-mirror: Move postfix email mirror integration to separate script.
This fixes a performance problem where we were previously starting up
a full Django process (~0.7s even on a fast machine) every time a new
email came in, potentially allowing users to accidentally DoS a Zulip
server.  Now, we just post over HTTPS, allowing the existing thread
pool support to do its job.

- Add script wrapper to communicate postfix pipe with django web server
  over HTTP(S). It uses shared_secret authentication mode.
- Add django view to process messages from email mirror server.
- Clean management command `email-mirror`. Left just functional
  for cron email processing.
- Add routes for new tornado view.
- Change pipe script in master process postfix config template
  based on updated script.
- Add tests.

Tweaked by tabbott to adjust the directory and set better defaults.

Fixes #2421.
2017-04-24 21:24:23 -07:00
hackerkid b2504084ab Replace timezone.now with timezone_now. 2017-04-16 12:28:56 -07:00
hackerkid 55c3d12078 Replace timezone.utc with timezone_utc. 2017-04-16 12:28:56 -07:00
Yago González e710110a9e i18n: Automatically strip Handlebars strings.
Some Handlebars strings contained whitespaces characters at their ends.
With this, such characters are removed, as well as multiple spaces
(like the ones produced by code indentation).

This also includes a couple of fixes that removes spaces that were
intentionally placed before/after the string to translate.
2017-04-14 17:37:25 -07:00
Yago González 31c92cdcbc refactor: Keep sub replacements' format uniform. 2017-04-14 17:37:25 -07:00
Umair Khan 9dec5306ce makemessages.py: Handle unicode strings.
This commit applies to Python 2. It seems that sorted()
cannot handle non-ascii character unless the strings are
marked as Unicode.
2017-04-11 22:32:06 -07:00
Abhijeet Kaur 8f88b045a4 Rename "Administration" to "Organization" in the settings UI.
This better sets expectatations for the fact that in Zulip, the
Organization settings UI is available read-only to non-administrator
users.

Tweaked by tabbott to update some additional references.
2017-04-07 17:32:56 -07:00
Umair Khan 846e8686c4 makemessages: Exclude hidden files.
Fixes #3981.
2017-04-05 11:09:32 -07:00
Harshit Bansal 664b23c495 commands: Rename `realm_alias` command to `realm_domain`. 2017-04-04 15:48:03 -07:00
Harshit Bansal 362ee482c0 commands/realm_alias.py: Remove any occurrences of `alias`. 2017-04-04 15:48:03 -07:00
Harshit Bansal 983225612d actions.py: Rename `get_realm_aliases()` to `get_realm_domains()`. 2017-04-04 15:48:03 -07:00
Harshit Bansal 92c512d679 zerver/models.py: Rename can_add_alias() to can_add_realm_domain(). 2017-04-04 15:48:03 -07:00
Harshit Bansal 6f0b46d84f create_realm.py: Remove unnecessary imports. 2017-04-04 15:48:03 -07:00
Harshit Bansal ac2172e233 models: Rename RealmAlias model to RealmDomain.
Includes a migration.
2017-04-04 15:48:03 -07:00
Rishi Gupta 30024d0a8f models: Remove Realm.domain. 2017-03-25 19:55:48 -07:00
Rishi Gupta caef5332d5 mailchimp: Fix error in add_users_to_mailing_list.py.
Old behavior is a holdover from development testing.
2017-03-21 16:13:00 -07:00
Rishi Gupta 92dd767519 management commands: Change help text to say realm instead of domain. 2017-03-14 17:17:42 -07:00
Rishi Gupta 0c032adbde create_realm.py: Remove --domain option.
Realm domains can now be managed via Admin Settings in the web app.
2017-03-14 17:17:42 -07:00
Rishi Gupta 11346e50bb management commands: Use Realm.string_id instead of domain in print statements. 2017-03-13 09:44:32 -07:00
Tim Abbott ca50e10569 sessions: Move session deletion code to sessions.py.
This new organization feels more thematically appropriate, and of
course shrinks the amount of stuff unnecessarily in actions.py
2017-03-08 02:48:18 -08:00
Tim Abbott e90df029bf create_realm: Eliminate obsolete deployment code.
Since we're not using this model anymore, we certainly don't need it
when creating realms.
2017-03-06 00:07:45 -08:00
Tim Abbott 8815a598d8 digest: Remove obsolete deployment code.
This system was quite complicated, and never had great semantics.

Eventually, we'll want some other system for gating which server
should generate digest emails for which realm controlled via the
database.
2017-03-06 00:05:21 -08:00
sinwar 6f0564e9f4 python: Fix remaining bare excepts in codebase.
Fixes #2862.
2017-03-05 16:17:04 -08:00
Tim Abbott d95412425a generate_realm_creation_link: Fix line-wrapping of help text. 2017-03-04 19:02:01 -08:00
feorlen 3c954096f6 generate_realm_creation_link: Check for an uninitialized database.
This provides a nice error message to users who miss a step in the
instructions.

Fixes #3672.
2017-03-04 19:01:01 -08:00
Raghav Jajodia a3a03bd6a5 mypy: Added Dict, List and Set imports.
Fixed mypy errors associated with the upgrade.
2017-03-04 14:33:44 -08:00
adnrs96 9eb47f108c Refactor: Change upload_avatar_image to accept two user profiles.
In this commit we just change the upload_avatar_image function to accept
two user_profiles acting_user_profile and target_user_profile. Basically
email param is dropped for a target_user_profile so that avatar's could
be moved lateron to user id based storage.
2017-03-03 18:15:15 -08:00
Umair Khan 87c5ace24d i18n: Use deterministic order in translations.json.
Fixes: #3897
2017-03-03 12:32:17 +05:00
Rishi Gupta 3d07ac0c49 Change timezone-naive datetimes to use timezone.now() where safe to do so.
Change timezone-naive datetimes to use timezone.now() in cases where there
is no change in behavior.
2017-03-01 22:54:28 -08:00
Rishi Gupta c388858e53 Fix timezone errors in ScheduledJob and digest creation filters.
datetime.utcnow() is a timezone-naive datetime. The Django ORM interprets it
in the settings.TIME_ZONE timezone (e.g. 'America/New_York' in the
development server). We perhaps haven't noticed errors yet since with
'America/New_York' all it means is that emails are sent 5 hours early, or a
slightly different set of messages are included in the digest.
2017-03-01 22:54:28 -08:00
Rishi Gupta 562bc6429c Replace datetime.now() with timezone.now() in Django ORM queries.
When you pass a naive datetime to the Django ORM, it uses settings.TIME_ZONE
for the time zone. In the development environment, both settings.TIME_ZONE
and datetime.now() use 'America/New_York', so there is no change in behavior
there. (fromtimestamp with no tz argument uses the same timezone as
datetime.now)

We are soon going to change settings.TIME_ZONE to UTC, so need to remove
naive datetimes from queries to the ORM.
2017-03-01 22:54:28 -08:00
Rishi Gupta 2b2be8120f Change datetime.now(tz=X) to timezone.now().
datetime.now with a timezone set is equivalent to timezone.now() if it's
never being printed out, but the latter is cleaner and more idiomatic.
2017-03-01 22:54:28 -08:00
Tim Abbott cf96b1b873 generate_realm_creation_link: Clean up instructions. 2017-02-21 20:19:16 -08:00
Tim Abbott 32bfebeb7a mypy: Fix inconsistencies in use of *args/**kwargs. 2017-02-18 18:39:44 -08:00
Umair Khan 00f8239563 Clean code of send_test_email command.
* Derive from sendtestemail command of Django.
* Remove unwanted imported.
* Allow test email to admin and managers.
2017-02-10 16:01:43 -08:00
Umair Khan ef0d2a4bb5 logging: Use django.server to filter 200 and 304.
Previously, we were monkey patching the runserver command
in zerver/management/commands/rundjango.py for this.
2017-02-10 15:55:17 -08:00
Harshit Bansal 7d10cbc32b Add RealmAlias.allow_subdomains to model, frontend, and API.
Includes a database migration.

Fixes #1868.
2017-02-08 22:03:27 -08:00
Tim Abbott 12e7be585d i18n: Fix compilemessages processing extra files in production.
In a Zulip production environment, STATIC_ROOT points to the shared
directory that static assets are served from, and so the
compilemessages management command was trying to process every
historical version in there.
2017-02-03 17:17:23 -08:00
Tim Abbott a889346c9f Remove zh_CN old Django name for Chinese. 2017-02-03 14:29:03 -08:00