Commit Graph

295 Commits

Author SHA1 Message Date
Eklavya Sharma 896c18a57b email-mirror.py: Make it pass on mypy in python 3.
* Replace filter by list comprehension.

* Add '# type: ignore' to statements which use attributes from
  modeule `posix`, since stubs for posix are missing on python 3.
2016-07-07 10:07:28 -07:00
Eklavya Sharma 17cb6e00bd create_stream.py: Fix broken code.
* get_realm returns None if no matching realm is present, but
  create_stream.py assumed it raises Realm.DoesNotExist.
* encoded/decode strings properly.
2016-07-07 10:06:39 -07:00
Umair Khan 47fb293f8a i18n: Add English locale to language list if missing.
The English locale data is only present if the user has run `manage.py
makemigrations` on the system.
2016-07-06 11:34:28 -07:00
Umair Khan 035fceb814 Add dynamically loaded language dropdown. 2016-07-04 11:56:02 -07:00
Vishnu Ks 1cbd39b768 Unique link generator for realm creation. 2016-06-30 15:52:41 -07:00
Tim Abbott e72f41cdec Remove old prototype data import/export tool.
This prototype from Dropbox Hack Week turned out to be too inefficient
to be used for realms with any significant amount of history, so we're
removing it.

It will be replaced by https://github.com/zulip/zulip/pull/673.
2016-06-27 13:47:08 -07:00
rahuldeve a3745178e5 Use django.utils.autoreload to restart queue workers at code change.
Fixes #621, #1045.
2016-06-26 20:12:11 -07:00
Umair Khan 134b165b1a Fix frontend translation string bug.
The translation string which contain \n are not captured
within {{#tr}}{{/tr}} blocks. Fix is to escape the slash.
2016-06-20 11:31:28 -07:00
Vishnu Ks 8213ca135a Move default_stream list to settings. 2016-06-17 16:01:38 -07:00
Umair Khan 6fcfed8d9e Update regexes to capture translation strings.
Strings are captured from:
- i18n.t
- compose_error
- placeholder
2016-06-13 09:03:56 -07:00
Eklavya Sharma d169cc5376 zerver/lib/bulk_create.py: Fix string annotations.
Change string type from `str` to `text_type` where required.
Also fix clashing annotations.
2016-06-13 10:08:14 +05:30
Tim Abbott d9d0515d3b Add mypy bug number for check_redis type: ignore. 2016-06-04 16:11:18 -07:00
medullaskyline 7c2c7fb31c Annotate zerver/lib/bulk_create.py. 2016-06-04 15:51:05 -07:00
Tim Abbott a1a27b1789 Annotate most Zulip management commands. 2016-06-04 10:12:06 -07:00
Tim Abbott c2bea0fa08 zulip_finish: Remove useless return statement. 2016-06-04 10:06:31 -07:00
Tim Abbott ab2d325a08 Update production default streams to be less engineering-centric. 2016-05-31 07:38:25 -07:00
Eklavya Sharma 94e4b39112 Replace python2.7 by python everywhere. 2016-05-29 05:03:08 -07:00
Eklavya Sharma 149938d468 Change shebangs from python2.7 to python. 2016-05-29 05:03:08 -07:00
Eklavya Sharma 30892b2f99 Make makemessages.py pass mypy check. 2016-05-25 15:04:39 -07:00
Umair Khan 82b5d9304b [third] Integrate i18next with Handlebars 2016-05-19 22:58:25 -07:00
Tomasz Kolek 8c18b8947f Add bot_type field to UserProfile.
This is intended to support creating different types of bots with
potentially limited permissions.
2016-05-19 22:37:37 -07:00
Tim Abbott 92bec8cfea Merge Zulip 1.3.12 security release. 2016-05-10 11:32:26 -07:00
Tim Abbott 9b65464b6b logout_all_users: Add option to logout deactivated users. 2016-05-10 09:50:57 -07:00
Umair Khan 5359e6b0d4 Convert Zulip to use Jinja2 templates.
This results in a substantial performance improvement for all of
Zulip's backend templates.

Changes in templates:
- Change `block.super` to `super()`.
- Remove `load` tag because Jinja2 doesn't support it.
- Use `minified_js()|safe` instead of `{% minified_js %}`.
- Use `compressed_css()|safe` instead of `{% compressed_css %}`.
- `forloop.first` -> `loop.first`.
- Use `{{ csrf_input }}` instead of `{% csrf_token %}`.
- Use `{# ... #}` instead of `{% comment %}`.
- Use `url()` instead of `{% url %}`.
- Use `_()` instead of `{% trans %}` because in Jinja `trans` is a block tag.
- Use `{% trans %}` instead of `{% blocktrans %}`.
- Use `{% raw %}` instead of `{% verbatim %}`.

Changes in tools:
- Check for `trans` block in `check-templates` instead of `blocktrans`

Changes in backend:
- Create custom `render_to_response` function which takes `request` objects
  instead of `RequestContext` object. There are two reasons to do this:
    1. `RequestContext` is not compatible with Jinja2
    2. `RequestContext` in `render_to_response` is deprecated.
- Add Jinja2 related support files in zproject/jinja2 directory. It
  includes a custom backend and a template renderer, compressors for js
  and css and Jinja2 environment handler.
- Enable `slugify` and `pluralize` filters in Jinja2 environment.

Fixes #620.
2016-05-09 09:55:18 -07:00
Preston Hansen 635828069f Add feature to mark all in stream/topic as read with mouse.
Fixes #736.
2016-05-08 09:02:46 -07:00
Tim Abbott 54022ac204 Fix unnecessary whitespace between , and ). 2016-05-04 14:16:53 -07:00
rahuldeve dde832b158 Add Attachment model to keep track of uploads.
This commit adds the capability to keep track and remove uploaded
files.  Unclaimed attachments are files that have been uploaded to the
server but are not referred in any messages.  A management command to
remove old unclaimed files after a week is also included.

Tests for getting the file referred in messages are also included.
2016-05-02 22:14:47 -07:00
Vladislav Manchev f5e6176aea Add custom realm emoji UI to administration page. 2016-04-26 13:15:54 -07:00
Tim Abbott d9dba5d2c2 deactivate_realm: Improve error handling for unknown realm. 2016-04-21 09:02:00 -07:00
Tim Abbott c2237c60c0 deactivate_realm: Fix help string. 2016-04-21 09:02:00 -07:00
Tim Abbott d890011442 Add a script to reactivated deactivated realms. 2016-04-21 09:02:00 -07:00
Tim Abbott 49799440a4 Replace use of django-guardian with fields on UserProfile.
As documented in https://github.com/zulip/zulip/issues/441, Guardian
has quite poor performance, and in fact almost 50% of the time spent
running the Zulip backend test suite on my laptop was inside Guardian.

As part of this migration, we also clean up the old API_SUPER_USERS
variable used to mark EMAIL_GATEWAY_BOT as an API super user; now that
permission is managed entirely via the database.

When rebasing past this commit, developers will need to do a
`manage.py migrate` in order to apply the migration changes before the
server will run again.

We can't yet remove Guardian from INSTALLED_APPS, requirements.txt,
etc. in this release, because otherwise the reverse migration won't
work.

Fixes #441.
2016-04-20 21:51:52 -07:00
Tomasz Kolek 82f1cdb085 Add send_webhook_fixture_message command.
This tool simplifies the process of producing nice screenshots for
documenting webhook integrations.

Fixes #658.
2016-04-20 10:45:27 -07:00
Varshit 4e1060076d Purge 'from typing import *' from zerver/.
This is a partial implementation of #636.
2016-04-07 14:07:07 -07:00
Ashish 909b0635c8 Add management command to test sending email.
Fixes: #622.

[With cleanups/doc tweaks by tabbott]
2016-04-03 16:50:16 -07:00
Tim Abbott b7dcf2181f Add PEP-484 type annotations to management commands. 2016-04-03 15:40:23 -07:00
Ryan Moore 2fe0700f55 Update memcache -> remote cache in inline documentation. 2016-03-31 12:56:42 -07:00
Ryan Moore 1489f0992c s/fill_memcached_cache/fill_remote_cache/g 2016-03-31 12:54:29 -07:00
Tim Abbott 06b33da709 process_queue: Fix missing worker.setup() in single-threaded codepath. 2016-03-27 23:17:16 -07:00
Tim Abbott cd2348e9ae Run queue processers multithreaded in development.
This change drops the memory used for Python processes run by Zulip in
development from about 1GB to 300MB on my laptop.

On the front of safety, http://pika.readthedocs.org/en/latest/faq.html
explains "Pika does not have any notion of threading in the code. If
you want to use Pika with threading, make sure you have a Pika
connection per thread, created in that thread. It is not safe to share
one Pika connection across threads.".  Since this code only connects
to rabbitmq inside the individual threads, I believe this should be
safe.

Progress towards #32.
2016-03-20 18:04:24 -07:00
Tim Abbott 06e68d52ce Remove dead AsyncDjangoHandler field. 2016-03-20 16:53:13 -07:00
Tim Abbott 7fabfe9cb9 Add __repr__s for ClientDescriptor and AsyncDjangoHandler. 2016-03-20 16:53:13 -07:00
Tim Abbott 3f55e26a9f Fix Tornado memory leak with synchronously handled requests.
The new Tornado handler tracking logic properly handled requests that
threw an exception or followed the RespondAsynchronously code path,
but did not properly de-allocated the handler in the syncronous case.

An easy reproducer for this is to load a new Zulip browser window;
that will leak 2 handler objects for the 2 synchronous requests made
from Django to Tornado as part of initial state fetching.
2016-03-20 16:53:13 -07:00
Tim Abbott 320428052a Fix AsyncDjangoHandler view exception code path.
The recent Tornado memory leak fix
(1396eb7022) didn't use the correct
variable name for the current handler ID, causing this cleanup code to
fail in the event that a view raised an exception.
2016-03-19 22:50:35 -07:00
Eklavya Sharma 9e3c3e14f5 Partially apply Python 3 libmodernize.fixes.fix_dict_six.
Refer to #256
2016-03-19 15:52:58 -07:00
Tim Abbott 1396eb7022 Fix Tornado memory leak of handler objects.
In 2ea0daab19, handlers were moved to
being tracked via the handlers_by_id dict, but nothing cleared this
dict, resulting in every handler object being leaked.  Since a Tornado
process uses a different handler object for every request, this
resulted in a significant memory leak.  We fix this by clearing the
handlers_by_id dict in the two code paths that would result in a
Tornado handler being de-allocated: the exception codepath and the
handler disconnect codepath.

Fixes #463.
2016-03-17 18:33:59 -07:00
Reid Barton 9735025167 Refactor logic around restricted_to_domain.
Add a function email_allowed_for_realm that checks whether a user with
given email is allowed to join a given realm (either because the email
has the right domain, or because the realm is open), and use it
whenever deciding whether to allow adding a user to a realm.

This commit is not intended to change any behavior, except in one case
where the Zulip realm's domain was not being converted to lowercase.
2016-02-07 10:54:52 -05:00
Tim Abbott 1af7cbfd64 runtornado: Move more imports to the top of the file.
This is needed for adding more specific type annotations but is
otherwise counterproductive since it increases the diff from the
original.
2016-02-03 19:47:14 -08:00
Tim Abbott 2259ce62f8 tornado: Fix AsyncDjangoHandler get() and friends missing args/kwargs. 2016-02-03 19:47:14 -08:00
Tim Abbott d7b7ae2d0f rundjango: Fix confusing return line in log_message_monkey. 2016-02-03 19:31:45 -08:00
Tim Abbott 05a40f11b3 runtornado: Add explicit return None. 2016-02-03 19:31:45 -08:00
Tim Abbott 37e987e250 Make realm_filters --show's empty list output more consistent. 2016-02-03 19:29:44 -08:00
Tim Abbott fcc32b1093 Fix check_redis abuse of setting self.trim global. 2016-02-03 19:29:44 -08:00
Tim Abbott 7595e4b05f process_queue: Fix worker variable being accessed before initialization. 2016-02-03 19:29:44 -08:00
Tim Abbott aad3bff193 Harden style rule for % comprehensions and fix existing errors. 2016-02-02 23:08:19 -08:00
Tim Abbott 6528b18ad3 Switch all urllib/urlparse usage to six.moves.urllib.
This provides Python 2+3 compatibility for our use of urllib.

Also add a test to avoid future regressions.
2016-01-26 21:09:43 -08:00
Tim Abbott c7e3c3ce38 Look up client descriptors by handler_id.
Previously, client descriptors were referenced directly from the
handler object.  Once we split the Tornado process into separate queue
and connection servers, these will no longer be in the same process,
so we need to reference them by ID instead.
2016-01-26 20:57:25 -08:00
Tim Abbott ea6211c041 Allocate handler ids in AsyncDjangoHandler setup process. 2016-01-26 20:56:53 -08:00
Tim Abbott dfaf45b2b6 Wrap Django runserver to prevent spammy logging.
Django's `manage.py runserver` prints a relatively low-information log
line for every request of the form:

[14/Dec/2015 00:43:06]"GET /static/js/message_list.js HTTP/1.0" 200 21969

This is pretty spammy, especially given that we already have our own
middleware printing a more detailed version of the same log lines:

2015-12-14 00:43:06,935 INFO     127.0.0.1       GET     200   0ms /static/js/message_list.js (unauth via ?)

Since runserver doesn't have support controlling whether these log
lines are printed, we wrap it with a small bit of code that silences
the log lines for 200/304 requests (aka the uninteresting ones).
2015-12-25 16:23:57 -08:00
Tim Abbott a79e89b28f Cleanup remaining usage of % comprehensions without explicit tuples. 2015-12-05 15:29:42 -08:00
Tim Abbott 3e794351ab Remove expunge_* management commands.
These management commands never worked and given that we removed the
retention_policy code that they call, it makes sense to remove them as
well.
2015-11-11 21:34:39 -08:00
Walter Heck f145b01d91 Update create_user management command.
notify_new_user was recently moved to zerver.lib.actions from
zerver.views and this wasn't properly updated. This would give an
error when doing a `manage.py create_user` from the command line.
2015-11-08 19:41:00 -08:00
Tim Abbott 3b185ad4de Apply Python 3 futurize transform libmodernize.fixes.fix_input_six. 2015-11-01 09:26:17 -08:00
Tim Abbott f3783fb4a1 Apply Python 3 futurize transform libfuturize.fixes.fix_print_with_import. 2015-11-01 09:26:16 -08:00
Tim Abbott 43abd83d1c Apply Python 3 futurize transform lib2to3.fixes.fix_ws_comma. 2015-11-01 09:26:14 -08:00
Tim Abbott 06f6ee6566 Apply Python 3 futurize transform lib2to3.fixes.fix_idioms. 2015-11-01 09:25:47 -08:00
Tim Abbott 5ce6a3c8f9 Apply Python 3 futurize transform lib2to3.fixes.fix_funcattrs. 2015-11-01 08:09:54 -08:00
Tim Abbott 8c34c40924 Apply Python 3 futurize transform lib2to3.fixes.fix_except. 2015-11-01 08:08:33 -08:00
Steven Oud d5435fad1d Consistently use /usr/bin/env python2.7 in shebangs and commands. 2015-10-21 22:58:21 +00:00
Tim Abbott 71a06d58de Convert uses of Realm.objects.get() to get_realm().
get_realm is better in two key ways:
* It uses memcached to fetch the data from the cache and thus is faster.
* It does a case-insensitive query and thus is more safe.
2015-10-15 09:16:58 -04:00
Tim Abbott 419d31a007 Expand documentation for the LDAP auth integration.
Fixes #134, #173.
2015-10-15 09:16:58 -04:00
Tim Abbott 858d0a984b Move API super users configuration into the database.
(imported from commit 3cc702f93e7252b42930dba4bde93a915b6dbf44)
2015-09-20 11:13:20 -07:00
Tim Abbott 3b84a9a010 Modernize management command for deactivating a realm.
(imported from commit 9ef681193ff04e447c0afbba66187357624d98f3)
2015-09-19 19:10:27 -07:00
Tim Abbott a5c1fc6d76 Remove unnecessary import_realm_filters.py management command.
It's superceded by realm_filters.py.

(imported from commit b53c8bc4daf965387f5ba88711dd7ece2db1dfd9)
2015-09-19 19:05:16 -07:00
Tim Abbott 930c64df8a Move zulip.com-related statsd configuration out of main settings.py.
This also removes the convenient way to run statsd in the Dev VM,
because we don't anticipate anyone doing that.  It's just 2 lines of
config to configure it anyway:

STATSD_HOST = 'localhost'
STATSD_PREFIX = 'user'

(imported from commit 5b09422ee0e956bc7f336dd1e575634380b8bfa2)
2015-08-22 13:49:43 -07:00
Tim Abbott 83a653efce Change semantics of ZULIP_COM to include ZULIP_COM_STAGING machines.
(imported from commit 2e84c756b4ac41c13d619ab6ceb99e347d1f48cb)
2015-08-21 10:41:14 -07:00
Tim Abbott a8261dd6ac Remove Deployment initialization from Development.
(imported from commit 4596fcf480c10c88496cd6506edb0a6e702d823c)
2015-08-21 10:40:57 -07:00
Tim Abbott 16e0f21687 Fix a bunch of Zulip Voyager related comments.
(imported from commit 176e875cbc310ba00d6137930ddb718bd68f1866)
2015-08-21 10:40:54 -07:00
David Roe e3f38acbce Enterprise => Voyager.
(imported from commit 41b9a67301aeaf5fd40bbbb8f34a326ca98431fd)
2015-08-21 10:33:35 -07:00
David Roe 3f7cb34b00 enterprise => voyager
(imported from commit 04be792bb480d5e5db1c91d296d1000cf1682571)
2015-08-21 10:33:35 -07:00
David Roe edf7e732a2 ENTERPRISE => VOYAGER.
(imported from commit 4f8080b9f506a87ca40bef32e39de5218cba916a)
2015-08-21 10:33:35 -07:00
David Roe d702ddb3cf DEPLOYED => PRODUCTION
(imported from commit a7d365c07916c5103fc721c712db94f9af977640)
2015-08-21 10:33:35 -07:00
Reid Barton ae0ae3dde8 Django 1.8: declare positional arguments in management commands
(imported from commit d9efca1376de92c8187d25f546c79fece8d2d8c6)
2015-08-20 23:35:40 -07:00
Reid Barton 9ded218950 Django 1.7 compatibility: miscellaneous changes
(imported from commit 1f63d936c3d195acd033bb42fddacd240af3889b)
2015-08-20 23:01:26 -07:00
Nemanja Stanarevic d9f111ec28 Added import_dump command to manage.py to import JSON database dumps
(imported from commit abd410e5299f15c1a5851e87d18664b59837fbb0)
2015-08-20 21:19:57 -07:00
David Roe 809efc4f2b Add realm option to show or not show digest e-mail.
(imported from commit e9cfe519aa9f2857a1dcc7c75f5e1889834fcb86)
2015-08-20 15:29:45 -07:00
Tim Abbott 4f1388706c Disable digests and Missed Message notifications for internal realm.
(imported from commit 24a25bcf6653c674ac8a34324a4674e438f0af38)
2015-02-05 18:40:13 -08:00
Jason Michalski 2a5ec20295 Add support for one time use email address
The one time use address are a unique token which maps to stored stated
in redis. We store the user_id, recipient_id, and subject. When an email
is received at this address it is sent to the stored recipient by the
stored user. Anyone with this address can send a single message as this
user.

(imported from commit 4219417bdc30c033a6cf7a0c7c0939f7d0308144)
2014-10-06 15:27:54 -07:00
Tim Abbott 0494e40c39 Merge zerver/tornado_callbacks.py into zerver/lib/event_queue.py.
It had stopped being a coherently distinct component a while ago.

(imported from commit 0617957bcfe8dcaf69143c88a96ddd51ecb31a98)
2014-04-23 17:22:31 -07:00
Jessica McKellar f9d5a02710 Fix bug that hid digest email settings box for customers.
(imported from commit 6c1e3f53998eca0232fbfe66c62ab87b009bccd8)
2014-02-27 20:25:26 -05:00
acrefoot 233cfc2593 revert changes to deliver_email.py introduced in dc5e78cccb
I'm pretty sure this was changed for purposes of testing

(imported from commit b4b1224a5812ceb248ded3755c0b1801797c1beb)
2014-02-10 15:22:20 -05:00
Jessica McKellar bb60e96910 Enable digest emails for CUSTOMER4.
(imported from commit 08b4adb4fd95fdfd661ccbd9cdb5782bf0d743f0)
2014-02-10 13:53:39 -05:00
Luke Faraone 0f96d2229e email-mirror: Also accept original recipient via argv
(imported from commit 9a29fe6906713e75ee208c924b67211575c011f2)
2014-01-31 15:33:15 -05:00
Steve Howell aca5586cc0 Fix command line help for ./manage.py show_admins
(imported from commit d5de8832214db7b8a754f155b225e706b655260b)
2014-01-28 17:40:00 -05:00
Leo Franchi 0917a27b79 Document realm filter regexp oddities
(imported from commit 987d8ea679f40c31f2fe2280b6d585f82b20e606)
2014-01-27 13:43:55 -05:00
Tim Abbott 6d64427c46 Fix email mirror.
(imported from commit 73c109db6e821b139203f662023ed780fcee0f24)
2014-01-24 18:11:20 -05:00
Steve Howell d431206a3f Have the "knight" command use actions.do_change_is_admin.
Using the higher level function has the benefit of sending
notifications to clients.

(imported from commit d680e3305b216fec3462479c696e572d008ee4c2)
2014-01-22 14:43:28 -05:00
Luke Faraone b1cf9c3e25 Rewrite email-mirror documentation.
(imported from commit 4af076b3063d7b82657765c12cf4dba0248cbf9f)
2014-01-15 16:07:19 -05:00
Luke Faraone 895b7dc9b1 Accept emails on stdin in email-mirror and queue for later processing
(imported from commit 7efccc294f87d3661e50f10962a80071b91e3879)
2014-01-15 16:07:18 -05:00
Luke Faraone 1005e2ae7c Use module docstring for usage information
(imported from commit 7c1cbf68eca4e3f31c75bfe13b50654e76be0df3)
2014-01-15 16:07:18 -05:00