Commit Graph

794 Commits

Author SHA1 Message Date
Tim Abbott cd2348e9ae Run queue processers multithreaded in development.
This change drops the memory used for Python processes run by Zulip in
development from about 1GB to 300MB on my laptop.

On the front of safety, http://pika.readthedocs.org/en/latest/faq.html
explains "Pika does not have any notion of threading in the code. If
you want to use Pika with threading, make sure you have a Pika
connection per thread, created in that thread. It is not safe to share
one Pika connection across threads.".  Since this code only connects
to rabbitmq inside the individual threads, I believe this should be
safe.

Progress towards #32.
2016-03-20 18:04:24 -07:00
Eklavya Sharma 9e3c3e14f5 Partially apply Python 3 libmodernize.fixes.fix_dict_six.
Refer to #256
2016-03-19 15:52:58 -07:00
Tim Abbott f9222de83e Auto-load commonly used modules in manage.py shell.
This automatically loads settings, zerver.models.* and
zerver.lib.actions.* when you start `manage.py shell`, which should
save a bit of time basically every time someone uses it.

Fixes #275.
2016-03-19 11:32:49 -07:00
Varshit 72033069ed Extend lint-all to check for newlines at the end of files. 2016-03-17 23:03:56 -07:00
Tim Abbott 3a46bae542 Fix shell script list computation to exclude files not in git.
This fixes an issue where the next commit's no-newline-and-end-of-file
check was incorrectly firing on tools/phantomjs.
2016-03-17 23:03:56 -07:00
Eklavya Sharma 37f9520666 Make the remaining ambiguous divisions python 3 compatible.
Refer to #256
2016-03-12 10:53:51 -08:00
Eklavya Sharma 14130a84ca Partially apply Python 3 transform libpasteurize.fixes.fix_newstyle
Refer to #256
2016-03-12 23:19:56 +05:30
Eklavya Sharma 3ab567db98 Add call to generate-fixtures in test-backend.
Add call to tools/generate-fixtures in tools/test-backend before
starting the tests.  Previously, test-backend could fail if called
after tools/test-js-with-casper had failed.

Fixes #501.
2016-03-12 09:24:32 -08:00
Eklavya Sharma 01bfa2d94d Apply Python 3 futurize transform libmodernize.fixes.fix_unicode_type
Refer to #256
2016-03-10 22:04:15 -08:00
Eklavya Sharma b9e792c4e6 Apply Python 3 futurize transform libmodernize.fixes.fix_xrange_six
Refer to #256
2016-03-10 22:03:58 -08:00
Eklavya Sharma aa505b0d55 Apply Python 3 futurize transform libmodernize.fixes.fix_map
Refer to #256
2016-03-10 22:03:44 -08:00
Eklavya Sharma 7b8cb105bf Apply Python 3 futurize transform libmodernize.fixes.fix_imports_six
Refer to #256
2016-03-10 22:03:30 -08:00
Eklavya Sharma def027a1ec Apply Python 3 futurize transform libmodernize.fixes.fix_filter
Refer to #256
2016-03-10 22:03:06 -08:00
Eklavya Sharma d3b63f9a2d Apply Python 3 futurize transform libmodernize.fixes.fix_file
Refer to #256
2016-03-10 22:02:34 -08:00
Eklavya Sharma c59185e119 Apply Python 3 futurize transform libfuturize.fixes.fix_print_with_import
Refer #256
2016-03-10 22:02:17 -08:00
Eklavya Sharma 3e7827358e Apply Python 3 futurize transform libfuturize.fixes.fix_next_call 2016-03-10 22:02:12 -08:00
Eklavya Sharma e2d5ec1868 Apply Python 3 futurize transform lib2to3.fixes.fix_ws_comma 2016-03-10 22:02:04 -08:00
Eklavya Sharma 4fb549abe8 Apply Python 3 futurize transform lib2to3.fixes.fix_idioms
Refer to #256
2016-03-10 22:02:01 -08:00
Eklavya Sharma f3d387e727 Apply Python 3 futurize transform lib2to3.fixes.fix_except 2016-03-10 22:01:50 -08:00
Tim Abbott df4ab3c788 run_dev: Fix return for twisted finish function.
This was flagged by mypy; it's not clear this should be needed.
2016-02-03 19:29:07 -08:00
Tim Abbott eee36618fe run-dev: Fix overwritten manage_args variable with the wrong type.
manage_args is set to a list of arguments a few lines later in the
function, making this initialization as the empty string useless and
confusing.

Discovered using mypy.
2016-02-03 19:25:18 -08:00
Tim Abbott aad3bff193 Harden style rule for % comprehensions and fix existing errors. 2016-02-02 23:08:19 -08:00
Tim Abbott 206dc3aafc Add python 3 compatibility check for libmodernize.fixes.fix_dict_six.
It's not clear whether this will end up being net negative in value in
the long term since it's kinda hard to understand the output, but in
the short term it should prevent regressions.
2016-01-26 21:11:25 -08:00
Tim Abbott 757e89260e Migrate use of StringIO to Python 2+3 compatible six.moves.cStringIO.
And add a check for additional usage of the original StringIO module.
2016-01-26 21:09:43 -08:00
Tim Abbott 1f44417fc1 Switch to using Python 3 style division everywhere.
Also add testing for this to our Python 3 compatibility test suite.
2016-01-26 21:09:43 -08:00
Tim Abbott 6528b18ad3 Switch all urllib/urlparse usage to six.moves.urllib.
This provides Python 2+3 compatibility for our use of urllib.

Also add a test to avoid future regressions.
2016-01-26 21:09:43 -08:00
Tim Abbott 700055c194 Apply modernize transform libmodernize.fixes.fix_file.
This replaces use of file() with open() which is python 3 compatible,
and also adds it to our python 3 support test suite.
2016-01-26 21:09:42 -08:00
Tim Abbott a5d4d0aae0 test_backend: Add option to profile the backend test suite. 2016-01-26 20:41:01 -08:00
Tim Abbott f9791558e9 test_runner: Support continuing running tests after a failure. 2016-01-26 20:41:01 -08:00
Tim Abbott b43aadad8b test-backend: Rewrite in python to support computing test coverage.
The code for doing test coverage is just a lot cleaner this way over
adding it to the shell script version.

Based on the basic test runner code here:
https://docs.djangoproject.com/en/1.9/topics/testing/advanced/
2016-01-26 20:41:01 -08:00
Tim Abbott 24fd3bbf55 travis: Test whether migrations are consistent with models.
This should automatically catch mistakes where someone updates the
database models but forgets to generate migrations afterwards.
2016-01-26 20:38:46 -08:00
Tim Abbott c6d06b0c4e Add zulip distribution tarballs to backups. 2016-01-23 11:44:15 -08:00
Tim Abbott d3588cb7d0 api: Include get_subscribers endpoint in public release.
It's possible we should just eliminate this mechanism, but this fixes
a proximal problem where the multi-line get_subscribers endpoint
description was being handled wrong.
2016-01-23 11:38:42 -08:00
Vladislav Manchev df4d1b3c14 Add linting code for detecting shebang bashisms.
This will prevent regressions in OpenBSD compatibility, since OpenBSD
doesn't support passing arguments in the #! line.
2016-01-21 22:33:55 -08:00
Vladislav Manchev dfbea01c8f Add support for running OpenBSD in development environment. 2016-01-21 22:33:55 -08:00
Tim Abbott 6943a142ea Fix postgres errors in Travis CI again.
Travis CI's model of installing every version of postgres on the test
VM and then shutting all the versions other than the one requested
down seems to not work very well with doing apt upgrades.  It seems
the best way to resolve this is to just uninstall the versions we
don't need.
2016-01-21 22:07:10 -08:00
Tim Abbott 26e9d55e16 deployments: Refactor locking libraries into zulip_tools.py.
The code in update-deployment and upgrade-zulip for managing the
deployment lock was nearly identical.
2016-01-11 21:36:42 -08:00
Tim Abbott 186f563176 Fix deployment locks being leaked when a deployment fails.
The point of the lock is to prevent two deployments happening at the
same time and racing with each other, not to prevent doing any future
deployments after an error happens (which is what the current
implementation does in practice).

Addresses part of #208.
2016-01-11 21:36:41 -08:00
Tim Abbott a98b0cf35d travis: Workaround postgres 9.1 conflict issues on trusty.
We ran into a bug with the Travis CI infrastructure where it postgres
9.1 is installed on the system, and so when we'd do an apt upgrade
with a new version of 9.1, the 9.1 daemon would end up getting started
and conflict with the 9.3 daemon we were trying to run.
2016-01-09 16:59:43 -08:00
Tim Abbott 24ebc10ec8 Travis: Display all errors, not just first one, in py3k testing. 2016-01-09 15:43:26 -08:00
Josh Mandel bdb9535251 Don't require link tags to close when checking templates. 2016-01-07 23:08:49 -08:00
Tim Abbott c5f08022f9 tools/clean-repo: Don't print out the list of .pyc files deleted.
This was never useful output and was pretty spammy.
2015-12-25 16:23:57 -08:00
Tim Abbott dfaf45b2b6 Wrap Django runserver to prevent spammy logging.
Django's `manage.py runserver` prints a relatively low-information log
line for every request of the form:

[14/Dec/2015 00:43:06]"GET /static/js/message_list.js HTTP/1.0" 200 21969

This is pretty spammy, especially given that we already have our own
middleware printing a more detailed version of the same log lines:

2015-12-14 00:43:06,935 INFO     127.0.0.1       GET     200   0ms /static/js/message_list.js (unauth via ?)

Since runserver doesn't have support controlling whether these log
lines are printed, we wrap it with a small bit of code that silences
the log lines for 200/304 requests (aka the uninteresting ones).
2015-12-25 16:23:57 -08:00
Tim Abbott 39e80b351d Add simple tools for fetching content of a pull request by ID. 2015-12-25 16:22:00 -08:00
Allie Jones 1bd1291f3c Replace json/send_message endpoint usage with json/message. 2015-12-12 18:14:08 -08:00
Kara McNair c947f3ed3c Add Zulip-specific details in jslint used_before_a error message.
Previously, it wasn't clear why you were getting an error when trying
to reference a newly added global variable.
2015-12-07 20:33:36 -08:00
Tim Abbott 2be7ac8d70 travis: Fix prompting for user input in production-helper. 2015-12-07 20:33:36 -08:00
Tim Abbott 0fe819eb57 lint: Tighten lint rules around whitespace around '%' comprehensions. 2015-12-05 15:29:42 -08:00
Tim Abbott 69b6b60017 lint: Clean up whitespace_rules.
Several of these rules only apply to one of Python and Javascript, and
this simplifies the logic and should make our linter code more readable.

In the process, we add support for per-rule/file pair exclusions to
handle the tab exception for codehilite.py.
2015-12-05 15:29:42 -08:00
Tim Abbott a712954c59 lint: Rewrite custom checks to use a more consistent framework. 2015-12-05 15:29:42 -08:00