Mirror
/
zulip
mirror of https://github.com/zulip/zulip.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
13,821 Commits 28 Branches 143 Tags 625 MiB
Commit Graph

1404 Commits

Author SHA1 Message Date
Vishnu Ks a7ead9e99d settings: Eliminate ADMIN_DOMAIN for creating initial realm. 
We now use `./manage.py generate_realm_creation_link` as the flow flow
for creating one's first realm.
 2016-08-25 09:37:33 -07:00
Tim Abbott 4c834172ed tools: Use 'upstream' as expected name for main zulip repo. 
Fixes: #1698.
 2016-08-24 14:23:37 -07:00
Tim Abbott 13a51737d8 update-prod-static: Fix buggy color code usage. 2016-08-22 16:38:23 -07:00
Tim Abbott 55f2c49bbf build-release-tarball: Fix code for handling update-prod-static.log. 
This is still messy code, but at least the links to where the log file
is are correct.
 2016-08-22 16:37:48 -07:00
Tim Abbott 2e2dee3396 lint: Ban lines longer than 140 characters without #ignorelongline. 2016-08-19 12:03:09 -07:00
Taranjeet Singh 7fc170d808 lint-all: Fix line with length greater than 120. 
This updates file tools/lint-all.
 2016-08-19 11:56:43 -07:00
Taranjeet Singh 1819157a5f show-last-messages: Fix line with length greater than 120 characters. 
This updates file tools/deprecated/iframe-bot/show-last-messages.
 2016-08-19 11:56:43 -07:00
Steve Howell c6a888561a tools: Extract build_custom_checkers() in tools/lint-all. 2016-08-18 14:15:01 -07:00
Steve Howell f955991120 tools: Add bright_red_output() to tools/lint-all 2016-08-18 14:15:01 -07:00
Steve Howell ffe648baa8 tools: Extract EXCLUDED_FILES in tools/lint-all. 2016-08-18 14:15:01 -07:00
Steve Howell dfe01c4f83 tools: Move run_parallel() out of run(). 2016-08-18 14:15:01 -07:00
Steve Howell e3a7b9b1b2 tools: Move check_pyflakes() out of run(). 2016-08-18 14:15:01 -07:00
Steve Howell 3c40157195 tools: Add a run() method to tools/lint-all. 2016-08-18 14:15:01 -07:00
Steve Howell 747744ad61 lint: Allow anchor tags to span up to four lines. 
This starts to address 1533.  I still think the </p> tags
should be on their own line lined up with the start tag,
so the linter won't let through the specific example
shown in the ticket.
 2016-08-18 14:12:32 -07:00
Steve Howell 88a7ea54d2 lint: Enforce that code blocks can't split lines. 
Fixes: #1644.
 2016-08-18 14:12:26 -07:00
Christie Koehler 096b098410 Flush stdout and stderr prior to exiting child process. 
This fixes a confusing bug that we ran into where the output from
certain child processes wouldn't appear when running `lint-all` via
`ssh` (without a tty) into the Vagrant environment.
 2016-08-16 18:11:14 -07:00
Taranjeet Singh d606b95242 zulip_tools.py: Move zulip_tools.py in scripts/lib. 
This commit moves zulip_tools.py as part of cleaning the root directory
and organizing proejct into better directory structure.
 2016-08-15 16:44:50 -07:00
Tim Abbott 88a123d5e0 Fix excessive CPU usage by rabbitmq-numconsumers Nagios checks. 
The previous model for these Nagios checks was kinda crazy -- every
minute, we'd run a full `rabbitmctl list_consumers` for each of the
dozen+ consumers that we have, and then do the exact same parsing
logic for each to determine whether the target queue has a running
consumer to write out a state file.

Because `rabbitmctl list_consumers` takes a small amount of resources,
on systems where CPU is very limited (e.g. t2 style AWS instances),
this minor CPU wastage could be problematic.

Now we just do that `rabbitmqctl list_consumers` once per minute, and
output all the state files from a single command.

Further TODO items on this front include removing the hardcoded list
of queues.
 2016-08-12 14:09:36 -07:00
Tim Abbott 6496fe2a53 travis: Remove rabbitmq nodename dependency on hostname. 
Because rabbitmq doesn't support changing the nodename of a running
rabbitmq node, Zulip installations suffered a plague of issues where
e.g. a Zulip server would reboot, the hostname would change, and
suddenly the local rabbitmq instance being used by Zulip would stop
working.

We address this problem by using, by default, a fixed rabbitmq
nodename, but providing server administrators the option to set the
rabbitmq nodename used by Zulip however they choose.

To upgrade an existing server to use this new configuration, one will
need to add something like the following to /etc/zulip/zulip.conf:

[rabbitmq]
nodename = zulip@localhost

However, I don't believe we have the puppet code in place to make this
work correctly at initial installation without rabbitmq-server being
already installed (but off), as we can easily setup in Travis CI but I
haven't been willing to do for the installer.  So for now, this just
fixes our Travis CI problems.

Fixes: #1579.
 2016-08-12 09:38:23 -07:00
Tim Abbott c7059c9751 travis: Update success-http-headers to match current certs. 
Travis CI seems to have changed the way the snakeoil SSL certs are
generated in their infrastructure, so we need to update our expected
"success" HTTP headers accordingly.
 2016-08-12 09:35:41 -07:00
Tim Abbott a648513580 production-helper: Remove /root/zulip during setup process. 
This fixes a problem that caused production-helper to not be
idempotent.
 2016-08-11 22:21:13 -07:00
Tim Abbott 7011c94465 production-helper: Use ln -nsf to install snakeoil symlinks. 
This fixes a problem where production-helper was not idempotent.
 2016-08-11 22:20:24 -07:00
Tim Abbott 28a7d991f6 setup-production: Remove more postgres packages. 
Travis CI seems to have added Postgres 9.5 to the default-installed
Postgres package list, and we also missed a few side packages
previously as well.
 2016-08-11 22:19:57 -07:00
Rishi Gupta 0301347e86 provision.py: Fix typo. 
Typo in the last line of the script; fixing since we'll see it every time
we provision.
 2016-08-11 15:20:29 -07:00
Tim Abbott cb21584ffe check_send_receive_time: Support being run in a cron job. 2016-08-11 14:48:21 -07:00
Tim Abbott 4dcbaf1e6b check_send_receive_time: Cleanup unnecessary print statement. 2016-08-11 14:48:21 -07:00
Tim Abbott dca585fa75 travis: re-disable test-queue-worker-reload. 
It seems to still fail sometimes in Travis CI.
 2016-08-10 16:43:00 -07:00
Umair Khan 740b5634ac Fix and re-enable test-queue-worker-reload. 
It seems that we no longer get the message, 'zerver/lib/actions.py
modified; restarting server', but the server reloads successfully
nonetheless.

Fixes: #1341.
 2016-08-10 16:17:23 -07:00
Tomasz Kolek 4a25b92298 Cleanup result message code in test-js-with-node. 
Remove duplicate code after quick fix in passing returned code.

Fixes: #1489.
 2016-08-09 16:46:24 -07:00
Steve Howell a904222947 Add tools/find-add-class to tools/lint-all. 
The find-add-class tool, when in lint mode, verifies that we can
understand all calls to addClass from our JS code.

When in non-lint mode, i.e. verbose mode, the tool prints out a
list of tuples of (fn, class) that we can use as we wish in other
tools.
 2016-08-08 15:32:48 -07:00
Steve Howell 95b19f9765 Add test_validate_django(). 2016-08-08 15:20:29 -07:00
Steve Howell d29e745142 Add test_validate_handlebars(). 2016-08-08 15:20:29 -07:00
Steve Howell 70bc932913 Add test_html_tag_tree(). 2016-08-08 15:20:29 -07:00
Steve Howell 65147e7368 Add tests for template_parser.tokenize. 2016-08-08 15:20:29 -07:00
Steve Howell 184e45b774 Fix bug with tools/html-grep. 
We were ignoring singleton tags like "input" tags in
html-grep.  This was an artifact of our tokenizer originally
being built to check indentation of templates, for which
singleton tags had been a distraction. This fix actually cleans up
the template checking logic as well, since it can now rely
on the tokenizer to classify special tags and singleton tags.
The tokenizer is more complete and more specific.
 2016-08-08 15:20:29 -07:00
Tim Abbott b3a768f4b2 settings: Improve ALLOWED_HOSTS defaults logic and docs. 
This removes the requirement for the user to put localhost/127.0.0.1
in their ALLOWED_HOSTS list, since it is now added automatically.

Fixes: #1358.
 2016-08-05 21:25:29 -07:00
Tim Abbott 5bff72c385 Revert "Use apt-add-repository to setup Zulip PPA." 
This reverts commit 3f95e567c1.

Apparently `apt-add-repository` fails periodically in CI.  I suspect
this is some sort of silly networking problem, but given that all
we're saving is a few lines of code, the old version was better if
this fails basically ever.
 2016-08-05 13:29:17 -07:00
Tim Abbott 3f95e567c1 Use apt-add-repository to setup Zulip PPA. 2016-08-04 22:17:07 -07:00
Tim Abbott b50abc5131 npm: Split out production-only dependencies. 2016-08-04 22:17:07 -07:00
Tim Abbott 39f1024740 test_messages: Fix remaining mypy errors. 2016-08-04 15:57:03 -07:00
Tim Abbott 89267f926e Annotate tools/run-dev.py. 2016-08-04 15:57:02 -07:00
Tim Abbott 0689485666 Add lint check for malformed type annotations missing :. 
Also fix the 2 annotations that weren't being checked because of this.
 2016-08-04 15:53:23 -07:00
Tim Abbott 604e2821f8 Remove tools/deprecated/backup. 2016-08-04 11:34:53 -07:00
Steve Howell 7cc1b1ebc4 Add test coverage for parsers in tools/lib. 
Now, `tools/test-all` calls a new program called `tools/tests-tools`
that runs unit tests in `test_css_parser.py` and 'test_template_parser.py`.

This puts 100% line coverage on tools/lib/css_parser.py.
This puts about 50% line coverage on tools/lib/template_parser.py.
 2016-08-04 10:47:09 -07:00
Steve Howell 5af47e0eef Make fn param optional with template_parser.validate(). 
The caller can now pass in text directly.  This is mostly for
testing, but it could be useful in other situations.
 2016-08-04 10:47:09 -07:00
Steve Howell 1f8ba1d1b5 Make minor changes to css_parser.py 2016-08-04 10:47:09 -07:00
Tim Abbott 09b677b605 test-js-with-node: Fix istanbul not being in PATH. 2016-08-04 09:29:30 -07:00
Tim Abbott 82b1218af9 test-js-with-node: Fix return status on failure. 2016-08-04 09:27:54 -07:00
Tomasz Kolek c96722b124 Add istanbul cover info to help in test-js-with-node. 2016-08-03 14:57:03 -07:00
Tomasz Kolek 87b554906d Add clear success message after node tests run. 
Fixes: #1489.
 2016-08-03 14:56:58 -07:00
Steve Howell 0f8d014096 Add tools/html-grep. 
This tools greps for HTML nodes that have relevants words
pertaining to classes, ids, or tags in themselves or their
parents.
 2016-08-03 11:29:37 -07:00
Steve Howell 3b5c187f55 Add tools/check-css and tools/lib/css_parser.py. 
`tools/lint-all` now calls the new `tools/check-css`

The css_parser library parsers CSS into a data structure
that remembers line numbers and columns of semantically
meaningful tokens and adjoining white space/tokens.  It
is intended to be used for various linting tasks.

The file `tools/check-css` runs a few files through the
parser and makes sure they round trip.  This has some value
right away, as files that fail to parse will cause an
exception to be thrown and thus alert developers to syntax
errors.  We expect to grow this into more advanced linting
tasks eventually.
 2016-08-02 16:22:46 -07:00
Taranjeet Singh b82836a901 tools/setup/emoji_dump/emoji_dump.py: Move bitmaps & *.ttx to var/. 
Also update .gitignore to remove the old .gitignore markings for
these files' old locations.
 2016-08-02 15:18:36 -07:00
Tim Abbott 0ee3178167 Move update-deployment to scripts/deploy-zulip-from-git. 2016-08-02 14:31:07 -07:00
Tim Abbott fef32af28c update-deployment: Switch to using update-zulip-part-2. 2016-08-02 14:31:07 -07:00
Steve Howell 9151ee42e9 Extract tools/lib/template_parser.py. 2016-08-01 16:27:50 -07:00
Steve Howell 9951e92b3b Extract tokenize() in tools/check-templates. 2016-08-01 16:05:07 -07:00
Steve Howell 7e772ed644 Clean up code in check-templates. 
I extracted a few helper methods.
 2016-08-01 16:05:07 -07:00
Tim Abbott 75b5d021fa Remove unused puppet-common third-party module code. 2016-07-31 19:24:42 -07:00
Tim Abbott 143575a5bd provision: Fetch tsearch_extras package from Zulip apt repository. 
This fixes installing a Zulip development environment on Ubuntu Xenial.
 2016-07-30 21:01:07 -07:00
Steve Howell 6a65b3482c Move index.js to frontend_tests/zjsunit. 2016-07-30 14:54:30 -07:00
Tim Abbott f0bf883772 provision: Retry node_modules setup on failures. 
`npm install` fails nondeterministically occasionally, and this makes
such failures likely to be automatically resolved in most cases by
simple retrying.
 2016-07-30 10:09:23 -07:00
Tim Abbott 315766ae02 provision: Provide a nice success message. 2016-07-30 10:09:23 -07:00
Tim Abbott ed95c34b83 check-templates: Remove now-removed terms.html. 2016-07-30 09:56:54 -07:00
Steve Howell 8b13530712 Add tools/analyze-url-coverage 2016-07-29 21:23:33 -07:00
Steve Howell 21f83afe3a Add --url-coverage option to ./tools/test-backend. 2016-07-29 21:23:33 -07:00
Steve Howell 4556bf528f Add --report-slow-tests option to tools/test-backend. 2016-07-29 16:41:20 -07:00
Steve Howell f3962994b5 Fix regression for running migrations during tests. 
When running ./tools/test-backend, the script to generate
fixtures, ./tools/setup/generate-fixtures, looks for a file
called migration-status to determine whether it can short
circuit doing database migrations.  This file got moved as
part of the effort to put files in "var," but the existence
check was still looking for that file in its old location.
 2016-07-29 08:57:56 -07:00
Tim Abbott 568dd0e142 lint-all: Improve regular expression for json_error. 2016-07-28 16:19:55 -07:00
Umair Khan 1a6e8282c8 Run 'check_send_receive_time' as 'zulip' user. 
Run '/puppet/zulip/files/nagios_plugins/zulip_app_frontend/check_send_receive_time'
script as 'zulip' user so that the connection to the database can be
made correctly.
 2016-07-28 13:39:29 -07:00
Steve Howell 713797a65c Add lint checks for self.client.{get,patch,put,post,delete}. 2016-07-27 20:50:54 -07:00
Tim Abbott c2bed83a84 run-dev: Hardcode 127.0.0.1 rather than localhost. 
Using "localhost" is problematic in Travis CI, where they have
/etc/hosts configured to make `localhost` resolve to both 127.0.0.1
and ::1.
 2016-07-27 11:39:12 -07:00
Eklavya Sharma 97d877f49e tools/: Make subprocess calls unicode-aware. 2016-07-26 12:06:41 -07:00
Taranjeet Singh 1328a10069 Move available-migration and migration-status to var/. 2016-07-25 14:39:35 -07:00
Taranjeet Singh b143d6ca6e Move test_credentials.js to var/casper. 2016-07-25 14:39:23 -07:00
Eklavya Sharma 29af1419db Make tools/update-deployment pass mypy check. 2016-07-25 22:06:05 +05:30
Eklavya Sharma b08d6769c1 tools/zulip-export/zulip-export: Fix how files are opened. 
Instead of using codecs.open, use the builtin open in binary mode
and write data after encoding it in utf-8.
 2016-07-24 22:32:08 +05:30
Eklavya Sharma df7dcdda5f Annotate tools/get-handlebar-vars. 2016-07-24 22:12:32 +05:30
Eklavya Sharma 61ffd835e1 Annotate tools/minify-js. 2016-07-24 21:58:02 +05:30
Eklavya Sharma 1e0a971178 Annotate tools/test-run-dev. 2016-07-24 21:33:40 +05:30
Eklavya Sharma 7edc38bbf7 Remove tools/test-js-with-casper from mypy exclude list. 2016-07-24 21:07:07 +05:30
Eklavya Sharma e6264fa6fb Annotate tools/compile-handlebars-templates. 2016-07-24 21:07:06 +05:30
Eklavya Sharma 1a77b599f6 Make tools/post-receive python 3 compatible. 2016-07-24 21:07:06 +05:30
Eklavya Sharma 00a120c34f Improve a string in tools/post-receive. 2016-07-24 21:07:02 +05:30
Eklavya Sharma ab8890b304 Annotate tools/check-templates. 2016-07-24 21:04:32 +05:30
Eklavya Sharma 2f889550cb tools/check-templates: Use __init__ in Record. 
Initialize Record by using __init__ instead of setting attributes
in validate.  This is needed because mypy complains when we set
new attributes outside __init__.
 2016-07-24 21:04:32 +05:30
Eklavya Sharma ecf2249536 tools/check-templates: Set modified's default to False. 
Set default value of options.modified to False.
 2016-07-24 21:04:31 +05:30
Eklavya Sharma 9c177fcd1b tools/check-templates: Remove unneeded return. 2016-07-24 21:04:31 +05:30
Eklavya Sharma bd0fa3e77b Annotate tools/lint-all. 2016-07-24 21:04:31 +05:30
Eklavya Sharma ec6b630537 Make tools/run-mypy pass mypy check. 2016-07-24 21:04:22 +05:30
Eklavya Sharma 3d87c376c2 Make zulip-ec2-configure-interfaces pass mypy check. 2016-07-24 12:52:58 +05:30
Eklavya Sharma 0a5aa2ccc7 Make check_send_receive_time pass mypy check. 2016-07-24 12:41:06 +05:30
Eklavya Sharma 1820f961b3 frontend_tests/run-casper: Fix annotations. 2016-07-24 10:52:54 +05:30
Taranjeet Singh 5e34e5bf40 iframe-bot/show-last-messages: Fix line with length greater than 120. 2016-07-23 14:33:11 -07:00
Eklavya Sharma 11732f9ab0 Make all scripts in scripts/ pass mypy check. 2016-07-24 00:17:21 +05:30
Tim Abbott 226a57d2c8 travis: Disable test-queue-worker-reload temporarily. 
It's failing nondetermistically pretty often, and I'd like to cut the
impact to other users' work.
 2016-07-22 11:36:52 -07:00
Eklavya Sharma fb59a07a89 Run mypy on extensionless python scripts in Travis. 2016-07-22 11:16:29 -07:00
Eklavya Sharma f4c557d2a7 tools/run-mypy: Exclude scripts which don't pass mypy. 2016-07-22 11:16:29 -07:00
Eklavya Sharma c12db5246d Use separate exclude list for scripts. 2016-07-22 11:16:29 -07:00
Eklavya Sharma 102fcda4ab tools/run-mypy: Add option --scripts-only. 
This option is for checking only extensionless python scripts.
 2016-07-22 11:16:29 -07:00
Eklavya Sharma baa157344c Patch activate script only in development. 2016-07-21 14:11:41 -07:00
Tim Abbott 30dd8cfd4a test-queue-worker-reload: Increase sleep resolution. 2016-07-21 10:21:29 -07:00
Umair Khan 7d687b0f79 Increase the timeout in test-queue-worker-reload. 
May fix #1341.
 2016-07-21 10:19:49 -07:00
Eklavya Sharma 7bd935bef2 Add option to only show extensionless files in lister. 2016-07-21 16:59:52 +05:30
Taranjeet Singh 44d23975ed Move update-prod-static.log to var/log. 2016-07-20 18:27:44 -07:00
Eklavya Sharma 6548f1dd1c Factor out venv-installing code into a module. 
Factor out the code in tools/provision.py which installs a python2
and python3 venv into a module (tools/setup/setup_venvs.py) which
can also be used as a script.
 2016-07-20 18:20:37 -07:00
Eklavya Sharma 2930a769a9 Add __init__.py to tools/ and tools/setup/. 
This will make it possible to import stuff from tools/setup/.
 2016-07-20 18:20:37 -07:00
Tim Abbott a07eca2639 Revert "Factor out venv-installation into a script." 
This reverts commit 852c49a44e.

I think this may have broken provisioning without a venv.
 2016-07-20 14:55:45 -07:00
Eklavya Sharma 852c49a44e Factor out venv-installation into a script. 2016-07-20 14:18:51 -07:00
Tim Abbott 741c21c3ce update_deployment: Make git repository URL configurable. 2016-07-19 21:02:47 -07:00
Tim Abbott a5a79280cb update_deployment: Add symlink for use in voyager installs. 2016-07-19 20:59:59 -07:00
Tim Abbott 35339f5117 Rename local_settings_template to prod_settings_template. 2016-07-19 20:59:59 -07:00
Tim Abbott 19b860ceec Rename local_settings.py symlink to prod_settings.py. 2016-07-19 20:59:59 -07:00
Tim Abbott 82f9f8f941 update-deployment: Compute ZULIP_COM setting. 2016-07-19 20:52:08 -07:00
Tim Abbott 7348841e65 update-prod-static: Copy static/locale rather than moving. 
The old behavior results in a large diff when deploying based on a git
commit.
 2016-07-19 20:52:08 -07:00
Tim Abbott 28fa4e8346 build_release_tarball: Remove local_settings.py overwrite hack. 2016-07-19 20:52:06 -07:00
Tim Abbott 78a0c7c557 tools: Rewrite install-server to not hardcode configuration. 
Now install-aws-server is a reusable script for setting up a Zulip
role server in AWS, without any hardcoded configuration.
 2016-07-19 20:12:43 -07:00
Eklavya Sharma d745f20b1b Run frontend and backend tests in python3 on Travis. 2016-07-19 14:15:35 -07:00
Eklavya Sharma fd007c4554 Make tools/test-run-dev python3 compatible. 
Use universal_newlines=True with subprocess.check_output in
tools/test-run-dev.
 2016-07-19 14:15:35 -07:00
Eklavya Sharma e86539649c Use tools/travis/activate-venv in test suites. 2016-07-19 14:15:35 -07:00
Eklavya Sharma 5054a8d6c9 tools/run-dev.py: Use twisted venv if on python 3. 2016-07-19 14:15:35 -07:00
Eklavya Sharma cfed816a52 tools/provision.py: Install a python2 venv with twisted. 
Twisted is not python 3 compatible.  So for now create a python2
venv and install twisted in it when running provision.py in python3
mode and use twisted from the python2 venv.
 2016-07-19 14:15:35 -07:00
Eklavya Sharma 9c66cb7130 tools/provision.py: Install both py2 and py3 venvs. 
Install both python2 and python3 venvs in tools/provision.py by
default.  Use old behavior when run with --travis to save time.
 2016-07-19 23:39:50 +05:30
Eklavya Sharma eb43f7f581 tools/provision.py: Factor out "--travis" in sys.argv. 2016-07-19 23:39:50 +05:30
Tim Abbott 13c2effb61 update-deployment: Make zulip-dropbox.png logic conditional. 2016-07-19 10:01:14 -07:00
Tim Abbott 6b0d8ecfba update-deployment: Run create-production-venv as root. 2016-07-19 09:51:13 -07:00
Tim Abbott 07473f4007 update-prod-static: Use a virtualenv to find dependencies. 
This is needed to support using update-deployment with a virtualenv.
 2016-07-19 09:51:13 -07:00
Eklavya Sharma 38b1353f42 tools/run-dev.py: Clear memcached. 
Clear memcached when tools/run-dev.py is run.  This prevents
errors on using a different python version because values are
pickled before being stored in memcached and different python
versions implement pickling differently.

Also provide a command-line option --no-clear-mc to prevent
memcached from being cleared.
 2016-07-18 14:25:13 -07:00
Taranjeet Singh 64332d8816 Update linecoverage directory to linecoverage-report. 
tools/provision.py: Create directory var/linecoverage-report.

tools/run-mypy: Update coverage dir to var/linecoverage-report.
 2016-07-18 14:13:33 -07:00
Taranjeet Singh 4d2cb3754c Update upload dir to var/uploads. 
tools/provision.py: Create directory var/uploads.

zproject/local_settings_template.py: Update Upload dir to var/uploads.

zproject/dev_settings.py: Update upload dir to var/uploads.
 2016-07-18 14:13:33 -07:00
Umair Khan b546391f0b Move locale to static/locale 2016-07-18 13:59:07 -07:00
Eklavya Sharma bc901ac6d8 tools/travis/py3-backend: Add tools/test-management. 2016-07-18 09:57:28 -07:00
Eklavya Sharma da36947400 Change unbuffering strategy in runtornado.py. 
runtornado unbuffers its output using
sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0).
This is not python 3 compatible since we can't specify
buffering on a text stream in python 3.  So use the '-u'
option of python when calling runtornado.py to make output
unbuffered.
 2016-07-17 10:31:15 -07:00
Taranjeet Singh 9db457e8fa provision.py: Create var/coverage directory to store coverage data. 
test-backend: Update coverage directory to var/coverage.

This commit updates the coverage directory to var/coverage as a part
of Issue 1132.
 2016-07-13 18:58:00 -07:00
Tim Abbott 6254e29ebf update-deployment: Use the virtualenv in deployment process. 2016-07-13 18:56:44 -07:00
Tim Abbott 599f12f94f update-prod-static: Fix deploy path for language_options.json. 2016-07-13 18:56:37 -07:00
Tim Abbott baeaf0f870 check-templates: Fix traceback with missing closing tag at end of file. 
Previously, this would crash with an unclear traceback in this situation.
 2016-07-13 18:55:40 -07:00
Tim Abbott c0a6672471 Update path to language_options.json in production. 
The previous code didn't correctly transport language_options.json to
the production environment.
 2016-07-13 12:30:45 -07:00
Taranjeet Singh 5971203864 settings: Store uploaded files under var/ in development environment. 2016-07-12 20:33:31 -07:00
Taranjeet Singh 03384deb86 provision: Create zulip/var/log directory. 
The purpose of this is to move a lot of the log and other generated
files used by the Zulip development environment into a consistent
hierarchy.

We also need to create this in tools/build-release-tarball as well,
since that runs a development environment out of a temporary
directory.
 2016-07-12 20:33:30 -07:00
Tim Abbott 2338421c6d lint: Add documentation lint check for JavaScript spelling. 2016-07-12 19:22:21 -07:00
Tim Abbott 4972154df9 setup-production: Improve debugging of build-release-tarball errors. 2016-07-12 19:09:28 -07:00
Tim Abbott 0b7788be6b Fix indentation in compose.html. 2016-07-12 18:36:02 -07:00
Tim Abbott d424813687 Fix indentation in deactivated.html. 2016-07-12 18:34:17 -07:00
Steve Howell f57a17abdf Lint-check more Django files for indentation. 
(This starts to address github ticket #1236.)
 2016-07-12 14:08:17 -07:00
Steve Howell 97f28f3792 Revert "Close HTML singleton tags in Casper files." 
This reverts commit 520b255d95,
and also blacklists Casper files from our linter.
 2016-07-12 13:35:50 -07:00
Tim Abbott 5d990c28d0 Fix running run-mypy from any working directory. 2016-07-12 10:39:33 -07:00
Tim Abbott 497142d7b1 Fix running test-backend from any working directory. 2016-07-12 10:39:33 -07:00
Eklavya Sharma 934a0f7c6c Add tools/test-backend to py3-backend test suite. 2016-07-12 09:27:55 -07:00
Eklavya Sharma 5de91c4115 Use subprocess_text_output in tools/provision.py. 2016-07-12 20:25:20 +05:30
Eklavya Sharma 158d67e702 Run tools/test-migrations in Travis. 2016-07-11 21:30:32 -07:00
Eklavya Sharma 57ce3f4af1 tools/travis/lint-all: Factor out error message. 2016-07-11 21:30:32 -07:00
Eklavya Sharma 08a4555e0f Merge mypy and py3k test suites in Travis. 2016-07-11 21:28:01 -07:00
Eklavya Sharma 0de3b17f19 tools/test-backend: Add option to skip generate-fixtures. 
Not calling generate-fixtures can reduce running time, which is
especially helpful when running a single test.
 2016-07-12 09:34:37 +05:30
Tim Abbott f513a68ac9 lint-all: Check for missing whitespace before { in CSS. 
Also fix the existing violations of this rule.
 2016-07-10 17:29:36 -07:00
Tim Abbott 88368397aa lint-all: Add Python check for space after if. 2016-07-10 10:58:16 -07:00
Steve Howell 538bc61c54 Check more html files in check-templates. 
Instead of just checking .html files in the templates directory,
we now also check them everywhere, including .html files used
for Casper tests and some static files.
 2016-07-09 17:34:49 -07:00
Steve Howell 704c57a141 Fix end tag issues in api.html 
(This fixes a minor style issue with the API keys/bots section,
but the change is mostly to make the HTML have balanced tags.)
 2016-07-08 16:30:49 -07:00
Steve Howell e35b84d438 Have lint-all pass along -m to check-templates. 2016-07-08 16:30:49 -07:00
Steve Howell 6154c73125 Support -m flag to check only modified templates. 2016-07-08 16:30:49 -07:00
Steve Howell d1964a243e Use lister module to find files for linting. 2016-07-08 16:30:49 -07:00
Steve Howell 1006b95898 Extract check_handlebar_templates. 2016-07-08 16:30:49 -07:00
Steve Howell 4e326ed138 Extract check_django_templates. 2016-07-08 16:30:49 -07:00
Steve Howell 3e9ceaeaf0 Move templates check to ok() function. 2016-07-08 16:30:49 -07:00
Tim Abbott b13eeae24c Remove finbot from Zulip repository. 
It's been split into its own repository,
https://github.com/zulip/finbot.
 2016-07-08 16:15:55 -07:00
Eklavya Sharma aa68fd1679 Run tools/lint-all on Travis in python 3. 2016-07-08 10:43:48 -07:00
Tomasz Kolek c15695e514 Add support for running test-js-with-node on particular files. 
Fixed: #1127.
 2016-07-07 14:32:07 -07:00
Eklavya Sharma 4cf7641ab1 zerver/tests/test_i18n.py: Ignore due to incomplete stubs. 
In python 3, http.cookies has incomplete stubs.
 2016-07-07 12:42:51 -07:00
Eklavya Sharma 6c3f1bb967 beanstalk.py: Encode and decode strings correctly. 2016-07-07 12:42:51 -07:00
Eklavya Sharma 26b8e7357a zerver/views/messages.py: Operate on bytes in highlight_string. 2016-07-07 12:42:51 -07:00
Eklavya Sharma 83640ed0cd runtornado.py: Ignore due to incorrect stubs. 2016-07-07 10:09:35 -07:00
Eklavya Sharma 6fd8906358 rename_stream.py: Fix broken code. 
* get_realm returns None if no matching realm is present, but
  create_stream.py assumed it raises Realm.DoesNotExist.
* encoded/decode strings properly.
 2016-07-07 10:08:29 -07:00
Eklavya Sharma c679c180f5 enqueue_file.py: Add type hint for mypy. 2016-07-07 10:07:51 -07:00
Eklavya Sharma 896c18a57b email-mirror.py: Make it pass on mypy in python 3. 
* Replace filter by list comprehension.

* Add '# type: ignore' to statements which use attributes from
  modeule `posix`, since stubs for posix are missing on python 3.
 2016-07-07 10:07:28 -07:00
Eklavya Sharma 17cb6e00bd create_stream.py: Fix broken code. 
* get_realm returns None if no matching realm is present, but
  create_stream.py assumed it raises Realm.DoesNotExist.
* encoded/decode strings properly.
 2016-07-07 10:06:39 -07:00
Eklavya Sharma 63d55bdd86 zerver/views/__init__.py: decode b64encoded ccache. 
Convert b64encoded ccache to `str` before passing to
subprocess.check_call.
 2016-07-07 10:02:08 -07:00
Eklavya Sharma 628e45defc zerver/lib/actions.py: Use text_type in truncate_ functions. 
The functions truncate_content, truncate_body and truncate_topic
are only meant to be used on text strings.  So change its
parameter types from AnyStr to text_type.
 2016-07-07 10:02:08 -07:00
Eklavya Sharma 976858f536 tornado_ioloop_logging.py: Ignore because of missing stub. 
There is no stub for select.epoll on python 3.  So ignore the
statement which uses it.
 2016-07-07 10:02:08 -07:00
Eklavya Sharma 9d2a539aaa zerver/lib/notifications.py: Add hints for mypy. 
Add type hints for mypy using isinstance in assert and if.
 2016-07-07 10:02:08 -07:00
Eklavya Sharma 5e81a4d93f zerver/lib/email_mirror.py: Improve annotation in python 3. 
Add asserts and if statements to help mypy.
 2016-07-07 10:01:30 -07:00
Eklavya Sharma 06a7a6caee zerver/lib/debug.py: Remove from mypy's exclude_py3. 
traceback.print_stack doesn't have a stub yet.  So ignore the
statement which uses it.
 2016-07-07 09:52:24 -07:00
Eklavya Sharma 6bb266d262 bugdown's __init__.py: Add python 3 compatibility. 
* Use Response.text instead of Response.content.
* Make unescaping work on python 3.
 2016-07-07 09:52:24 -07:00
Eklavya Sharma 05046d9288 bugdown's codehilite.py: Fix annotations in python 3 mode. 
Many stubs in xml.etree.ElementTree use Union[str, bytes] as
return type.  Mypy wants us to correctly handle each case.  This
is correct, but not useful for us since we know that we'll always
get str.  So force the return value to text_type, to supress mypy
errors.
 2016-07-07 09:52:24 -07:00
Eklavya Sharma 610f19c791 zerver/lib/camo.py: Type ignore statement with hex encode. 2016-07-07 09:52:24 -07:00
Eklavya Sharma 6a63870136 zerver/decorator.py: Correctly encode/decode strings. 2016-07-07 09:52:24 -07:00
Eklavya Sharma 4e698ab1f6 Remove zerver/lib/parallel.py from mypy's exclude_py3. 2016-07-07 09:52:24 -07:00
Eklavya Sharma 5e8dba4c75 Add mypy in python 3 mode to travis. 
This helps catch Python 3 compatibility issues.
 2016-07-07 09:44:00 -07:00
Taranjeet a137bf15ed Wrap some lines with length greater than 120. 
With some tweaks by tabbott.
 2016-07-06 14:35:16 -07:00
Eklavya Sharma 4f181acb83 Add python 3 mode to provision.py. 
When provision.py is run using python 2, retain original behavior
of creating a python 2 venv using requirements/py2_dev.txt and
creating a python 3 venv using requirements/mypy.txt.
When provision.py is run using python 3, install a single python 3
venv with requirements/py3_dev.txt.
 2016-07-06 13:50:55 -07:00
Eklavya Sharma cdc067e751 tools/provision.py: Use universal_newlines=True. 2016-07-06 13:50:55 -07:00
Eklavya Sharma ce6ddd574a zerver/lib/test_helpers.py: Replace os.path.walk by os.walk. 
os.path.walk has been removed in python 3.
 2016-07-06 11:49:55 -07:00
Eklavya Sharma 6505cbf2bf zerver/lib/timeout.py: Replace isAlive by is_alive. 
In threading.Thread, isAlive has been removed in python 3.
We should use is_alive instead.
 2016-07-06 11:49:48 -07:00
Eklavya Sharma 0b714ea6c6 Use python2-specific requirements files. 2016-07-02 11:53:26 -07:00
Eklavya Sharma dcd568960b Add files to mypy's python 3 exclude list. 2016-07-02 10:38:48 -07:00
Eklavya Sharma b44ae38bff tools/run-mypy: Add Python 3 mode. 2016-07-02 10:38:48 -07:00
Tim Abbott 3b16daad18 provision: Add a shebang line. 2016-06-30 22:34:58 -07:00
Tim Abbott a78c1b9750 Fix casper tests hanging while development server is running. 
This works around a nasty problem with Webpack that you can't run two
copies of the Webpack development server on the same project at the
same time (even if on different ports).  The second copy doesn't fail,
it just hangs waiting for some lock, which is confusing; but even if
that were to be solved, we don't actually need the webpack development
server running to run the Casper tests; we just need bundle.js built.
So the easy solution is to just run webpack manually and be sure to
include bundle.js in the JS_SPECS entry.

As a follow-up to this change, we should clean up how test_settings.py
is implemented to not require duplicating code from settings.py.

Fixes #878.
 2016-06-30 14:18:50 -07:00
Vishnu Ks e4b72c3a65 Move respond_to_message to compose.js. 2016-06-29 13:04:02 -07:00
Tim Abbott 5ce0637da9 run-mypy: Explode exclude lists to specific files. 2016-06-28 16:42:59 -07:00
Tim Abbott 56e7a2f6f3 Annotate zerver.migrations. 2016-06-28 16:38:33 -07:00
Tim Abbott c8aae360be test-backend: Fix spelling of an example in usage text. 2016-06-28 16:24:58 -07:00
Tim Abbott 7ed0ab8c4a Ensure Zulip virtualenvs are world-readable. 
This is important for both ensuring the Nagios checks work correctly
in production, as well as making sure the `zulip` user can access the
virtualenv (owned by the `travis` user) in Travis CI.
 2016-06-28 12:03:01 -07:00
Tim Abbott 40aa8a2336 clean-venv-cache: Fix python 3 compatibility. 2016-06-27 20:39:22 -07:00
Tim Abbott 404c61ba97 clean-venv-cache: Use better approach to pruning caches. 
Rather than looking at which venvs are used by this particular build,
we instead look at which venvs have a hash that is the hash_reqs value
of a current requirements.txt file.
 2016-06-27 20:30:39 -07:00
Tim Abbott 7d5c1864f7 clean-venv-cache: Avoid cleaning production venvs. 2016-06-27 20:08:45 -07:00
Tim Abbott beb8b50623 Move provision.py under tools/. 2016-06-27 19:04:32 -07:00
Tim Abbott e1860e5b46 Remove tools/provision/zulip-dev.conf. 
This was originally supervisord configuration for the Zulip
development environment; it never really worked how we wanted it to.
 2016-06-27 19:04:32 -07:00
acrefoot 7ff89dc137 Add markdown-specific whitespace lint rule to /tools/lint-all. 
Match one space or three+ spaces after content, or positive
number of spaces on an empty line.
 2016-06-27 18:43:20 -07:00
Aakash Tyagi 9154d93669 Added help text for running single backend tests. 2016-06-27 14:15:30 -07:00
Tim Abbott 101820bc29 Move docker development environment scripts to tools/. 
scripts/ is for scripts which are used in production.
 2016-06-27 13:38:06 -07:00
rahuldeve 3ca53df152 Add test for worker queue autoreloading. 2016-06-26 20:12:37 -07:00
Tim Abbott 7d3b451902 lint: Check for extremely long lines. 
Currently, we check for lines longer than 180 characters; we can lower
this as we clean up or wrap longer strings.
 2016-06-25 10:52:03 -07:00
Umair Khan a70d5041d2 Don't allow literal strings in report_error. 2016-06-23 16:23:01 -07:00
Umair Khan b7bb49c6f5 Don't allow literal string in report_success. 2016-06-23 16:23:01 -07:00
Tim Abbott 039c175d68 production-helper: Hold tons of packages. 
This saves almost a minute doing apt upgrades in the production test
suite.
 2016-06-22 10:41:09 -07:00
Tim Abbott 9b436c7190 setup-production: Remove more irrelevant postgres packages. 2016-06-22 10:41:09 -07:00
Tim Abbott 5a109cf816 travis: Skip development database setup in production tests. 
This saves about 20s on the runtime of the production build.
 2016-06-22 10:41:09 -07:00
Tim Abbott 0f2729f5fb production-helper: use dist-upgrade to match install script. 
Previously, we were wasting time every time we installed packages,
because `apt-get upgrade` would only install most of the packages
`apt-get dist-upgrade` would.
 2016-06-22 10:38:27 -07:00
Eklavya Sharma 121a5f26fb Move hash_reqs.py from tools/ to scripts/lib/. 
This is needed because hash_reqs.py is used to create a virtualenv.
Currently we only use virtualenv in development, but we will soon
start using it in production.  Scripts used in production should be
put in scripts/.
 2016-06-22 18:12:08 +05:30
Tim Abbott 6b8e9c7254 Exclude static/locale from linters. 2016-06-21 17:17:34 -07:00
Tim Abbott 50f723f50b Split test_narrow.py out of test_messages.py. 2016-06-21 12:25:08 -07:00
Alex Wilson f68a392250 Run django.setup() after coverage.start() in tests to fix coverage report. 
Fixes #1007.
 2016-06-21 10:37:45 -07:00
Eklavya Sharma 8148cbe173 Add tools/hash_reqs.py. 
tools/hash_reqs.py generates a hash of a requirements file.  It
does that by generating a sorted list of unique dependencies referred
to by that requirements file.  To do that, it also recurses into other
requirements files specified inside that requirements file.
 2016-06-20 11:09:20 -07:00
Tim Abbott eb71173be3 lint: Fix warnings under confirmation/. 2016-06-20 08:19:54 -07:00
Tim Abbott 3899b4c913 Move py3k requirements to requirements/py3k.txt. 2016-06-18 16:41:40 -07:00
Tim Abbott 5c92639f81 Move mypy requirements to requirements/ directory. 2016-06-18 16:41:40 -07:00
Tim Abbott 6d1d7471e6 Refactor out zproject/dev_settings.py. 2016-06-17 12:15:15 -07:00
Tim Abbott d117ec8664 lint: Fix redundant stripping of strings. 2016-06-17 10:41:39 -07:00
Tim Abbott 267a71cf20 Run Python custom RE checks in parallel with non-python. 
This saves about 1s (out of originally 3.3s) of time running all the
linters in my development environment.
 2016-06-17 10:26:50 -07:00
Tim Abbott 5b05644c95 Run mypy on zerver/tests/test_hooks.py. 2016-06-16 14:07:34 -07:00
Tim Abbott 6723525fd3 Annotate zerver/tests/tests.py. 2016-06-16 14:07:34 -07:00
John Hergenroeder 16a19226f6 Add linter check for redundant REQ whence argument. 2016-06-16 13:53:39 -07:00
Tim Abbott 0071731915 test-run-dev: Display original error messages on failures. 
This helped with debugging a failure recently.
 2016-06-15 22:44:04 -07:00
Tim Abbott 39a8c82957 travis: Remove verbose coverage report from end of build output. 2016-06-15 15:05:38 -07:00
Eklavya Sharma 553a9d0b75 tools/run-mypy: Exclude py files which have a pyi. 
If a .py file has a corresponding .pyi stub file, exclude that
.py file from mypy.
 2016-06-15 22:40:31 +05:30
Tim Abbott 9e6b9dacf6 test-migrations: Fix missing exit status on error. 
Previously this test was correctly detecting missing migrations, but
not causing the build to fail.
 2016-06-15 09:26:04 -07:00
Umair Khan c1f5ac375c Add linter checks for javascript strings and HTML placeholders. 
Checks are added for:
- i18n.t
- placeholder
- compose_error
 2016-06-13 09:05:27 -07:00
Tim Abbott 062287d0b0 Coveralls: Add coverage reporting to backend test suite. 2016-06-10 17:20:00 -07:00
Tim Abbott 584887e588 lint: Require folding of } on same line in else if statements. 2016-06-09 14:05:34 -07:00
Tim Abbott c35781d505 lint: Require folding of } on same line as else statements. 2016-06-09 14:02:49 -07:00
Tim Abbott 0c1b5006f7 lint: Check for space after if in javascript. 2016-06-09 13:47:12 -07:00
Tim Abbott eba0d6339f lint: Require space after // in JS comments. 2016-06-09 13:44:24 -07:00
Tim Abbott a09b950097 lint-all: Check for relative URLs in JSON calls from JS. 2016-06-09 11:59:40 -07:00
Umair Khan 938291a922 Add run-dev.py test to Travis CI test suites. 
Fixes #919
 2016-06-08 14:10:37 +05:00
Vishnu Ks d30ea0bc44 Move recent_subjects to stream_data. 2016-06-07 17:50:16 -07:00
Tim Abbott be53c9e39e tests: Check for presence of typing in venv guard. 
Since relatively few systems have the typing module, this makes the
checks for whether the user is properly running our test scripts in
the virtualenv more likely to trigger well.
 2016-06-06 13:41:34 -07:00
Ernesto Vargas ffb2f9e84b lint/tests: Give nice error message for common import failures. 
This should make users much more likely to be able to debug issues
where they ran Zulip outside the Vagrant environment or virtualenv.

[error messages tweaked by tabbott]
 2016-06-06 13:39:26 -07:00
Tim Abbott 553ef81f92 Add lint rules for mypy type annotations. 2016-06-05 13:00:55 -07:00
James Porter 2a8e8129d1 Make emoji_dump delete old ttx files on start. 2016-06-04 17:05:48 -07:00
James Porter f5f2d72178 Wrap main bit of emoji_dump.py in a main function. 2016-06-04 17:05:48 -07:00