Commit Graph

1230 Commits

Author SHA1 Message Date
Tim Abbott cb21584ffe check_send_receive_time: Support being run in a cron job. 2016-08-11 14:48:21 -07:00
Tim Abbott 4dcbaf1e6b check_send_receive_time: Cleanup unnecessary print statement. 2016-08-11 14:48:21 -07:00
Tim Abbott dca585fa75 travis: re-disable test-queue-worker-reload.
It seems to still fail sometimes in Travis CI.
2016-08-10 16:43:00 -07:00
Umair Khan 740b5634ac Fix and re-enable test-queue-worker-reload.
It seems that we no longer get the message, 'zerver/lib/actions.py
modified; restarting server', but the server reloads successfully
nonetheless.

Fixes: #1341.
2016-08-10 16:17:23 -07:00
Tomasz Kolek 4a25b92298 Cleanup result message code in test-js-with-node.
Remove duplicate code after quick fix in passing returned code.

Fixes: #1489.
2016-08-09 16:46:24 -07:00
Steve Howell a904222947 Add tools/find-add-class to tools/lint-all.
The find-add-class tool, when in lint mode, verifies that we can
understand all calls to addClass from our JS code.

When in non-lint mode, i.e. verbose mode, the tool prints out a
list of tuples of (fn, class) that we can use as we wish in other
tools.
2016-08-08 15:32:48 -07:00
Steve Howell 95b19f9765 Add test_validate_django(). 2016-08-08 15:20:29 -07:00
Steve Howell d29e745142 Add test_validate_handlebars(). 2016-08-08 15:20:29 -07:00
Steve Howell 70bc932913 Add test_html_tag_tree(). 2016-08-08 15:20:29 -07:00
Steve Howell 65147e7368 Add tests for template_parser.tokenize. 2016-08-08 15:20:29 -07:00
Steve Howell 184e45b774 Fix bug with tools/html-grep.
We were ignoring singleton tags like "input" tags in
html-grep.  This was an artifact of our tokenizer originally
being built to check indentation of templates, for which
singleton tags had been a distraction. This fix actually cleans up
the template checking logic as well, since it can now rely
on the tokenizer to classify special tags and singleton tags.
The tokenizer is more complete and more specific.
2016-08-08 15:20:29 -07:00
Tim Abbott b3a768f4b2 settings: Improve ALLOWED_HOSTS defaults logic and docs.
This removes the requirement for the user to put localhost/127.0.0.1
in their ALLOWED_HOSTS list, since it is now added automatically.

Fixes: #1358.
2016-08-05 21:25:29 -07:00
Tim Abbott 5bff72c385 Revert "Use apt-add-repository to setup Zulip PPA."
This reverts commit 3f95e567c1.

Apparently `apt-add-repository` fails periodically in CI.  I suspect
this is some sort of silly networking problem, but given that all
we're saving is a few lines of code, the old version was better if
this fails basically ever.
2016-08-05 13:29:17 -07:00
Tim Abbott 3f95e567c1 Use apt-add-repository to setup Zulip PPA. 2016-08-04 22:17:07 -07:00
Tim Abbott b50abc5131 npm: Split out production-only dependencies. 2016-08-04 22:17:07 -07:00
Tim Abbott 39f1024740 test_messages: Fix remaining mypy errors. 2016-08-04 15:57:03 -07:00
Tim Abbott 89267f926e Annotate tools/run-dev.py. 2016-08-04 15:57:02 -07:00
Tim Abbott 0689485666 Add lint check for malformed type annotations missing :.
Also fix the 2 annotations that weren't being checked because of this.
2016-08-04 15:53:23 -07:00
Tim Abbott 604e2821f8 Remove tools/deprecated/backup. 2016-08-04 11:34:53 -07:00
Steve Howell 7cc1b1ebc4 Add test coverage for parsers in tools/lib.
Now, `tools/test-all` calls a new program called `tools/tests-tools`
that runs unit tests in `test_css_parser.py` and 'test_template_parser.py`.

This puts 100% line coverage on tools/lib/css_parser.py.
This puts about 50% line coverage on tools/lib/template_parser.py.
2016-08-04 10:47:09 -07:00
Steve Howell 5af47e0eef Make fn param optional with template_parser.validate().
The caller can now pass in text directly.  This is mostly for
testing, but it could be useful in other situations.
2016-08-04 10:47:09 -07:00
Steve Howell 1f8ba1d1b5 Make minor changes to css_parser.py 2016-08-04 10:47:09 -07:00
Tim Abbott 09b677b605 test-js-with-node: Fix istanbul not being in PATH. 2016-08-04 09:29:30 -07:00
Tim Abbott 82b1218af9 test-js-with-node: Fix return status on failure. 2016-08-04 09:27:54 -07:00
Tomasz Kolek c96722b124 Add istanbul cover info to help in test-js-with-node. 2016-08-03 14:57:03 -07:00
Tomasz Kolek 87b554906d Add clear success message after node tests run.
Fixes: #1489.
2016-08-03 14:56:58 -07:00
Steve Howell 0f8d014096 Add tools/html-grep.
This tools greps for HTML nodes that have relevants words
pertaining to classes, ids, or tags in themselves or their
parents.
2016-08-03 11:29:37 -07:00
Steve Howell 3b5c187f55 Add tools/check-css and tools/lib/css_parser.py.
`tools/lint-all` now calls the new `tools/check-css`

The css_parser library parsers CSS into a data structure
that remembers line numbers and columns of semantically
meaningful tokens and adjoining white space/tokens.  It
is intended to be used for various linting tasks.

The file `tools/check-css` runs a few files through the
parser and makes sure they round trip.  This has some value
right away, as files that fail to parse will cause an
exception to be thrown and thus alert developers to syntax
errors.  We expect to grow this into more advanced linting
tasks eventually.
2016-08-02 16:22:46 -07:00
Taranjeet Singh b82836a901 tools/setup/emoji_dump/emoji_dump.py: Move bitmaps & *.ttx to var/.
Also update .gitignore to remove the old .gitignore markings for
these files' old locations.
2016-08-02 15:18:36 -07:00
Tim Abbott 0ee3178167 Move update-deployment to scripts/deploy-zulip-from-git. 2016-08-02 14:31:07 -07:00
Tim Abbott fef32af28c update-deployment: Switch to using update-zulip-part-2. 2016-08-02 14:31:07 -07:00
Steve Howell 9151ee42e9 Extract tools/lib/template_parser.py. 2016-08-01 16:27:50 -07:00
Steve Howell 9951e92b3b Extract tokenize() in tools/check-templates. 2016-08-01 16:05:07 -07:00
Steve Howell 7e772ed644 Clean up code in check-templates.
I extracted a few helper methods.
2016-08-01 16:05:07 -07:00
Tim Abbott 75b5d021fa Remove unused puppet-common third-party module code. 2016-07-31 19:24:42 -07:00
Tim Abbott 143575a5bd provision: Fetch tsearch_extras package from Zulip apt repository.
This fixes installing a Zulip development environment on Ubuntu Xenial.
2016-07-30 21:01:07 -07:00
Steve Howell 6a65b3482c Move index.js to frontend_tests/zjsunit. 2016-07-30 14:54:30 -07:00
Tim Abbott f0bf883772 provision: Retry node_modules setup on failures.
`npm install` fails nondeterministically occasionally, and this makes
such failures likely to be automatically resolved in most cases by
simple retrying.
2016-07-30 10:09:23 -07:00
Tim Abbott 315766ae02 provision: Provide a nice success message. 2016-07-30 10:09:23 -07:00
Tim Abbott ed95c34b83 check-templates: Remove now-removed terms.html. 2016-07-30 09:56:54 -07:00
Steve Howell 8b13530712 Add tools/analyze-url-coverage 2016-07-29 21:23:33 -07:00
Steve Howell 21f83afe3a Add --url-coverage option to ./tools/test-backend. 2016-07-29 21:23:33 -07:00
Steve Howell 4556bf528f Add --report-slow-tests option to tools/test-backend. 2016-07-29 16:41:20 -07:00
Steve Howell f3962994b5 Fix regression for running migrations during tests.
When running ./tools/test-backend, the script to generate
fixtures, ./tools/setup/generate-fixtures, looks for a file
called migration-status to determine whether it can short
circuit doing database migrations.  This file got moved as
part of the effort to put files in "var," but the existence
check was still looking for that file in its old location.
2016-07-29 08:57:56 -07:00
Tim Abbott 568dd0e142 lint-all: Improve regular expression for json_error. 2016-07-28 16:19:55 -07:00
Umair Khan 1a6e8282c8 Run 'check_send_receive_time' as 'zulip' user.
Run '/puppet/zulip/files/nagios_plugins/zulip_app_frontend/check_send_receive_time'
script as 'zulip' user so that the connection to the database can be
made correctly.
2016-07-28 13:39:29 -07:00
Steve Howell 713797a65c Add lint checks for self.client.{get,patch,put,post,delete}. 2016-07-27 20:50:54 -07:00
Tim Abbott c2bed83a84 run-dev: Hardcode 127.0.0.1 rather than localhost.
Using "localhost" is problematic in Travis CI, where they have
/etc/hosts configured to make `localhost` resolve to both 127.0.0.1
and ::1.
2016-07-27 11:39:12 -07:00
Eklavya Sharma 97d877f49e tools/: Make subprocess calls unicode-aware. 2016-07-26 12:06:41 -07:00
Taranjeet Singh 1328a10069 Move available-migration and migration-status to var/. 2016-07-25 14:39:35 -07:00
Taranjeet Singh b143d6ca6e Move test_credentials.js to var/casper. 2016-07-25 14:39:23 -07:00
Eklavya Sharma 29af1419db Make tools/update-deployment pass mypy check. 2016-07-25 22:06:05 +05:30
Eklavya Sharma b08d6769c1 tools/zulip-export/zulip-export: Fix how files are opened.
Instead of using codecs.open, use the builtin open in binary mode
and write data after encoding it in utf-8.
2016-07-24 22:32:08 +05:30
Eklavya Sharma df7dcdda5f Annotate tools/get-handlebar-vars. 2016-07-24 22:12:32 +05:30
Eklavya Sharma 61ffd835e1 Annotate tools/minify-js. 2016-07-24 21:58:02 +05:30
Eklavya Sharma 1e0a971178 Annotate tools/test-run-dev. 2016-07-24 21:33:40 +05:30
Eklavya Sharma 7edc38bbf7 Remove tools/test-js-with-casper from mypy exclude list. 2016-07-24 21:07:07 +05:30
Eklavya Sharma e6264fa6fb Annotate tools/compile-handlebars-templates. 2016-07-24 21:07:06 +05:30
Eklavya Sharma 1a77b599f6 Make tools/post-receive python 3 compatible. 2016-07-24 21:07:06 +05:30
Eklavya Sharma 00a120c34f Improve a string in tools/post-receive. 2016-07-24 21:07:02 +05:30
Eklavya Sharma ab8890b304 Annotate tools/check-templates. 2016-07-24 21:04:32 +05:30
Eklavya Sharma 2f889550cb tools/check-templates: Use __init__ in Record.
Initialize Record by using __init__ instead of setting attributes
in validate.  This is needed because mypy complains when we set
new attributes outside __init__.
2016-07-24 21:04:32 +05:30
Eklavya Sharma ecf2249536 tools/check-templates: Set modified's default to False.
Set default value of options.modified to False.
2016-07-24 21:04:31 +05:30
Eklavya Sharma 9c177fcd1b tools/check-templates: Remove unneeded return. 2016-07-24 21:04:31 +05:30
Eklavya Sharma bd0fa3e77b Annotate tools/lint-all. 2016-07-24 21:04:31 +05:30
Eklavya Sharma ec6b630537 Make tools/run-mypy pass mypy check. 2016-07-24 21:04:22 +05:30
Eklavya Sharma 3d87c376c2 Make zulip-ec2-configure-interfaces pass mypy check. 2016-07-24 12:52:58 +05:30
Eklavya Sharma 0a5aa2ccc7 Make check_send_receive_time pass mypy check. 2016-07-24 12:41:06 +05:30
Eklavya Sharma 1820f961b3 frontend_tests/run-casper: Fix annotations. 2016-07-24 10:52:54 +05:30
Taranjeet Singh 5e34e5bf40 iframe-bot/show-last-messages: Fix line with length greater than 120. 2016-07-23 14:33:11 -07:00
Eklavya Sharma 11732f9ab0 Make all scripts in scripts/ pass mypy check. 2016-07-24 00:17:21 +05:30
Tim Abbott 226a57d2c8 travis: Disable test-queue-worker-reload temporarily.
It's failing nondetermistically pretty often, and I'd like to cut the
impact to other users' work.
2016-07-22 11:36:52 -07:00
Eklavya Sharma fb59a07a89 Run mypy on extensionless python scripts in Travis. 2016-07-22 11:16:29 -07:00
Eklavya Sharma f4c557d2a7 tools/run-mypy: Exclude scripts which don't pass mypy. 2016-07-22 11:16:29 -07:00
Eklavya Sharma c12db5246d Use separate exclude list for scripts. 2016-07-22 11:16:29 -07:00
Eklavya Sharma 102fcda4ab tools/run-mypy: Add option --scripts-only.
This option is for checking only extensionless python scripts.
2016-07-22 11:16:29 -07:00
Eklavya Sharma baa157344c Patch activate script only in development. 2016-07-21 14:11:41 -07:00
Tim Abbott 30dd8cfd4a test-queue-worker-reload: Increase sleep resolution. 2016-07-21 10:21:29 -07:00
Umair Khan 7d687b0f79 Increase the timeout in test-queue-worker-reload.
May fix #1341.
2016-07-21 10:19:49 -07:00
Eklavya Sharma 7bd935bef2 Add option to only show extensionless files in lister. 2016-07-21 16:59:52 +05:30
Taranjeet Singh 44d23975ed Move update-prod-static.log to var/log. 2016-07-20 18:27:44 -07:00
Eklavya Sharma 6548f1dd1c Factor out venv-installing code into a module.
Factor out the code in tools/provision.py which installs a python2
and python3 venv into a module (tools/setup/setup_venvs.py) which
can also be used as a script.
2016-07-20 18:20:37 -07:00
Eklavya Sharma 2930a769a9 Add __init__.py to tools/ and tools/setup/.
This will make it possible to import stuff from tools/setup/.
2016-07-20 18:20:37 -07:00
Tim Abbott a07eca2639 Revert "Factor out venv-installation into a script."
This reverts commit 852c49a44e.

I think this may have broken provisioning without a venv.
2016-07-20 14:55:45 -07:00
Eklavya Sharma 852c49a44e Factor out venv-installation into a script. 2016-07-20 14:18:51 -07:00
Tim Abbott 741c21c3ce update_deployment: Make git repository URL configurable. 2016-07-19 21:02:47 -07:00
Tim Abbott a5a79280cb update_deployment: Add symlink for use in voyager installs. 2016-07-19 20:59:59 -07:00
Tim Abbott 35339f5117 Rename local_settings_template to prod_settings_template. 2016-07-19 20:59:59 -07:00
Tim Abbott 19b860ceec Rename local_settings.py symlink to prod_settings.py. 2016-07-19 20:59:59 -07:00
Tim Abbott 82f9f8f941 update-deployment: Compute ZULIP_COM setting. 2016-07-19 20:52:08 -07:00
Tim Abbott 7348841e65 update-prod-static: Copy static/locale rather than moving.
The old behavior results in a large diff when deploying based on a git
commit.
2016-07-19 20:52:08 -07:00
Tim Abbott 28fa4e8346 build_release_tarball: Remove local_settings.py overwrite hack. 2016-07-19 20:52:06 -07:00
Tim Abbott 78a0c7c557 tools: Rewrite install-server to not hardcode configuration.
Now install-aws-server is a reusable script for setting up a Zulip
role server in AWS, without any hardcoded configuration.
2016-07-19 20:12:43 -07:00
Eklavya Sharma d745f20b1b Run frontend and backend tests in python3 on Travis. 2016-07-19 14:15:35 -07:00
Eklavya Sharma fd007c4554 Make tools/test-run-dev python3 compatible.
Use universal_newlines=True with subprocess.check_output in
tools/test-run-dev.
2016-07-19 14:15:35 -07:00
Eklavya Sharma e86539649c Use tools/travis/activate-venv in test suites. 2016-07-19 14:15:35 -07:00
Eklavya Sharma 5054a8d6c9 tools/run-dev.py: Use twisted venv if on python 3. 2016-07-19 14:15:35 -07:00
Eklavya Sharma cfed816a52 tools/provision.py: Install a python2 venv with twisted.
Twisted is not python 3 compatible.  So for now create a python2
venv and install twisted in it when running provision.py in python3
mode and use twisted from the python2 venv.
2016-07-19 14:15:35 -07:00
Eklavya Sharma 9c66cb7130 tools/provision.py: Install both py2 and py3 venvs.
Install both python2 and python3 venvs in tools/provision.py by
default.  Use old behavior when run with --travis to save time.
2016-07-19 23:39:50 +05:30
Eklavya Sharma eb43f7f581 tools/provision.py: Factor out "--travis" in sys.argv. 2016-07-19 23:39:50 +05:30