Commit Graph

1418 Commits

Author SHA1 Message Date
Tim Abbott 0ac78ca454 setup-production: Fix path to update-prod-static.log.
This regressed in 55f2c49bbf.
2016-12-01 11:15:20 -08:00
Rafid Aslam c5316b4002 lint: Fix E127 pep8 violations.
Fix pep8: E127 continuation line over-indented for visual indent
style issue.
2016-12-01 10:23:55 -08:00
Tim Abbott fd81cc1d93 test-backend: Clarify missing template tests error message. 2016-12-01 10:12:38 -08:00
Vamshi Balanaga 5f60258a85 pep8: Fix E211 violations. 2016-11-30 20:13:10 -08:00
Bickio 6b0df43463 pep8: Fix E125. 2016-11-30 20:03:29 -08:00
Bickio e009383460 pep8: Fix E231. 2016-11-30 19:59:25 -08:00
Tim Abbott 995a692f7f provision: Check whether the system has at least 2GB of RAM.
Fixes #2336.
2016-11-30 16:07:57 -08:00
Tim Abbott e73ee5a4c4 test-backend: Fix count for templates without tests. 2016-11-30 15:38:19 -08:00
Tommy Ip e4091c6413 pep8: Fix E222 violations. 2016-11-30 21:49:02 +00:00
Tommy Ip 46b7d54b3e pep8: Fix E701 violations. 2016-11-30 20:45:09 +00:00
Heidi Dong ba7d4e7452 mypy: Annotate bots/gcal-bot. 2016-11-29 20:49:11 -08:00
Anders Kaseorg 6549dfd1d0 build-release-tarball: Add --py3 option for python3 shebang lines.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-29 17:45:18 -08:00
Tim Abbott 4f6a075331 test-backend: Fix missing output for untested templates.
Our enforcement logic for untested templates unfortunately did not
give a clear error message unless you passed the --verbose option.
2016-11-29 16:38:18 -08:00
Tommy Ip 844e32ccc5 tools: Remove deprecated generate-activity-metrics.py.
The script uses hardcoded hostnames that do not exist anymore,
its functionality is available through directly querying the Zulip
database.
2016-11-29 09:27:59 -08:00
Rafid Aslam 41bd88d5ed pep8: Fix E301 pep8 violations.
Fix "E301: expected (1 or 2) blank line" pep8 violations.
2016-11-29 08:51:44 -08:00
Rafid Aslam 7a2282986a pep8: Fix E225 pep8 violations. 2016-11-28 15:21:15 -08:00
sylvan1 443cf92640 mypy: Change Generator[None, None, None] to Iterator[None].
Fixes #1648.
2016-11-27 10:42:16 -08:00
Tim Abbott 1fcf2ff525 tornado: Move zerver.tornadoviews to zerver.tornado.views.
This furthers the overall goal of moving all the Tornado-specific code
to zerver/tornado/.
2016-11-26 22:29:28 -08:00
Tim Abbott 8739f4d9f1 tornado: Move tornado_ioloop_logging to new zerver/tornado tree. 2016-11-26 21:24:05 -08:00
Tim Abbott 92d1b6d6da run-dev: Fix regression ignoring --interface argument.
When we migrated run-dev.py from Twisted to Tornado a few weeks ago,
the --interface argument wasn't properly ported and thus was ignored.

This restores the original functionality of defaulting to only
listening on localhost.

Ideally, we'd replace the vagrant/zulipdev user check with something
that just checks whether a special file that is created by the
Vagrant/remote-dev-vm creation process exists; that would be more
robust.
2016-11-26 17:23:31 -08:00
Steve Howell 6ed1dc9341 Create api docs from url coverage data from tests.
I also retire tools/analyze-url-coverage in this commit,
since the API docs cover most of the functionality.
2016-11-26 14:49:23 -08:00
Anders Kaseorg 207cf6302b Always start python via shebang lines.
This is preparation for supporting using Python 3 in production.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-26 14:46:37 -08:00
Anders Kaseorg 1ea8abe493 Replace python -u with PYTHONUNBUFFERED=1
(Why is -u needed at all?  I’m not sure, but test-run-dev spins forever
“Polling run-dev...” without it.)

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-26 13:20:22 -08:00
Arpith Siromoney f0df1db9fb tools: Remove tools/node and call node directly instead.
Previously, we didn't install/pin our own node.js version, and thus
had this wrapper script to manage it.  Now that install our own node
via nvm, there's no reason we still need this.

Fixes #2409.
2016-11-26 12:42:58 -08:00
Tim Abbott 25c6b9f586 pre-commit: Call lint-all with --force. 2016-11-26 12:30:16 -08:00
Steve Howell 8110c3f799 tools: Check provisioning version in lint-all. 2016-11-26 11:36:53 -08:00
Steve Howell 5f5e6b6d83 tests: Enforce 100% URL coverage.
We now instrument URL coverage whenever you run the back end tests,
and if you run the full suite and fail to test all endpoints, we
exit with a non-zero exit code and report failures to you.

If you are running just a subset of the test suite, you'll still
be able to see var/url_coverage.txt, which has some useful info.

With some tweaks to the output from tabbott.

Fixes #1441.
2016-11-18 18:10:29 -08:00
Tim Abbott 2abe11e209 lint-all: Fix progress bar from `npm run lint` in linter.
Also fixes printing the `eslint` help output in the event that there
are no files to check.

See https://github.com/zulip/zulip/pull/2246#issuecomment-261673662
for details on the original problem.
2016-11-18 16:19:05 -08:00
Arpith Siromoney 497c7702cc Run eslint in tools/lint-all with npm run --silent lint
This commit adds a basic eslintrc that emulates jslint defaults.
Rules that conflict with our existing code have been switched to
warnings instead of errors. Globals have been added to the eslintrc. The
bundled js file (generated by webpack) and blueslip.js are ignored with
.eslintignore.

To display warnings, run npm run lint-loud. This runs eslint without the
--quiet option on static/js and frontend_tests.

npm run --silent lint is run by tools/lint-all (in addition to jslint).
The --silent option is used to suppress the default output from npm run.

Fixes #535.
2016-11-18 16:15:45 -08:00
Vladislav Manchev d7e1e4a2c0 Add initial implementation of custom realm filters.
This PR was abandoned by Vladislav and then substantially modified by
Igor Tokarev and Tim Abbott to complete it and fix a number of bugs.

Fixes #544.
2016-11-17 17:11:25 -08:00
paxapy 012ec2beda test-backend: Use a fixed whitelist of shallow-tested templates.
This enforces the requirement that all templates not on this list are
tested.  Part of #1677.
2016-11-17 11:34:50 -08:00
Pweaver (Paul Weaver) 059cd1e7aa tests: enable the pep8 linter in travis CI. 2016-11-17 00:11:00 -08:00
Pweaver (Paul Weaver) 8f05ea877d lint: Fix pep8 linter error regressions.
Since `lint-all --pep8` wasn't running in CI, a few errors have leaked
in.
2016-11-17 00:09:21 -08:00
Christie Koehler 391ff638d5 build_emoji: Add nice error message for symlink errors. 2016-11-15 21:10:39 -08:00
Christie Koehler 2488698430 provision: Add symlink creation check.
This helps automatically detect a common issue where users try to use
the Vagrant development environment on windows without administrator
permissions, which results in errors trying to create symlinks.
2016-11-15 21:10:39 -08:00
Tomasz Kolek 2335e1b8b9 Add escaping charactes (=,>,<) by checking correct spelling of GitHub word 2016-11-15 10:48:12 -08:00
Brock Whittaker 75db036733 Add toggle component.
This adds the toggle component which removes the necessity to have HTML.
2016-11-15 10:20:37 -08:00
Steve Howell 59750a1cb9 Move more unread logic out of stream_list.js.
There are two split-outs code of the code here:

    - Animation stuff is in unread_ui.js.
    - PM unread counts get handled by pm_list.js.
2016-11-14 17:50:55 -08:00
Umair Khan 3d3800314f Django 1.10: Setup django before running queue processors. 2016-11-14 16:09:12 -08:00
Steve Howell 7b0c6459b4 Extract static/js/pm_list.js.
This handles most of the details of building the Private Messages
section in the upper left corner of the app.
2016-11-11 12:12:13 -08:00
Umair Khan 07d73996d4 Django 1.10: Add tool to check urls.
Checking urls through linter was proving to be difficult as
the urls can span multiple lines.
2016-11-10 16:20:04 +05:00
Shashank 702d3cf103 Check provision version in run-dev.py.
Don't start run-dev.py if the provisioning version is
incorrect.
2016-11-09 16:06:16 -08:00
paxapy 6d93b3b60c lint: Fix E703 pep8 violations. 2016-11-09 15:18:35 -08:00
paxapy 456e761294 lint: Add pep8 checker to the project.
We set this up initially with all of the rules that Zulip violates
disabled.

Also, the pep8 linter is substantially slower than the other Zulip
linters, so we've put it behind an option to `tools/lint-all`.
2016-11-09 15:17:49 -08:00
Tim Abbott 44767c59a2 Coverage: Exclude migrations and management commands.
These are not particularly interesting to measure test coverage for,
since migrations are not run as the part of the test suite by
construction, and management commands aren't being tested by this test
suite.
2016-11-06 18:34:23 -08:00
Tomasz Kolek 704ac5aa6c Fix lint-all checker by adding "_" as a escape character before Github. 2016-11-05 22:12:25 -07:00
Tim Abbott ee361cbe30 js: Eliminate process_loaded_for_unread global. 2016-11-05 11:33:04 -07:00
sonali0901 df4b777b35 tests: Fix handling of subdirs and backup files in test-backend.
Fixes #1950.
2016-11-04 21:49:23 -07:00
umkay b5b8623f46 analytics: Update linter to ignore interpolation in test_counts.
Ignore whitespace error for % interpolation in sql queries.
2016-11-03 16:50:39 -07:00
Tim Abbott a93530e413 Remove unused route for development tornado proxy server.
This was a duplicate of the other sockjs line.
2016-11-03 10:46:18 -07:00
Tim Abbott 50361ca980 test-run-dev: Fix buggy site argument.
This bug was exposed by 9578908601.
2016-11-03 09:34:56 -07:00
K.Kanakhin b248a2c33e docs-test: Check only http 404 error for external links.
- Raise exception only for http 404 error rensponse, for other
  http error codes used logging to notify devs about wrong links.
2016-11-01 21:45:41 -07:00
K.Kanakhin 870de03ede run-dev: Add websockets support to tornado dev proxy server.
Fixes #1036.
2016-11-01 20:35:31 -07:00
K.Kanakhin 09e17fbe17 run-dev: Use tornado for dev proxy server for HTTP requests.
- Use tornado as proxy server for development environment,
  replacing twisted (this doesn't support websockets).
- Upgrade tornado version to 4.4.1 (needs to be coupled to the
  above since neither change works without the other)
2016-11-01 20:35:31 -07:00
Tim Abbott 6a8f8c2abf run-dev: Fix minor whitespace issues. 2016-11-01 20:27:44 -07:00
Tim Abbott 4fe5fc849e travis: Disable test-documentation temporarily. 2016-10-28 11:27:22 -07:00
Tim Abbott 9b7a3f040c Remove now-unused /json/get_events endpoint. 2016-10-27 21:34:58 -07:00
Steve Howell 4f38cfdc7f Extract topic_list.update_count_in_dom().
This creates the new topic_list.js module, and the first
function that we extract is topic_list.update_count_in_dom().

This function needed to be decoupled from some non-topic-list
stuff which was overly complicated.
2016-10-27 15:49:25 -07:00
Tim Abbott 76fd7c0735 provision: Fix creation of cache directories. 2016-10-27 12:05:43 -07:00
K.Kanakhin 63f0e04621 docs-test: Exclude remote servers internal response errors.
Sometimes remote servers are unavailable or on maintance
what is the cause of broken back-end tests. Such response errors
is excluded from raised exception cases and just notify about
wrong link in test log.
2016-10-26 01:00:33 -07:00
Tim Abbott 129f1472fb provision: Fix hardcoding of current user.
This fixes a regression in 8e82257444
and 2b8324b778.
2016-10-25 21:58:15 -07:00
Tim Abbott 2b8324b778 emoji: Fix caching permissions issues.
Previously, you needed to be root to update the emoji cache, which
caused problems with how Zulip is upgraded in production.
2016-10-25 17:52:19 -07:00
Tim Abbott 8e82257444 Fix node_cache code to not require root.
The previous code caused problems using a system where the zulip user
doesn't have sudo rights.
2016-10-25 17:52:19 -07:00
Tim Abbott 442f3a91e5 test-documentation: Add pretty success/failure output. 2016-10-25 11:41:12 -07:00
K.Kanakhin 455301ca13 docs-test: Add documentation test scripts to backend test case.
- Add script to compile documentation build and start crawler
  to check documentation.
- Add documentation test script to backend travis test case.
- Add log level argument to test-documentation script.

Fixes #1492
2016-10-25 11:36:43 -07:00
K.Kanakhin 6b31fcd63a docs-test: Add scrapy crawler to check documentation not having broken links.
- Add scrapy project with rewrited 'crawl' command and spider to check documentation.
  Command was rewrited due to return exit status by the exception existing,
  it returns exit code 0 if exception happens inside spider in standard behavour.

- Add scrapy requirements to for dev environment. It was added to twisted requirements list.

Fixes #1492
2016-10-25 11:35:16 -07:00
deekshaarul be903382f8 lint: Add rule to check for misspellings of 'GitHub'. 2016-10-22 17:09:55 -07:00
Umair Khan a2c57db630 Use argparse in provision.py 2016-10-21 08:35:30 -07:00
Prabhakar Gupta bc6421fbfb tools: Remove empty wrapper script build_emoji.
This renames the old `emoji_dump.py` to `build_emoji`, removing the
old shell essentially empty shell script.  `emoji_dump.py` was always
a weird name, and this makes it a bit easier to read the code for this
system.
2016-10-20 22:57:45 -07:00
Tim Abbott f9f8b18e2f lint: Ban use of json_success({}) instead of json_success(). 2016-10-20 22:34:04 -07:00
Umair Khan 041cd6c787 Add --force argument to provision.py.
Fixes: #2026
2016-10-20 22:04:52 -07:00
Umair Khan 35dea5246c Regenerate test DB only when template DB has changed. 2016-10-20 22:04:52 -07:00
Tim Abbott 4a4664d268 mypy: Remove a bunch of now-unnecessary type: ignore annotations.
Since mypy and typeshed have advanced a lot over the last several
months, we no longer need these `type: ignore` annotations.
2016-10-17 11:48:34 -07:00
Tim Abbott 36e336edc3 puppet: Rename zulip_internal to zulip_ops.
The old "zulip_internal" name was from back when Zulip, Inc. had two
distributions of Zulip, the enterprise distribution in puppet/zulip/
and the "internal" SAAS distribution in puppet/zulip_internal.  I
think the name is a bit confusing in the new fully open-source Zulip
work, so we're replacing it with "zulip_ops".  I don't think the new
name is perfect, but it's better.

In the following commits, we'll delete a bunch of pieces of Zulip,
Inc.'s infrastructure that don't exist anymore and thus are no longer
useful (e.g. the old Trac configuration), with the goal of cleaning
the repository of as much unnecessary content as possible.
2016-10-16 19:23:27 -07:00
Steve Howell 6e628e23aa Add --force option to test-backend. 2016-10-16 14:52:14 -07:00
Steve Howell 10667b3cdb Check provisioning status in tools/test-backend. 2016-10-16 14:52:14 -07:00
Steve Howell 8bc027e71b Add tools/lib/test_script.py w/get_provisioning_status(). 2016-10-16 14:52:13 -07:00
Steve Howell 37842d5d15 Have provision.py write to var/provision_version. 2016-10-16 14:44:45 -07:00
Tim Abbott df617225fd lint-all: Check for untranslated placeholders in handlebars templates. 2016-10-16 12:59:41 -07:00
Sumana Harihareswara a4ac09cb8f Clarify comment on empty json_rules list.
Fixes: #2030.
2016-10-16 12:28:16 -07:00
Sumana Harihareswara bef73be824 lint: Improve documentation lint check for JavaScript capitalization.
Move JavaScript capitalization rule to new ruleset for HTML and
Markdown checkers to run, and add exclusion to avoid catching
URLs and divs/hrefs. Also fix affected HTML pages, and remove
now-obsolete exclusion for doc that no longer exists.

For exhaustive thought-process on this change see:
https://zulip.tabbott.net/#narrow/stream/test.20suites/topic/watch.20over.20my.20shoulder.20as.20I.20improve.20the.20linter
2016-10-16 12:28:16 -07:00
Tim Abbott fc20c86d8d install: Move apt-get update into setup-apt-repo. 2016-10-16 01:13:50 -07:00
Tim Abbott cf6feac65f provision: Remove unnecessary second apt-get update.
Since we do this slightly later (after setting up our apt repository,
where it's actually required), there's no need to do this here.
2016-10-16 01:13:50 -07:00
Tim Abbott 217212b773 run-mypy: Use the experimental new mypy cache.
This makes iteratively running mypy approximately 5x faster.

Fixes #1981.
2016-10-15 22:57:11 -07:00
Tim Abbott 9e503f2dcf run-mypy: Check scripts by default.
Previously, we checked scripts in a separate run to work around mypy
not supporting multiple scripts with the same name.  Since we have
fixed that issue, we can restore the original behavior.

We leave the --scripts-only option available, though I'm not sure it's
particularly useful and we'll probably eventually remove it.
2016-10-15 22:53:29 -07:00
Tim Abbott 4dbdf7e373 run-mypy: Consolidate exclude lists. 2016-10-15 22:53:29 -07:00
Tim Abbott 67e9f35f6f run-mypy: Use new --scripts-are-modules argument.
This allows us to check all the Zulip scripts in just 3s, rather than
inefficiently looping through all the Zulip scripts.
2016-10-15 22:53:29 -07:00
Tim Abbott 1a751af148 run-mypy: Exclude puppet/zulip_internal. 2016-10-15 22:53:29 -07:00
Tim Abbott 172809d4e2 Annotate the rest of scripts in tools/. 2016-10-15 22:36:10 -07:00
Tim Abbott 75daba345a Finish annotating test-backend. 2016-10-15 22:36:10 -07:00
Tim Abbott 58a8934a86 tools: Remove unused post-receive script. 2016-10-15 22:36:10 -07:00
Tim Abbott 1a2b593a7c tools: Remove now-unused update-deployment symlink. 2016-10-15 22:36:10 -07:00
Tim Abbott 0c5f419be0 Annotate test-queue-worker-reload. 2016-10-15 22:36:10 -07:00
Tim Abbott 14f6e4c740 scripts: Stop using apt-add-repository.
Unfortunately, apt-add-repository is highly unreliable and was causing
problems both in Travis CI and with developers provisioning their
environment.
2016-10-11 22:10:36 -07:00
Tim Abbott dd352dd456 views: Move invitation views to their own file. 2016-10-11 21:27:06 -07:00
Christie Koehler 24e2690ec2 tools: Improve git scripts.
Minor fixes that enable the ability to:

- Re-run fetch-rebase-pull-request.
- Specify the name of the remote repo as an optional second parameter.
  The default remains 'upstream'.
2016-10-11 13:38:38 -07:00
Eklavya Sharma 81617fcdb0 Annotate zerver/tests/test_upload.py. 2016-10-11 11:29:14 -07:00
Eklavya Sharma f7092b1a90 Annotate zerver/tests/test_email_mirror.py. 2016-10-11 11:28:52 -07:00
Eklavya Sharma 1dbabfe1ab Annotate zerver/tests/test_bugdown.py. 2016-10-11 11:27:35 -07:00
Umair Khan e0c10ed1e8 Cache emoji dump output.
This saves a bunch of time building release tarballs, provisioning,
and upgrading Zulip from git that was spent regenerating the Zulip
emoji sprite sheet.

[commit message tweaked by tabbott]
2016-10-10 11:15:43 -07:00