Commit Graph

171 Commits

Author SHA1 Message Date
Sumana Harihareswara a4ac09cb8f Clarify comment on empty json_rules list.
Fixes: #2030.
2016-10-16 12:28:16 -07:00
Sumana Harihareswara bef73be824 lint: Improve documentation lint check for JavaScript capitalization.
Move JavaScript capitalization rule to new ruleset for HTML and
Markdown checkers to run, and add exclusion to avoid catching
URLs and divs/hrefs. Also fix affected HTML pages, and remove
now-obsolete exclusion for doc that no longer exists.

For exhaustive thought-process on this change see:
https://zulip.tabbott.net/#narrow/stream/test.20suites/topic/watch.20over.20my.20shoulder.20as.20I.20improve.20the.20linter
2016-10-16 12:28:16 -07:00
Tim Abbott dd352dd456 views: Move invitation views to their own file. 2016-10-11 21:27:06 -07:00
Steve Howell 0bdc9fef5c Upgrade caspersjs to version 1.1.3. (w/acrefoot)
(Most of this work was done by acrefoot in an earlier branch.
I took over the branch to fix casper tests that were broken during
the upgrade (which were fixed in a different commit).  I also
made most of the changes to run-casper.)

This also upgrades phantomjs to 2.1.7.

The huge structural change here is that we no longer vendor casperjs
or download phantomjs with our own script.  Instead, we just use
casperjs and phantomjs from npm, via package.json.

Another thing that we do now is run casperjs tests individually, so
that we don't get strange test flakes from test interactions.  (Tests
can still influence each other in terms of changing data, since we
don't yet have code to clear the test database in between tests.)

A lot of this diff is just removing files and obsolete configurations.

The main new piece is in package.json, which causes npm to install the
new version.

Also, run-casper now runs files individually, as mentioned above.

We had vendored casperjs in the past.  I didn't bring over any of our
changes.  Some of the changes were performance-related (primarily
5fd58cf249), so the upgraded version may
be slower in some instances.  (I didn't do much measurement of that,
since most of our slowness when running tests is about the setup
environment, not casper itself.)  Any bug fixes that we may have
implemented in the past were either magically fixed by changes to
casper itself or by improvements we have made in the tests themselves
over the years.

Tim tested the Casper suite on his machine and running the full Casper
test suite is faster than it was before this change (1m30 vs. 1m50),
so we're at least not regressing overall performance.
2016-10-08 12:08:43 -07:00
umkay d260a22637 Add a new statistics/analytics framework.
This is a first pass at building a framework for collecting various
stats about realms, users, streams, etc. Includes:
* New analytics tables for storing counts data
* Raw SQL queries for pulling data from zerver/models.py tables
* Aggregation functions for aggregating hourly stats into daily stats, and
  aggregating user/stream level stats into realm level stats
* A management command for pulling the data

Note that counts.py was added to the linter exclude list due to errors
around %%s.
2016-10-04 17:18:54 -07:00
Tim Abbott d25bbdf574 lint: Fix missing cast causing mypy errors. 2016-09-27 18:38:32 -07:00
Tim Abbott aaa221233a lint: Ban 2-space indentation in our CSS.
Probably most properly we should check for any number of spaces that
isn't 4, but that's a bit more work to do with our linter framework,
and in practice basically every CSS whitespace error we see is 2-space.
2016-09-27 18:06:44 -07:00
Steve Howell 4c5eb3d06e Fix transaction behavior for update_subscriptions_backend().
This commit extracts compose_views() from update_subscriptions_backend(),
and it implements the correct behavior for forcing transactions to roll
back, which is to raise an exception.

There were really three steps in this commit:

- Extract buggy code to compose_views().
- Add tests on compose_views().
- Fix bugs exposed by the new tests by converting errors to exceptions.
2016-09-12 22:03:39 -07:00
Gordon P. Hemsley c9d1a4247f Improve test coverage for tools.
* Replace generic Exception with TemplateParserException.
* Add tests to cover many of the uncovered lines in
  tools/lib/template_parser.py.
* Add an exclusion line to the naïve pattern for checking for missing
  tuples in format strings, to keep the linter happy.
2016-08-30 18:51:40 -04:00
Tim Abbott 2e2dee3396 lint: Ban lines longer than 140 characters without #ignorelongline. 2016-08-19 12:03:09 -07:00
Taranjeet Singh 7fc170d808 lint-all: Fix line with length greater than 120.
This updates file tools/lint-all.
2016-08-19 11:56:43 -07:00
Steve Howell c6a888561a tools: Extract build_custom_checkers() in tools/lint-all. 2016-08-18 14:15:01 -07:00
Steve Howell f955991120 tools: Add bright_red_output() to tools/lint-all 2016-08-18 14:15:01 -07:00
Steve Howell ffe648baa8 tools: Extract EXCLUDED_FILES in tools/lint-all. 2016-08-18 14:15:01 -07:00
Steve Howell dfe01c4f83 tools: Move run_parallel() out of run(). 2016-08-18 14:15:01 -07:00
Steve Howell e3a7b9b1b2 tools: Move check_pyflakes() out of run(). 2016-08-18 14:15:01 -07:00
Steve Howell 3c40157195 tools: Add a run() method to tools/lint-all. 2016-08-18 14:15:01 -07:00
Christie Koehler 096b098410 Flush stdout and stderr prior to exiting child process.
This fixes a confusing bug that we ran into where the output from
certain child processes wouldn't appear when running `lint-all` via
`ssh` (without a tty) into the Vagrant environment.
2016-08-16 18:11:14 -07:00
Tim Abbott c7059c9751 travis: Update success-http-headers to match current certs.
Travis CI seems to have changed the way the snakeoil SSL certs are
generated in their infrastructure, so we need to update our expected
"success" HTTP headers accordingly.
2016-08-12 09:35:41 -07:00
Steve Howell a904222947 Add tools/find-add-class to tools/lint-all.
The find-add-class tool, when in lint mode, verifies that we can
understand all calls to addClass from our JS code.

When in non-lint mode, i.e. verbose mode, the tool prints out a
list of tuples of (fn, class) that we can use as we wish in other
tools.
2016-08-08 15:32:48 -07:00
Tim Abbott 0689485666 Add lint check for malformed type annotations missing :.
Also fix the 2 annotations that weren't being checked because of this.
2016-08-04 15:53:23 -07:00
Steve Howell 3b5c187f55 Add tools/check-css and tools/lib/css_parser.py.
`tools/lint-all` now calls the new `tools/check-css`

The css_parser library parsers CSS into a data structure
that remembers line numbers and columns of semantically
meaningful tokens and adjoining white space/tokens.  It
is intended to be used for various linting tasks.

The file `tools/check-css` runs a few files through the
parser and makes sure they round trip.  This has some value
right away, as files that fail to parse will cause an
exception to be thrown and thus alert developers to syntax
errors.  We expect to grow this into more advanced linting
tasks eventually.
2016-08-02 16:22:46 -07:00
Tim Abbott 75b5d021fa Remove unused puppet-common third-party module code. 2016-07-31 19:24:42 -07:00
Tim Abbott 568dd0e142 lint-all: Improve regular expression for json_error. 2016-07-28 16:19:55 -07:00
Steve Howell 713797a65c Add lint checks for self.client.{get,patch,put,post,delete}. 2016-07-27 20:50:54 -07:00
Eklavya Sharma bd0fa3e77b Annotate tools/lint-all. 2016-07-24 21:04:31 +05:30
Tim Abbott 2338421c6d lint: Add documentation lint check for JavaScript spelling. 2016-07-12 19:22:21 -07:00
Tim Abbott f513a68ac9 lint-all: Check for missing whitespace before { in CSS.
Also fix the existing violations of this rule.
2016-07-10 17:29:36 -07:00
Tim Abbott 88368397aa lint-all: Add Python check for space after if. 2016-07-10 10:58:16 -07:00
Steve Howell e35b84d438 Have lint-all pass along -m to check-templates. 2016-07-08 16:30:49 -07:00
acrefoot 7ff89dc137 Add markdown-specific whitespace lint rule to /tools/lint-all.
Match one space or three+ spaces after content, or positive
number of spaces on an empty line.
2016-06-27 18:43:20 -07:00
Tim Abbott 7d3b451902 lint: Check for extremely long lines.
Currently, we check for lines longer than 180 characters; we can lower
this as we clean up or wrap longer strings.
2016-06-25 10:52:03 -07:00
Umair Khan a70d5041d2 Don't allow literal strings in report_error. 2016-06-23 16:23:01 -07:00
Umair Khan b7bb49c6f5 Don't allow literal string in report_success. 2016-06-23 16:23:01 -07:00
Tim Abbott 6b8e9c7254 Exclude static/locale from linters. 2016-06-21 17:17:34 -07:00
Tim Abbott eb71173be3 lint: Fix warnings under confirmation/. 2016-06-20 08:19:54 -07:00
Tim Abbott 6d1d7471e6 Refactor out zproject/dev_settings.py. 2016-06-17 12:15:15 -07:00
Tim Abbott d117ec8664 lint: Fix redundant stripping of strings. 2016-06-17 10:41:39 -07:00
Tim Abbott 267a71cf20 Run Python custom RE checks in parallel with non-python.
This saves about 1s (out of originally 3.3s) of time running all the
linters in my development environment.
2016-06-17 10:26:50 -07:00
John Hergenroeder 16a19226f6 Add linter check for redundant REQ whence argument. 2016-06-16 13:53:39 -07:00
Umair Khan c1f5ac375c Add linter checks for javascript strings and HTML placeholders.
Checks are added for:
- i18n.t
- placeholder
- compose_error
2016-06-13 09:05:27 -07:00
Tim Abbott 584887e588 lint: Require folding of } on same line in else if statements. 2016-06-09 14:05:34 -07:00
Tim Abbott c35781d505 lint: Require folding of } on same line as else statements. 2016-06-09 14:02:49 -07:00
Tim Abbott 0c1b5006f7 lint: Check for space after if in javascript. 2016-06-09 13:47:12 -07:00
Tim Abbott eba0d6339f lint: Require space after // in JS comments. 2016-06-09 13:44:24 -07:00
Tim Abbott a09b950097 lint-all: Check for relative URLs in JSON calls from JS. 2016-06-09 11:59:40 -07:00
Ernesto Vargas ffb2f9e84b lint/tests: Give nice error message for common import failures.
This should make users much more likely to be able to debug issues
where they ran Zulip outside the Vagrant environment or virtualenv.

[error messages tweaked by tabbott]
2016-06-06 13:39:26 -07:00
Tim Abbott 553ef81f92 Add lint rules for mypy type annotations. 2016-06-05 13:00:55 -07:00
Tim Abbott 03debdf82f Fix malformed error message when creating invalid Realm Emoji.
Thanks to Greg McCoy for his help finding this bug.
2016-06-03 23:12:36 -07:00
Nathan Florea 5fe9076631 Remove some mutable default arguments.
These ones don't fix any bugs, because the mutable arg is never passed
outside of the callable or mutated.  But it's good practice to not use
them in case those invariants are changed in the future.
2016-06-03 09:16:56 -07:00
Umair Khan bd4e471706 Check json_error and JsonableError in linter.
Mainly the check is to ensure that all the strings that pass through
these two functions are captured by `makemessages`.
2016-05-31 07:40:42 -07:00
Umair Khan 4b28fcd2f3 Add option to linter to exclude lines from files. 2016-05-31 07:40:42 -07:00
Tim Abbott 41336f3782 lint-all: Check for use of '== None'. 2016-05-31 07:02:04 -07:00
Eklavya Sharma 1c04560def Re-enable pyflakes in linter and remove python 3 pyflakes errors. 2016-05-25 19:25:13 +05:30
Eklavya Sharma 1a6257394c Make tools/lint-all run on python 3.
Since pyflakes catches some extra errors in python 3, disable
pyflakes for now.
2016-05-25 19:23:13 +05:30
Eklavya Sharma 3185b7e750 Remove unneeded imports from tools/lint-all. 2016-05-25 19:12:09 +05:30
Tim Abbott 542af0d6b6 lint: Add option to print verbose timing output. 2016-05-04 14:22:52 -07:00
Tim Abbott e2aeee0c35 lint: Add check for unnecessary whitespace between % and (. 2016-05-04 14:22:52 -07:00
Tim Abbott 72ee9f5137 lint: Check for unnecssary whitespace after ','s. 2016-05-04 14:17:27 -07:00
Tim Abbott 391a225595 lint: Check for missing space after comments. 2016-05-02 22:10:47 -07:00
Tim Abbott 302da832fa lint: Enforce whitespace between : and value in dicts. 2016-04-27 22:23:40 -07:00
Tim Abbott 8a278cbe3a Switch to using a Zulip version of @login_required.
Currently the code is the unmodified Django upstream implementation;
this commit is preparation for modifying it.
2016-04-21 14:59:39 -07:00
Tim Abbott 9c56027627 lint: Add CSS lint rule for whitespace after {. 2016-04-20 11:50:01 -07:00
Tim Abbott a46b5d7bbe Add lint check for missing whitespace after =. 2016-04-20 11:37:03 -07:00
Tim Abbott a72385246e Fix missing whitespace after '=' in python/js code. 2016-04-20 11:36:14 -07:00
Tim Abbott a2b59b8b51 lint: Check whitespace rules for txt files. 2016-04-14 14:36:29 -07:00
Tim Abbott 39950b8f4f lint: Check whitespace rules in markdown files too. 2016-04-14 14:30:29 -07:00
Tim Abbott 2b76f6223e Make 'no newline at end of file' lint error more actionable.
There's a handy sed command to fix this, so we might as well document
it.
2016-04-14 10:55:06 -07:00
Tim Abbott e71d8bb4b6 lint-all: Require newlines at end of JSON files. 2016-04-14 10:49:12 -07:00
Tim Abbott 9584ae1ab8 Add CSS linter for missing space after : in rules. 2016-04-08 21:04:43 -07:00
Tim Abbott 209e6ef7a1 Run trailing whitespace linter on bash files. 2016-04-08 11:52:11 -07:00
Tim Abbott caba24b2af Fix existing trailing whitespace in bash scripts. 2016-04-08 11:52:11 -07:00
Tim Abbott 4fa63c29ca Run whitespace linters on html files. 2016-04-08 11:52:11 -07:00
Tim Abbott d670e902a9 Run whitespace linters on handlebars templates. 2016-04-08 11:52:11 -07:00
Tim Abbott c6d01ab76b Run whitespace linters on CSS files. 2016-04-08 11:47:10 -07:00
Tim Abbott 1b84617771 Don't skip running python custom linters inside comments.
This fixes an issue where we weren't checking for trailing whitespace
in comments.
2016-04-08 11:47:10 -07:00
Tim Abbott 14b5e265c2 Remove unuseful suspicious code lint check. 2016-04-08 11:47:10 -07:00
Tim Abbott efd14e7ad9 Revert "Exclude 'from typing import *' from linter."
This reverts commit d936bf61f9.

We no longer need this since we've migrated to specifying the
dependencies in the typing module that we're actually using.
2016-04-07 14:12:18 -07:00
Tim Abbott 78e289f904 Exclude puppet-common tests from puppet linter. 2016-04-04 17:08:29 -07:00
Eklavya Sharma 81fdeae0ea Remove '.'s from extensions in lister.py interface and lint-all.
In lint-all, change occurences of '.py', '.js', ... to 'py', 'js', ....
2016-04-01 15:27:16 -07:00
Eklavya Sharma ad4c20a3e6 Migrate lint-all to lister.py for getting files.
This has the side effect of making lint-all check all shell scripts,
not just those under scripts/, tools/, and bin/.

[commit message expanded by tabbott]
2016-04-01 15:24:43 -07:00
Tim Abbott d936bf61f9 Exclude 'from typing import *' from linter. 2016-03-30 21:50:31 -07:00
Tim Abbott f9222de83e Auto-load commonly used modules in manage.py shell.
This automatically loads settings, zerver.models.* and
zerver.lib.actions.* when you start `manage.py shell`, which should
save a bit of time basically every time someone uses it.

Fixes #275.
2016-03-19 11:32:49 -07:00
Varshit 72033069ed Extend lint-all to check for newlines at the end of files. 2016-03-17 23:03:56 -07:00
Tim Abbott 3a46bae542 Fix shell script list computation to exclude files not in git.
This fixes an issue where the next commit's no-newline-and-end-of-file
check was incorrectly firing on tools/phantomjs.
2016-03-17 23:03:56 -07:00
Eklavya Sharma aa505b0d55 Apply Python 3 futurize transform libmodernize.fixes.fix_map
Refer to #256
2016-03-10 22:03:44 -08:00
Eklavya Sharma def027a1ec Apply Python 3 futurize transform libmodernize.fixes.fix_filter
Refer to #256
2016-03-10 22:03:06 -08:00
Eklavya Sharma c59185e119 Apply Python 3 futurize transform libfuturize.fixes.fix_print_with_import
Refer #256
2016-03-10 22:02:17 -08:00
Tim Abbott aad3bff193 Harden style rule for % comprehensions and fix existing errors. 2016-02-02 23:08:19 -08:00
Tim Abbott 700055c194 Apply modernize transform libmodernize.fixes.fix_file.
This replaces use of file() with open() which is python 3 compatible,
and also adds it to our python 3 support test suite.
2016-01-26 21:09:42 -08:00
Vladislav Manchev df4d1b3c14 Add linting code for detecting shebang bashisms.
This will prevent regressions in OpenBSD compatibility, since OpenBSD
doesn't support passing arguments in the #! line.
2016-01-21 22:33:55 -08:00
Tim Abbott 0fe819eb57 lint: Tighten lint rules around whitespace around '%' comprehensions. 2015-12-05 15:29:42 -08:00
Tim Abbott 69b6b60017 lint: Clean up whitespace_rules.
Several of these rules only apply to one of Python and Javascript, and
this simplifies the logic and should make our linter code more readable.

In the process, we add support for per-rule/file pair exclusions to
handle the tab exception for codehilite.py.
2015-12-05 15:29:42 -08:00
Tim Abbott a712954c59 lint: Rewrite custom checks to use a more consistent framework. 2015-12-05 15:29:42 -08:00
Tim Abbott 8a18e78a65 Add lint rules checking for our % comprehension style. 2015-12-05 15:29:42 -08:00
Tim Abbott b81ecc4064 Add whitespace lint rules checking for missing spaces before {. 2015-11-10 10:06:47 -08:00
Tim Abbott f1074aa491 Move frontend tests out of zerver/tests/.
This fixes an unfortunate bug where the backend tests in
zerver/tests.py were not being run automatically, and also makes these
a bit easier to find.
2015-10-28 10:11:47 -07:00
Steven Oud d5435fad1d Consistently use /usr/bin/env python2.7 in shebangs and commands. 2015-10-21 22:58:21 +00:00
Tim Abbott 232de3015a Add lint checks for tab-based whitespace.
We exclude the one file we have with issues that comes from a
third-party vendor.

(imported from commit 47e097b39440aea8e4d7468892176ffce4818b5f)
2015-09-19 23:23:02 -07:00
Yoyo Zhou c25d9679a5 Utility for unescaping HTML entities leftover from MediaWiki->rST conversion
(imported from commit 2f2507f299c3d09c67bfd26d6c11fab47431d935)
2015-08-18 17:31:33 -07:00