Commit Graph

79 Commits

Author SHA1 Message Date
Tim Abbott e7389b4162 test-backend: Require 100% coverage on tests by default. 2017-03-04 23:39:34 -08:00
Tim Abbott 6c12a49d04 test_auth_backends: Remove now-unnecessary compatibility code.
This code was added as part of the Django 1.10 migration to make our
tests work with both Django 1.8 and 1.10.  Now that we're on 1.10,
it's no longer required.
2017-03-04 23:38:46 -08:00
Tim Abbott 39704bac6c test_bugdown: Remove unused common_bugdown_test. 2017-03-04 23:35:46 -08:00
Tim Abbott 1a8bc5f383 coverage: Bring test_decorators to 100% coverage. 2017-03-04 23:34:28 -08:00
Tim Abbott c73c3c52f8 test_email_mirror: Fix TestCommandMTA.
The test didn't actually use the mock, due to an incorrect import
path.
2017-03-04 23:28:48 -08:00
Tim Abbott 6ea3a9cb1d coverage: Bring test_templates.py to 100% coverage. 2017-03-04 23:28:48 -08:00
Tim Abbott d8171b2efc coverage: Bring tests.py to 100% coverage. 2017-03-04 23:17:36 -08:00
Tim Abbott 768307d921 coverage: Bring test_events to 100% coverage. 2017-03-04 23:12:32 -08:00
Tim Abbott 538570b7c1 test_subs: Bring to 100% test coverage. 2017-03-04 23:07:56 -08:00
Tim Abbott 065651b4fe coverage: Exclude uncovered lines in test_signup. 2017-03-04 23:01:50 -08:00
Tim Abbott bc0cd7eb90 coverage: Move coverage config to tools/coveragerc. 2017-03-04 22:48:05 -08:00
Tim Abbott 63664264b8 test-backend: Disable verbose coverage display. 2017-03-04 22:48:05 -08:00
Tim Abbott d90f7c72a3 zephyr: Fix broken postgres regular expression logic and add tests.
Like many rare-case code with new tests, it turns out that the logic
for handling null characters in our Zephyr postgres query escaping
never worked, in multiple ways.  First, it always changed the second
character in s, not the current one being inspected, and second, the
value it replaced it with was no the correct postgres escape of the
null byte.  We fix this and add tests.

This completes the effort to get zerver/views/messages.py to 100%
test coverage.

Fixes #1006.
2017-03-01 10:38:48 -08:00
Elliott Jin 7ed10da4ad test-backend: Raise zerver/views/report.py test coverage to 100%. 2017-02-28 20:06:00 -08:00
Elliott Jin 11ba94f11a test-backend: Raise zerver/views/integrations.py test coverage to 100%. 2017-02-28 09:31:06 -08:00
Elliott Jin 6bdefb92e9 test-backend: Raise zerver/views/zephyr.py test coverage to 100%. 2017-02-26 20:54:25 -08:00
Elliott Jin f3cd3e8b8d test-backend: Raise zerver/views/invite.py test coverage to 100%. 2017-02-26 16:15:25 -08:00
Elliott Jin cebc67f9b0 test-backend: Raise zerver/views/unsubscribe.py test coverage to 100%. 2017-02-26 16:13:35 -08:00
K.Kanakhin 257bb40698 realm-icon: Add realm icon feature.
- Add realm icon fields to realm model.
- Add migration for new realm model's field.
- Add views for icon uploading and deleting.
- Add routes for realm icons views.
- Add JS widget for realm icon upload setting.
- Add realm icon upload to administration
  organization setting.
- Add tests for realm icons.

Fixes #3660.
2017-02-26 12:16:07 -08:00
Elliott Jin f038cd47d9 test-backend: Raise zerver/views/realm.py test coverage to 100%. 2017-02-25 18:18:29 -08:00
Umair Khan 128beb910b testing: Use TestResult in run_test.
Internally, Django uses TestResult to gather results from testcases.
This commit upgrades the run_test function to use TestResult to compile
result.
2017-02-19 21:04:06 -08:00
Tim Abbott 9ac66b0121 test-backend: Update blacklist of covered files based on CI.
After accidentally merging ecadb33fbc
before I go CI input, I discovered that coverage in my development
environment differs from that in CI :(.
2017-02-18 18:52:59 -08:00
Tim Abbott ecadb33fbc test-backend: Add 100% test coverage assertions.
This adds an assertion, when `test-backend` is run with `--coverage`,
that we have 100% test coverage on a list of files that we expect to.
There's a whitelist/blacklist, managed in tools/test-backend.

Fixes #3363.
2017-02-18 16:34:40 -08:00
Umair Khan e5a16ceb0a testing: Use failfast instead of fatal_errors.
`failfast` has the same meaning as `fatal_errors` in Django's test
runner.
2017-02-10 16:01:43 -08:00
sinwar eab355b0cd tools: Create more consistent checks for venv.
This helps make the Zulip development environment somewhat more robust
to new contributors, since it will give them a nice warning if they
try running any of our development tools outside the Zulip virtualenv.

Fixes #3468.
2017-02-06 21:50:32 -08:00
Tomasz Kolek 7de45951e2 Make webhooks as separate modules with view and tests.
Create python packege for every webhook with view.py and tests.py
2017-01-25 23:14:19 -08:00
Tim Abbott aadf621466 test-backend: Only check for unrendered templates if tests passed. 2017-01-20 10:56:03 -08:00
reallyly 94b28e0afb pep8: Fix E128 violations.
With some line-wrapping tweaks by tabbott.
2016-12-13 10:50:14 -08:00
Adarsh S 31be2534b3 Fix Error: templates have no tests! while running TemplateTestCase.
We need to test all the templates only when we are running the full
 test suite. While running just the Test Class we need not check all
 the templates.
2016-12-09 12:09:55 -08:00
Tim Abbott fd81cc1d93 test-backend: Clarify missing template tests error message. 2016-12-01 10:12:38 -08:00
Tim Abbott e73ee5a4c4 test-backend: Fix count for templates without tests. 2016-11-30 15:38:19 -08:00
Tim Abbott 4f6a075331 test-backend: Fix missing output for untested templates.
Our enforcement logic for untested templates unfortunately did not
give a clear error message unless you passed the --verbose option.
2016-11-29 16:38:18 -08:00
Steve Howell 5f5e6b6d83 tests: Enforce 100% URL coverage.
We now instrument URL coverage whenever you run the back end tests,
and if you run the full suite and fail to test all endpoints, we
exit with a non-zero exit code and report failures to you.

If you are running just a subset of the test suite, you'll still
be able to see var/url_coverage.txt, which has some useful info.

With some tweaks to the output from tabbott.

Fixes #1441.
2016-11-18 18:10:29 -08:00
paxapy 012ec2beda test-backend: Use a fixed whitelist of shallow-tested templates.
This enforces the requirement that all templates not on this list are
tested.  Part of #1677.
2016-11-17 11:34:50 -08:00
Tim Abbott 44767c59a2 Coverage: Exclude migrations and management commands.
These are not particularly interesting to measure test coverage for,
since migrations are not run as the part of the test suite by
construction, and management commands aren't being tested by this test
suite.
2016-11-06 18:34:23 -08:00
sonali0901 df4b777b35 tests: Fix handling of subdirs and backup files in test-backend.
Fixes #1950.
2016-11-04 21:49:23 -07:00
Steve Howell 6e628e23aa Add --force option to test-backend. 2016-10-16 14:52:14 -07:00
Steve Howell 10667b3cdb Check provisioning status in tools/test-backend. 2016-10-16 14:52:14 -07:00
Tim Abbott 75daba345a Finish annotating test-backend. 2016-10-15 22:36:10 -07:00
Rishi Gupta 7707d2a005 test-backend: remove trailing forward slash from test suite names.
Previously, running `tools/test-backend analytics/` (or any other test suite
name ending with a '/') would give a cryptic error about modules not
importing properly. This commit rstrip's the trailing slash from test suite
names given on the command line.
2016-10-08 11:52:55 -07:00
Tomasz Kolek dbeab6aa6f Optimize checks of test database state by moving into Python.
Previously, the generate-fixtures shell script by called into Django
multiple times in order to check whether the database was in a
reasonable state.  Since there's a lot of overhead to starting up
Django, this resulted in `test-backend` and `test-js-with-casper`
being quite slow to run a single small test (2.8s or so) even on my
very fast laptop.

We fix this is by moving the checks into a new Python library, so that
we can avoid paying the Django startup overhead 3 times unnecessarily.
The result saves about 1.2s (~40%) from the time required to run a
single backend test.

Fixes #1221.
2016-10-05 10:40:19 -07:00
umkay d260a22637 Add a new statistics/analytics framework.
This is a first pass at building a framework for collecting various
stats about realms, users, streams, etc. Includes:
* New analytics tables for storing counts data
* Raw SQL queries for pulling data from zerver/models.py tables
* Aggregation functions for aggregating hourly stats into daily stats, and
  aggregating user/stream level stats into realm level stats
* A management command for pulling the data

Note that counts.py was added to the linter exclude list due to errors
around %%s.
2016-10-04 17:18:54 -07:00
Tim Abbott 5ed37beb02 test-backend: Simplify argument rewriting logic. 2016-09-28 11:52:05 -07:00
sonali0901 4869e1b0b2 test-backend: Add convenience arguments to run subsets of the tests.
This optimizes the process of running individual or small groups of
backend tests (./tools/test-backend
zerver.tests.test_bugdown.FencedBlockPreprocessorTest.test_simple_quoting)
to allow the following syntaxes:

    ./tools/test-backend zerver/tests/test_bugdown.py
    ./tools/test-backend zerver.tests.test_bugdown.py
    ./tools/test-backend zerver/tests/test_bugdown
    ./tools/test-backend zerver.tests.test_bugdown
    ./tools/test-backend test_bugdown.py
    ./tools/test-backend test_bugdown
    ./tools/test-backend FencedBlockPreprocessorTest
    ./tools/test-backend FencedBlockPreprocessorTest.test_simple_quoting

Fixes #1670.
2016-09-28 11:52:05 -07:00
Steve Howell 21f83afe3a Add --url-coverage option to ./tools/test-backend. 2016-07-29 21:23:33 -07:00
Steve Howell 4556bf528f Add --report-slow-tests option to tools/test-backend. 2016-07-29 16:41:20 -07:00
Taranjeet Singh 9db457e8fa provision.py: Create var/coverage directory to store coverage data.
test-backend: Update coverage directory to var/coverage.

This commit updates the coverage directory to var/coverage as a part
of Issue 1132.
2016-07-13 18:58:00 -07:00
Tim Abbott 497142d7b1 Fix running test-backend from any working directory. 2016-07-12 10:39:33 -07:00
Eklavya Sharma 0de3b17f19 tools/test-backend: Add option to skip generate-fixtures.
Not calling generate-fixtures can reduce running time, which is
especially helpful when running a single test.
2016-07-12 09:34:37 +05:30
Tim Abbott c8aae360be test-backend: Fix spelling of an example in usage text. 2016-06-28 16:24:58 -07:00