This commit changes the backend testing framework to run
in parallel mode which is same as --processes=4. If --coverage
is supplied, we enforce serial mode, --processes=1, because
coverage is not compatible with parallel mode at the moment.
This fixes the fact that our test suites would have trouble connecting
to the other parts of the Zulip service when run with a proxy
configuration (e.g. trying to send requests to localhost through the
proxy!).
Thanks for Vishnu Ks for his work on this.
Fixes#971.
In backend tests, only call generate-fixtures when --generate-fixtures
is explicitly passed or is_template_database_current() returns False.
We don't need to flush cache for backend tests because we bounce the key
prefix used to create cache keys before running every test
This adds the option '--rerun' to the `test-backend` infrastructure.
It runs the tests that failed during the last 'test-backend' run. It
works by stailing failed test info at var/last_test_failure.json
Cleaned up by Umair Khan and Tim Abbott.
This code was added as part of the Django 1.10 migration to make our
tests work with both Django 1.8 and 1.10. Now that we're on 1.10,
it's no longer required.
Like many rare-case code with new tests, it turns out that the logic
for handling null characters in our Zephyr postgres query escaping
never worked, in multiple ways. First, it always changed the second
character in s, not the current one being inspected, and second, the
value it replaced it with was no the correct postgres escape of the
null byte. We fix this and add tests.
This completes the effort to get zerver/views/messages.py to 100%
test coverage.
Fixes#1006.
This adds an assertion, when `test-backend` is run with `--coverage`,
that we have 100% test coverage on a list of files that we expect to.
There's a whitelist/blacklist, managed in tools/test-backend.
Fixes#3363.
This helps make the Zulip development environment somewhat more robust
to new contributors, since it will give them a nice warning if they
try running any of our development tools outside the Zulip virtualenv.
Fixes#3468.
We now instrument URL coverage whenever you run the back end tests,
and if you run the full suite and fail to test all endpoints, we
exit with a non-zero exit code and report failures to you.
If you are running just a subset of the test suite, you'll still
be able to see var/url_coverage.txt, which has some useful info.
With some tweaks to the output from tabbott.
Fixes#1441.
These are not particularly interesting to measure test coverage for,
since migrations are not run as the part of the test suite by
construction, and management commands aren't being tested by this test
suite.
Previously, running `tools/test-backend analytics/` (or any other test suite
name ending with a '/') would give a cryptic error about modules not
importing properly. This commit rstrip's the trailing slash from test suite
names given on the command line.
Previously, the generate-fixtures shell script by called into Django
multiple times in order to check whether the database was in a
reasonable state. Since there's a lot of overhead to starting up
Django, this resulted in `test-backend` and `test-js-with-casper`
being quite slow to run a single small test (2.8s or so) even on my
very fast laptop.
We fix this is by moving the checks into a new Python library, so that
we can avoid paying the Django startup overhead 3 times unnecessarily.
The result saves about 1.2s (~40%) from the time required to run a
single backend test.
Fixes#1221.
This is a first pass at building a framework for collecting various
stats about realms, users, streams, etc. Includes:
* New analytics tables for storing counts data
* Raw SQL queries for pulling data from zerver/models.py tables
* Aggregation functions for aggregating hourly stats into daily stats, and
aggregating user/stream level stats into realm level stats
* A management command for pulling the data
Note that counts.py was added to the linter exclude list due to errors
around %%s.
This optimizes the process of running individual or small groups of
backend tests (./tools/test-backend
zerver.tests.test_bugdown.FencedBlockPreprocessorTest.test_simple_quoting)
to allow the following syntaxes:
./tools/test-backend zerver/tests/test_bugdown.py
./tools/test-backend zerver.tests.test_bugdown.py
./tools/test-backend zerver/tests/test_bugdown
./tools/test-backend zerver.tests.test_bugdown
./tools/test-backend test_bugdown.py
./tools/test-backend test_bugdown
./tools/test-backend FencedBlockPreprocessorTest
./tools/test-backend FencedBlockPreprocessorTest.test_simple_quoting
Fixes#1670.
Since relatively few systems have the typing module, this makes the
checks for whether the user is properly running our test scripts in
the virtualenv more likely to trigger well.
This should make users much more likely to be able to debug issues
where they ran Zulip outside the Vagrant environment or virtualenv.
[error messages tweaked by tabbott]
Add two options to the `test-backend` script:
1. verbose
If given the `test-backend` script will give detailed output.
2. no-shallow
Default value is False. If given the `test-backend` script will
fail if it finds a template which is shallow tested.
Add call to tools/generate-fixtures in tools/test-backend before
starting the tests. Previously, test-backend could fail if called
after tools/test-js-with-casper had failed.
Fixes#501.
The #! line processing interpreted the argument to pass to `env` as
"python2.7 -u", which obviously isn't a real program.
We fix this by setting the PYTHONUNBUFFERED environment variable
inside the program, which has the same effect.
Thanks to Dan Fedele for the bug report and suggested solution!
This includes a hack to preserve humbug/backends.py as a symlink, so
that we don't need to regenerate all our old sessions.
(imported from commit b7918988b31c71ec01bbdc270db7017d4069221d)
Umair Khan
K.Kanakhin
Ayush Jain
Adarsh S
Tim Abbott
Elliott Jin
sinwar
Tomasz Kolek
reallyly
Steve Howell
paxapy
sonali0901
Rishi Gupta
umkay
Taranjeet Singh
Tim Abbott
Eklavya Sharma
Aakash Tyagi
Alex Wilson
Ernesto Vargas
Vladislav Manchev
Steven Oud
Tim Abbott
Tim Abbott