We now instrument URL coverage whenever you run the back end tests,
and if you run the full suite and fail to test all endpoints, we
exit with a non-zero exit code and report failures to you.
If you are running just a subset of the test suite, you'll still
be able to see var/url_coverage.txt, which has some useful info.
With some tweaks to the output from tabbott.
Fixes#1441.
These are not particularly interesting to measure test coverage for,
since migrations are not run as the part of the test suite by
construction, and management commands aren't being tested by this test
suite.
Previously, running `tools/test-backend analytics/` (or any other test suite
name ending with a '/') would give a cryptic error about modules not
importing properly. This commit rstrip's the trailing slash from test suite
names given on the command line.
Previously, the generate-fixtures shell script by called into Django
multiple times in order to check whether the database was in a
reasonable state. Since there's a lot of overhead to starting up
Django, this resulted in `test-backend` and `test-js-with-casper`
being quite slow to run a single small test (2.8s or so) even on my
very fast laptop.
We fix this is by moving the checks into a new Python library, so that
we can avoid paying the Django startup overhead 3 times unnecessarily.
The result saves about 1.2s (~40%) from the time required to run a
single backend test.
Fixes#1221.
This is a first pass at building a framework for collecting various
stats about realms, users, streams, etc. Includes:
* New analytics tables for storing counts data
* Raw SQL queries for pulling data from zerver/models.py tables
* Aggregation functions for aggregating hourly stats into daily stats, and
aggregating user/stream level stats into realm level stats
* A management command for pulling the data
Note that counts.py was added to the linter exclude list due to errors
around %%s.
This optimizes the process of running individual or small groups of
backend tests (./tools/test-backend
zerver.tests.test_bugdown.FencedBlockPreprocessorTest.test_simple_quoting)
to allow the following syntaxes:
./tools/test-backend zerver/tests/test_bugdown.py
./tools/test-backend zerver.tests.test_bugdown.py
./tools/test-backend zerver/tests/test_bugdown
./tools/test-backend zerver.tests.test_bugdown
./tools/test-backend test_bugdown.py
./tools/test-backend test_bugdown
./tools/test-backend FencedBlockPreprocessorTest
./tools/test-backend FencedBlockPreprocessorTest.test_simple_quoting
Fixes#1670.
Since relatively few systems have the typing module, this makes the
checks for whether the user is properly running our test scripts in
the virtualenv more likely to trigger well.
This should make users much more likely to be able to debug issues
where they ran Zulip outside the Vagrant environment or virtualenv.
[error messages tweaked by tabbott]
Add two options to the `test-backend` script:
1. verbose
If given the `test-backend` script will give detailed output.
2. no-shallow
Default value is False. If given the `test-backend` script will
fail if it finds a template which is shallow tested.
Add call to tools/generate-fixtures in tools/test-backend before
starting the tests. Previously, test-backend could fail if called
after tools/test-js-with-casper had failed.
Fixes#501.
The #! line processing interpreted the argument to pass to `env` as
"python2.7 -u", which obviously isn't a real program.
We fix this by setting the PYTHONUNBUFFERED environment variable
inside the program, which has the same effect.
Thanks to Dan Fedele for the bug report and suggested solution!
This includes a hack to preserve humbug/backends.py as a symlink, so
that we don't need to regenerate all our old sessions.
(imported from commit b7918988b31c71ec01bbdc270db7017d4069221d)
This needs to be deployed to both staging and prod at the same
off-peak time (and the schema migration run).
At the time it is deployed, we need to make a few changes directly in
the database:
(1) UPDATE django_content_type set app_label='zerver' where app_label='zephyr';
(2) UPDATE south_migrationhistory set app_name='zerver' where app_name='zephyr';
(imported from commit eb3fd719571740189514ef0b884738cb30df1320)
Does a test-all using /tmp.
Not safe to run with other tests simultaneously.
You use it just like tools/review, and on a successful test-all
it will submit the code review, without tying up your git working directory.
It depends on new patches to the acrefoot-bulk_create_with_id-1.5.1 branch of
django. Updating this is the [manual] step required. Since test-all doesn't happen
on staging or prod, this is not required for this commit to be deployed, but other
django patches will help with the SSL connection errors we've been seeing.
(imported from commit 3fd3ff00240a2d648c4c54748a8a70616067ef7d)
Currently our test database is backed by sqlite; this commit moves
us to using postgres for our all database needs. This, in conjunction
with the patched django on github, allow us to have fewer hacks and
more true-to-life tests. It also sets the stage for testing the bulk_create
and schema search_path patches made to django.
Developers will need to run:
./tools/postgres-init-test-db
./tools/do-destroy-rebuild-test-database
this is assuming that they have already run:
./tools/postgres-init-db
./tools/do-destroy-rebuild-database
at some point on this pg_cluster. (The ordering is important; it will other-
wise complain about the south_migration table).
(imported from commit c56c6f27e13df7ae10b2e643e65d669dde61af3d)
Tomasz Kolek
Tim Abbott
reallyly
Adarsh S
Steve Howell
paxapy
sonali0901
Rishi Gupta
umkay
Taranjeet Singh
Tim Abbott
Eklavya Sharma
Aakash Tyagi
Alex Wilson
Ernesto Vargas
Umair Khan
Vladislav Manchev
Steven Oud
Tim Abbott
Tim Abbott
acrefoot