linux/lib/kunit
David Gow acd8e8407b kunit: Print test statistics on failure
When a number of tests fail, it can be useful to get higher-level
statistics of how many tests are failing (or how many parameters are
failing in parameterised tests), and in what cases or suites. This is
already done by some non-KUnit tests, so add support for automatically
generating these for KUnit tests.

This change adds a 'kunit.stats_enabled' switch which has three values:
- 0: No stats are printed (current behaviour)
- 1: Stats are printed only for tests/suites with more than one
     subtest (new default)
- 2: Always print test statistics

For parameterised tests, the summary line looks as follows:
"    # inode_test_xtimestamp_decoding: pass:16 fail:0 skip:0 total:16"
For test suites, there are two lines looking like this:
"# ext4_inode_test: pass:1 fail:0 skip:0 total:1"
"# Totals: pass:16 fail:0 skip:0 total:16"

The first line gives the number of direct subtests, the second "Totals"
line is the accumulated sum of all tests and test parameters.

This format is based on the one used by kselftest[1].

[1]: https://elixir.bootlin.com/linux/latest/source/tools/testing/selftests/kselftest.h#L109

Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-08-13 13:38:31 -06:00
..
.kunitconfig kunit: tool: make --kunitconfig accept dirs, add lib/kunit fragment 2021-04-02 14:14:36 -06:00
assert.c kunit: make KUNIT_EXPECT_STREQ() quote values, don't print literals 2021-04-02 14:14:16 -06:00
debugfs.c kunit: Support skipped tests 2021-06-25 11:31:03 -06:00
debugfs.h kunit: add debugfs /sys/kernel/debug/kunit/<suite>/results display 2020-03-26 14:07:18 -06:00
executor_test.c kunit: add unit test for filtering suites by names 2021-06-25 11:44:37 -06:00
executor.c kunit: add unit test for filtering suites by names 2021-06-25 11:44:37 -06:00
Kconfig kunit: add kunit.filter_glob cmdline option to filter suites 2021-02-08 16:09:27 -07:00
kunit-example-test.c kunit: test: Add example tests which are always skipped 2021-06-25 11:31:03 -06:00
kunit-test.c kunit: Support skipped tests 2021-06-25 11:31:03 -06:00
Makefile kunit: test: create a single centralized executor for all tests 2020-10-09 14:37:34 -06:00
string-stream-test.c kunit: allow kunit tests to be loaded as a module 2020-01-09 16:42:29 -07:00
string-stream.c kunit: generalize kunit_resource API beyond allocated resources 2020-06-26 14:12:00 -06:00
string-stream.h kunit: Add gnu_printf specifiers 2021-06-23 17:06:04 -06:00
test.c kunit: Print test statistics on failure 2021-08-13 13:38:31 -06:00
try-catch-impl.h kunit: hide unexported try-catch interface in try-catch-impl.h 2020-01-09 16:42:09 -07:00
try-catch.c kunit: remove timeout dependence on sysctl_hung_task_timeout_seconds 2020-01-09 16:42:55 -07:00