Commit e2ed78d5 authored by Linus Torvalds's avatar Linus Torvalds

Merge tag 'linux-kselftest-kunit-next-6.2-rc1' of...

Merge tag 'linux-kselftest-kunit-next-6.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest

Pull KUnit updates from Shuah Khan:
 "Several enhancements, fixes, clean-ups, documentation updates,
  improvements to logging and KTAP compliance of KUnit test output:

   - log numbers in decimal and hex

   - parse KTAP compliant test output

   - allow conditionally exposing static symbols to tests when KUNIT is
     enabled

   - make static symbols visible during kunit testing

   - clean-ups to remove unused structure definition"

* tag 'linux-kselftest-kunit-next-6.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest: (29 commits)
  Documentation: dev-tools: Clarify requirements for result description
  apparmor: test: make static symbols visible during kunit testing
  kunit: add macro to allow conditionally exposing static symbols to tests
  kunit: tool: make parser preserve whitespace when printing test log
  Documentation: kunit: Fix "How Do I Use This" / "Next Steps" sections
  kunit: tool: don't include KTAP headers and the like in the test log
  kunit: improve KTAP compliance of KUnit test output
  kunit: tool: parse KTAP compliant test output
  mm: slub: test: Use the kunit_get_current_test() function
  kunit: Use the static key when retrieving the current test
  kunit: Provide a static key to check if KUnit is actively running tests
  kunit: tool: make --json do nothing if --raw_ouput is set
  kunit: tool: tweak error message when no KTAP found
  kunit: remove KUNIT_INIT_MEM_ASSERTION macro
  Documentation: kunit: Remove redundant 'tips.rst' page
  Documentation: KUnit: reword description of assertions
  Documentation: KUnit: make usage.rst a superset of tips.rst, remove duplication
  kunit: eliminate KUNIT_INIT_*_ASSERT_STRUCT macros
  kunit: tool: remove redundant file.close() call in unit test
  kunit: tool: unit tests all check parser errors, standardize formatting a bit
  ...
parents 23a68d14 054be257
......@@ -80,8 +80,8 @@ have the number 1 and the number then must increase by 1 for each additional
subtest within the same test at the same nesting level.
The description is a description of the test, generally the name of
the test, and can be any string of words (can't include #). The
description is optional, but recommended.
the test, and can be any string of characters other than # or a
newline. The description is optional, but recommended.
The directive and any diagnostic data is optional. If either are present, they
must follow a hash sign, "#".
......
......@@ -16,7 +16,6 @@ KUnit - Linux Kernel Unit Testing
api/index
style
faq
tips
running_tips
This section details the kernel unit testing framework.
......@@ -100,14 +99,11 @@ Read also :ref:`kinds-of-tests`.
How do I use it?
================
* Documentation/dev-tools/kunit/start.rst - for KUnit new users.
* Documentation/dev-tools/kunit/architecture.rst - KUnit architecture.
* Documentation/dev-tools/kunit/run_wrapper.rst - run kunit_tool.
* Documentation/dev-tools/kunit/run_manual.rst - run tests without kunit_tool.
* Documentation/dev-tools/kunit/usage.rst - write tests.
* Documentation/dev-tools/kunit/tips.rst - best practices with
examples.
* Documentation/dev-tools/kunit/api/index.rst - KUnit APIs
used for testing.
* Documentation/dev-tools/kunit/faq.rst - KUnit common questions and
answers.
You can find a step-by-step guide to writing and running KUnit tests in
Documentation/dev-tools/kunit/start.rst
Alternatively, feel free to look through the rest of the KUnit documentation,
or to experiment with tools/testing/kunit/kunit.py and the example test under
lib/kunit/kunit-example-test.c
Happy testing!
......@@ -294,13 +294,11 @@ Congrats! You just wrote your first KUnit test.
Next Steps
==========
* Documentation/dev-tools/kunit/architecture.rst - KUnit architecture.
* Documentation/dev-tools/kunit/run_wrapper.rst - run kunit_tool.
* Documentation/dev-tools/kunit/run_manual.rst - run tests without kunit_tool.
* Documentation/dev-tools/kunit/usage.rst - write tests.
* Documentation/dev-tools/kunit/tips.rst - best practices with
examples.
* Documentation/dev-tools/kunit/api/index.rst - KUnit APIs
used for testing.
* Documentation/dev-tools/kunit/faq.rst - KUnit common questions and
answers.
If you're interested in using some of the more advanced features of kunit.py,
take a look at Documentation/dev-tools/kunit/run_wrapper.rst
If you'd like to run tests without using kunit.py, check out
Documentation/dev-tools/kunit/run_manual.rst
For more information on writing KUnit tests (including some common techniques
for testing different things), see Documentation/dev-tools/kunit/usage.rst
.. SPDX-License-Identifier: GPL-2.0
============================
Tips For Writing KUnit Tests
============================
Exiting early on failed expectations
------------------------------------
``KUNIT_EXPECT_EQ`` and friends will mark the test as failed and continue
execution. In some cases, it's unsafe to continue and you can use the
``KUNIT_ASSERT`` variant to exit on failure.
.. code-block:: c
void example_test_user_alloc_function(struct kunit *test)
{
void *object = alloc_some_object_for_me();
/* Make sure we got a valid pointer back. */
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, object);
do_something_with_object(object);
}
Allocating memory
-----------------
Where you would use ``kzalloc``, you should prefer ``kunit_kzalloc`` instead.
KUnit will ensure the memory is freed once the test completes.
This is particularly useful since it lets you use the ``KUNIT_ASSERT_EQ``
macros to exit early from a test without having to worry about remembering to
call ``kfree``.
Example:
.. code-block:: c
void example_test_allocation(struct kunit *test)
{
char *buffer = kunit_kzalloc(test, 16, GFP_KERNEL);
/* Ensure allocation succeeded. */
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, buffer);
KUNIT_ASSERT_STREQ(test, buffer, "");
}
Testing static functions
------------------------
If you don't want to expose functions or variables just for testing, one option
is to conditionally ``#include`` the test file at the end of your .c file, e.g.
.. code-block:: c
/* In my_file.c */
static int do_interesting_thing();
#ifdef CONFIG_MY_KUNIT_TEST
#include "my_kunit_test.c"
#endif
Injecting test-only code
------------------------
Similarly to the above, it can be useful to add test-specific logic.
.. code-block:: c
/* In my_file.h */
#ifdef CONFIG_MY_KUNIT_TEST
/* Defined in my_kunit_test.c */
void test_only_hook(void);
#else
void test_only_hook(void) { }
#endif
This test-only code can be made more useful by accessing the current kunit
test, see below.
Accessing the current test
--------------------------
In some cases, you need to call test-only code from outside the test file, e.g.
like in the example above or if you're providing a fake implementation of an
ops struct.
There is a ``kunit_test`` field in ``task_struct``, so you can access it via
``current->kunit_test``.
Here's a slightly in-depth example of how one could implement "mocking":
.. code-block:: c
#include <linux/sched.h> /* for current */
struct test_data {
int foo_result;
int want_foo_called_with;
};
static int fake_foo(int arg)
{
struct kunit *test = current->kunit_test;
struct test_data *test_data = test->priv;
KUNIT_EXPECT_EQ(test, test_data->want_foo_called_with, arg);
return test_data->foo_result;
}
static void example_simple_test(struct kunit *test)
{
/* Assume priv is allocated in the suite's .init */
struct test_data *test_data = test->priv;
test_data->foo_result = 42;
test_data->want_foo_called_with = 1;
/* In a real test, we'd probably pass a pointer to fake_foo somewhere
* like an ops struct, etc. instead of calling it directly. */
KUNIT_EXPECT_EQ(test, fake_foo(1), 42);
}
Note: here we're able to get away with using ``test->priv``, but if you wanted
something more flexible you could use a named ``kunit_resource``, see
Documentation/dev-tools/kunit/api/test.rst.
Failing the current test
------------------------
But sometimes, you might just want to fail the current test. In that case, we
have ``kunit_fail_current_test(fmt, args...)`` which is defined in ``<kunit/test-bug.h>`` and
doesn't require pulling in ``<kunit/test.h>``.
E.g. say we had an option to enable some extra debug checks on some data structure:
.. code-block:: c
#include <kunit/test-bug.h>
#ifdef CONFIG_EXTRA_DEBUG_CHECKS
static void validate_my_data(struct data *data)
{
if (is_valid(data))
return;
kunit_fail_current_test("data %p is invalid", data);
/* Normal, non-KUnit, error reporting code here. */
}
#else
static void my_debug_function(void) { }
#endif
Customizing error messages
--------------------------
Each of the ``KUNIT_EXPECT`` and ``KUNIT_ASSERT`` macros have a ``_MSG`` variant.
These take a format string and arguments to provide additional context to the automatically generated error messages.
.. code-block:: c
char some_str[41];
generate_sha1_hex_string(some_str);
/* Before. Not easy to tell why the test failed. */
KUNIT_EXPECT_EQ(test, strlen(some_str), 40);
/* After. Now we see the offending string. */
KUNIT_EXPECT_EQ_MSG(test, strlen(some_str), 40, "some_str='%s'", some_str);
Alternatively, one can take full control over the error message by using ``KUNIT_FAIL()``, e.g.
.. code-block:: c
/* Before */
KUNIT_EXPECT_EQ(test, some_setup_function(), 0);
/* After: full control over the failure message. */
if (some_setup_function())
KUNIT_FAIL(test, "Failed to setup thing for testing");
Next Steps
==========
* Optional: see the Documentation/dev-tools/kunit/usage.rst page for a more
in-depth explanation of KUnit.
......@@ -112,11 +112,45 @@ terminates the test case if the condition is not satisfied. For example:
KUNIT_EXPECT_LE(test, a[i], a[i + 1]);
}
In this example, the method under test should return pointer to a value. If the
pointer returns null or an errno, we want to stop the test since the following
expectation could crash the test case. `ASSERT_NOT_ERR_OR_NULL(...)` allows us
to bail out of the test case if the appropriate conditions are not satisfied to
complete the test.
In this example, we need to be able to allocate an array to test the ``sort()``
function. So we use ``KUNIT_ASSERT_NOT_ERR_OR_NULL()`` to abort the test if
there's an allocation error.
.. note::
In other test frameworks, ``ASSERT`` macros are often implemented by calling
``return`` so they only work from the test function. In KUnit, we stop the
current kthread on failure, so you can call them from anywhere.
Customizing error messages
--------------------------
Each of the ``KUNIT_EXPECT`` and ``KUNIT_ASSERT`` macros have a ``_MSG``
variant. These take a format string and arguments to provide additional
context to the automatically generated error messages.
.. code-block:: c
char some_str[41];
generate_sha1_hex_string(some_str);
/* Before. Not easy to tell why the test failed. */
KUNIT_EXPECT_EQ(test, strlen(some_str), 40);
/* After. Now we see the offending string. */
KUNIT_EXPECT_EQ_MSG(test, strlen(some_str), 40, "some_str='%s'", some_str);
Alternatively, one can take full control over the error message by using
``KUNIT_FAIL()``, e.g.
.. code-block:: c
/* Before */
KUNIT_EXPECT_EQ(test, some_setup_function(), 0);
/* After: full control over the failure message. */
if (some_setup_function())
KUNIT_FAIL(test, "Failed to setup thing for testing");
Test Suites
~~~~~~~~~~~
......@@ -546,24 +580,6 @@ By reusing the same ``cases`` array from above, we can write the test as a
{}
};
Exiting Early on Failed Expectations
------------------------------------
We can use ``KUNIT_EXPECT_EQ`` to mark the test as failed and continue
execution. In some cases, it is unsafe to continue. We can use the
``KUNIT_ASSERT`` variant to exit on failure.
.. code-block:: c
void example_test_user_alloc_function(struct kunit *test)
{
void *object = alloc_some_object_for_me();
/* Make sure we got a valid pointer back. */
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, object);
do_something_with_object(object);
}
Allocating Memory
-----------------
......@@ -625,17 +641,23 @@ as shown in next section: *Accessing The Current Test*.
Accessing The Current Test
--------------------------
In some cases, we need to call test-only code from outside the test file.
For example, see example in section *Injecting Test-Only Code* or if
we are providing a fake implementation of an ops struct. Using
``kunit_test`` field in ``task_struct``, we can access it via
``current->kunit_test``.
In some cases, we need to call test-only code from outside the test file. This
is helpful, for example, when providing a fake implementation of a function, or
to fail any current test from within an error handler.
We can do this via the ``kunit_test`` field in ``task_struct``, which we can
access using the ``kunit_get_current_test()`` function in ``kunit/test-bug.h``.
``kunit_get_current_test()`` is safe to call even if KUnit is not enabled. If
KUnit is not enabled, was built as a module (``CONFIG_KUNIT=m``), or no test is
running in the current task, it will return ``NULL``. This compiles down to
either a no-op or a static key check, so will have a negligible performance
impact when no test is running.
The example below includes how to implement "mocking":
The example below uses this to implement a "mock" implementation of a function, ``foo``:
.. code-block:: c
#include <linux/sched.h> /* for current */
#include <kunit/test-bug.h> /* for kunit_get_current_test */
struct test_data {
int foo_result;
......@@ -644,7 +666,7 @@ The example below includes how to implement "mocking":
static int fake_foo(int arg)
{
struct kunit *test = current->kunit_test;
struct kunit *test = kunit_get_current_test();
struct test_data *test_data = test->priv;
KUNIT_EXPECT_EQ(test, test_data->want_foo_called_with, arg);
......@@ -675,7 +697,7 @@ Each test can have multiple resources which have string names providing the same
flexibility as a ``priv`` member, but also, for example, allowing helper
functions to create resources without conflicting with each other. It is also
possible to define a clean up function for each resource, making it easy to
avoid resource leaks. For more information, see Documentation/dev-tools/kunit/api/test.rst.
avoid resource leaks. For more information, see Documentation/dev-tools/kunit/api/resource.rst.
Failing The Current Test
------------------------
......@@ -703,3 +725,9 @@ structures as shown below:
static void my_debug_function(void) { }
#endif
``kunit_fail_current_test()`` is safe to call even if KUnit is not enabled. If
KUnit is not enabled, was built as a module (``CONFIG_KUNIT=m``), or no test is
running in the current task, it will do nothing. This compiles down to either a
no-op or a static key check, so will have a negligible performance impact when
no test is running.
......@@ -315,7 +315,7 @@ static void drm_test_fb_xrgb8888_to_gray8(struct kunit *test)
iosys_map_set_vaddr(&src, xrgb8888);
drm_fb_xrgb8888_to_gray8(&dst, &result->dst_pitch, &src, &fb, &params->clip);
KUNIT_EXPECT_EQ(test, memcmp(buf, result->expected, dst_size), 0);
KUNIT_EXPECT_MEMEQ(test, buf, result->expected, dst_size);
}
static void drm_test_fb_xrgb8888_to_rgb332(struct kunit *test)
......@@ -345,7 +345,7 @@ static void drm_test_fb_xrgb8888_to_rgb332(struct kunit *test)
iosys_map_set_vaddr(&src, xrgb8888);
drm_fb_xrgb8888_to_rgb332(&dst, &result->dst_pitch, &src, &fb, &params->clip);
KUNIT_EXPECT_EQ(test, memcmp(buf, result->expected, dst_size), 0);
KUNIT_EXPECT_MEMEQ(test, buf, result->expected, dst_size);
}
static void drm_test_fb_xrgb8888_to_rgb565(struct kunit *test)
......@@ -375,10 +375,10 @@ static void drm_test_fb_xrgb8888_to_rgb565(struct kunit *test)
iosys_map_set_vaddr(&src, xrgb8888);
drm_fb_xrgb8888_to_rgb565(&dst, &result->dst_pitch, &src, &fb, &params->clip, false);
KUNIT_EXPECT_EQ(test, memcmp(buf, result->expected, dst_size), 0);
KUNIT_EXPECT_MEMEQ(test, buf, result->expected, dst_size);
drm_fb_xrgb8888_to_rgb565(&dst, &result->dst_pitch, &src, &fb, &params->clip, true);
KUNIT_EXPECT_EQ(test, memcmp(buf, result->expected_swab, dst_size), 0);
KUNIT_EXPECT_MEMEQ(test, buf, result->expected_swab, dst_size);
}
static void drm_test_fb_xrgb8888_to_rgb888(struct kunit *test)
......@@ -408,7 +408,7 @@ static void drm_test_fb_xrgb8888_to_rgb888(struct kunit *test)
iosys_map_set_vaddr(&src, xrgb8888);
drm_fb_xrgb8888_to_rgb888(&dst, &result->dst_pitch, &src, &fb, &params->clip);
KUNIT_EXPECT_EQ(test, memcmp(buf, result->expected, dst_size), 0);
KUNIT_EXPECT_MEMEQ(test, buf, result->expected, dst_size);
}
static void drm_test_fb_xrgb8888_to_xrgb2101010(struct kunit *test)
......@@ -439,7 +439,7 @@ static void drm_test_fb_xrgb8888_to_xrgb2101010(struct kunit *test)
drm_fb_xrgb8888_to_xrgb2101010(&dst, &result->dst_pitch, &src, &fb, &params->clip);
buf = le32buf_to_cpu(test, buf, dst_size / sizeof(u32));
KUNIT_EXPECT_EQ(test, memcmp(buf, result->expected, dst_size), 0);
KUNIT_EXPECT_MEMEQ(test, buf, result->expected, dst_size);
}
static struct kunit_case drm_format_helper_test_cases[] = {
......
......@@ -90,19 +90,6 @@ void kunit_unary_assert_format(const struct kunit_assert *assert,
const struct va_format *message,
struct string_stream *stream);
/**
* KUNIT_INIT_UNARY_ASSERT_STRUCT() - Initializes &struct kunit_unary_assert.
* @cond: A string representation of the expression asserted true or false.
* @expect_true: True if of type KUNIT_{EXPECT|ASSERT}_TRUE, false otherwise.
*
* Initializes a &struct kunit_unary_assert. Intended to be used in
* KUNIT_EXPECT_* and KUNIT_ASSERT_* macros.
*/
#define KUNIT_INIT_UNARY_ASSERT_STRUCT(cond, expect_true) { \
.condition = cond, \
.expected_true = expect_true \
}
/**
* struct kunit_ptr_not_err_assert - An expectation/assertion that a pointer is
* not NULL and not a -errno.
......@@ -123,20 +110,6 @@ void kunit_ptr_not_err_assert_format(const struct kunit_assert *assert,
const struct va_format *message,
struct string_stream *stream);
/**
* KUNIT_INIT_PTR_NOT_ERR_ASSERT_STRUCT() - Initializes a
* &struct kunit_ptr_not_err_assert.
* @txt: A string representation of the expression passed to the expectation.
* @val: The actual evaluated pointer value of the expression.
*
* Initializes a &struct kunit_ptr_not_err_assert. Intended to be used in
* KUNIT_EXPECT_* and KUNIT_ASSERT_* macros.
*/
#define KUNIT_INIT_PTR_NOT_ERR_STRUCT(txt, val) { \
.text = txt, \
.value = val \
}
/**
* struct kunit_binary_assert_text - holds strings for &struct
* kunit_binary_assert and friends to try and make the structs smaller.
......@@ -173,27 +146,6 @@ void kunit_binary_assert_format(const struct kunit_assert *assert,
const struct va_format *message,
struct string_stream *stream);
/**
* KUNIT_INIT_BINARY_ASSERT_STRUCT() - Initializes a binary assert like
* kunit_binary_assert, kunit_binary_ptr_assert, etc.
*
* @text_: Pointer to a kunit_binary_assert_text.
* @left_val: The actual evaluated value of the expression in the left slot.
* @right_val: The actual evaluated value of the expression in the right slot.
*
* Initializes a binary assert like kunit_binary_assert,
* kunit_binary_ptr_assert, etc. This relies on these structs having the same
* fields but with different types for left_val/right_val.
* This is ultimately used by binary assertion macros like KUNIT_EXPECT_EQ, etc.
*/
#define KUNIT_INIT_BINARY_ASSERT_STRUCT(text_, \
left_val, \
right_val) { \
.text = text_, \
.left_value = left_val, \
.right_value = right_val \
}
/**
* struct kunit_binary_ptr_assert - An expectation/assertion that compares two
* pointer values (for example, KUNIT_EXPECT_PTR_EQ(test, foo, bar)).
......@@ -240,4 +192,30 @@ void kunit_binary_str_assert_format(const struct kunit_assert *assert,
const struct va_format *message,
struct string_stream *stream);
/**
* struct kunit_mem_assert - An expectation/assertion that compares two
* memory blocks.
* @assert: The parent of this type.
* @text: Holds the textual representations of the operands and comparator.
* @left_value: The actual evaluated value of the expression in the left slot.
* @right_value: The actual evaluated value of the expression in the right slot.
* @size: Size of the memory block analysed in bytes.
*
* Represents an expectation/assertion that compares two memory blocks. For
* example, to expect that the first three bytes of foo is equal to the
* first three bytes of bar, you can use the expectation
* KUNIT_EXPECT_MEMEQ(test, foo, bar, 3);
*/
struct kunit_mem_assert {
struct kunit_assert assert;
const struct kunit_binary_assert_text *text;
const void *left_value;
const void *right_value;
const size_t size;
};
void kunit_mem_assert_format(const struct kunit_assert *assert,
const struct va_format *message,
struct string_stream *stream);
#endif /* _KUNIT_ASSERT_H */
......@@ -9,16 +9,63 @@
#ifndef _KUNIT_TEST_BUG_H
#define _KUNIT_TEST_BUG_H
#define kunit_fail_current_test(fmt, ...) \
__kunit_fail_current_test(__FILE__, __LINE__, fmt, ##__VA_ARGS__)
#if IS_BUILTIN(CONFIG_KUNIT)
#include <linux/jump_label.h> /* For static branch */
#include <linux/sched.h>
/* Static key if KUnit is running any tests. */
DECLARE_STATIC_KEY_FALSE(kunit_running);
/**
* kunit_get_current_test() - Return a pointer to the currently running
* KUnit test.
*
* If a KUnit test is running in the current task, returns a pointer to its
* associated struct kunit. This pointer can then be passed to any KUnit
* function or assertion. If no test is running (or a test is running in a
* different task), returns NULL.
*
* This function is safe to call even when KUnit is disabled. If CONFIG_KUNIT
* is not enabled, it will compile down to nothing and will return quickly no
* test is running.
*/
static inline struct kunit *kunit_get_current_test(void)
{
if (!static_branch_unlikely(&kunit_running))
return NULL;
return current->kunit_test;
}
/**
* kunit_fail_current_test() - If a KUnit test is running, fail it.
*
* If a KUnit test is running in the current task, mark that test as failed.
*
* This macro will only work if KUnit is built-in (though the tests
* themselves can be modules). Otherwise, it compiles down to nothing.
*/
#define kunit_fail_current_test(fmt, ...) do { \
if (static_branch_unlikely(&kunit_running)) { \
__kunit_fail_current_test(__FILE__, __LINE__, \
fmt, ##__VA_ARGS__); \
} \
} while (0)
extern __printf(3, 4) void __kunit_fail_current_test(const char *file, int line,
const char *fmt, ...);
#else
static inline struct kunit *kunit_get_current_test(void) { return NULL; }
/* We define this with an empty helper function so format string warnings work */
#define kunit_fail_current_test(fmt, ...) \
__kunit_fail_current_test(__FILE__, __LINE__, fmt, ##__VA_ARGS__)
static inline __printf(3, 4) void __kunit_fail_current_test(const char *file, int line,
const char *fmt, ...)
{
......
......@@ -16,6 +16,7 @@
#include <linux/container_of.h>
#include <linux/err.h>
#include <linux/init.h>
#include <linux/jump_label.h>
#include <linux/kconfig.h>
#include <linux/kref.h>
#include <linux/list.h>
......@@ -27,6 +28,9 @@
#include <asm/rwonce.h>
/* Static key: true if any KUnit tests are currently running */
DECLARE_STATIC_KEY_FALSE(kunit_running);
struct kunit;
/* Size of log associated with test. */
......@@ -515,22 +519,25 @@ void kunit_do_failed_assertion(struct kunit *test,
fmt, \
##__VA_ARGS__)
/* Helper to safely pass around an initializer list to other macros. */
#define KUNIT_INIT_ASSERT(initializers...) { initializers }
#define KUNIT_UNARY_ASSERTION(test, \
assert_type, \
condition, \
expected_true, \
condition_, \
expected_true_, \
fmt, \
...) \
do { \
if (likely(!!(condition) == !!expected_true)) \
if (likely(!!(condition_) == !!expected_true_)) \
break; \
\
_KUNIT_FAILED(test, \
assert_type, \
kunit_unary_assert, \
kunit_unary_assert_format, \
KUNIT_INIT_UNARY_ASSERT_STRUCT(#condition, \
expected_true), \
KUNIT_INIT_ASSERT(.condition = #condition_, \
.expected_true = expected_true_), \
fmt, \
##__VA_ARGS__); \
} while (0)
......@@ -590,9 +597,9 @@ do { \
assert_type, \
assert_class, \
format_func, \
KUNIT_INIT_BINARY_ASSERT_STRUCT(&__text, \
__left, \
__right), \
KUNIT_INIT_ASSERT(.text = &__text, \
.left_value = __left, \
.right_value = __right), \
fmt, \
##__VA_ARGS__); \
} while (0)
......@@ -651,9 +658,42 @@ do { \
assert_type, \
kunit_binary_str_assert, \
kunit_binary_str_assert_format, \
KUNIT_INIT_BINARY_ASSERT_STRUCT(&__text, \
__left, \
__right), \
KUNIT_INIT_ASSERT(.text = &__text, \
.left_value = __left, \
.right_value = __right), \
fmt, \
##__VA_ARGS__); \
} while (0)
#define KUNIT_MEM_ASSERTION(test, \
assert_type, \
left, \
op, \
right, \
size_, \
fmt, \
...) \
do { \
const void *__left = (left); \
const void *__right = (right); \
const size_t __size = (size_); \
static const struct kunit_binary_assert_text __text = { \
.operation = #op, \
.left_text = #left, \
.right_text = #right, \
}; \
\
if (likely(memcmp(__left, __right, __size) op 0)) \
break; \
\
_KUNIT_FAILED(test, \
assert_type, \
kunit_mem_assert, \
kunit_mem_assert_format, \
KUNIT_INIT_ASSERT(.text = &__text, \
.left_value = __left, \
.right_value = __right, \
.size = __size), \
fmt, \
##__VA_ARGS__); \
} while (0)
......@@ -673,7 +713,7 @@ do { \
assert_type, \
kunit_ptr_not_err_assert, \
kunit_ptr_not_err_assert_format, \
KUNIT_INIT_PTR_NOT_ERR_STRUCT(#ptr, __ptr), \
KUNIT_INIT_ASSERT(.text = #ptr, .value = __ptr), \
fmt, \
##__VA_ARGS__); \
} while (0)
......@@ -928,6 +968,60 @@ do { \
fmt, \
##__VA_ARGS__)
/**
* KUNIT_EXPECT_MEMEQ() - Expects that the first @size bytes of @left and @right are equal.
* @test: The test context object.
* @left: An arbitrary expression that evaluates to the specified size.
* @right: An arbitrary expression that evaluates to the specified size.
* @size: Number of bytes compared.
*
* Sets an expectation that the values that @left and @right evaluate to are
* equal. This is semantically equivalent to
* KUNIT_EXPECT_TRUE(@test, !memcmp((@left), (@right), (@size))). See
* KUNIT_EXPECT_TRUE() for more information.
*
* Although this expectation works for any memory block, it is not recommended
* for comparing more structured data, such as structs. This expectation is
* recommended for comparing, for example, data arrays.
*/
#define KUNIT_EXPECT_MEMEQ(test, left, right, size) \
KUNIT_EXPECT_MEMEQ_MSG(test, left, right, size, NULL)
#define KUNIT_EXPECT_MEMEQ_MSG(test, left, right, size, fmt, ...) \
KUNIT_MEM_ASSERTION(test, \
KUNIT_EXPECTATION, \
left, ==, right, \
size, \
fmt, \
##__VA_ARGS__)
/**
* KUNIT_EXPECT_MEMNEQ() - Expects that the first @size bytes of @left and @right are not equal.
* @test: The test context object.
* @left: An arbitrary expression that evaluates to the specified size.
* @right: An arbitrary expression that evaluates to the specified size.
* @size: Number of bytes compared.
*
* Sets an expectation that the values that @left and @right evaluate to are
* not equal. This is semantically equivalent to
* KUNIT_EXPECT_TRUE(@test, memcmp((@left), (@right), (@size))). See
* KUNIT_EXPECT_TRUE() for more information.
*
* Although this expectation works for any memory block, it is not recommended
* for comparing more structured data, such as structs. This expectation is
* recommended for comparing, for example, data arrays.
*/
#define KUNIT_EXPECT_MEMNEQ(test, left, right, size) \
KUNIT_EXPECT_MEMNEQ_MSG(test, left, right, size, NULL)
#define KUNIT_EXPECT_MEMNEQ_MSG(test, left, right, size, fmt, ...) \
KUNIT_MEM_ASSERTION(test, \
KUNIT_EXPECTATION, \
left, !=, right, \
size, \
fmt, \
##__VA_ARGS__)
/**
* KUNIT_EXPECT_NULL() - Expects that @ptr is null.
* @test: The test context object.
......
/* SPDX-License-Identifier: GPL-2.0 */
/*
* KUnit API to allow symbols to be conditionally visible during KUnit
* testing
*
* Copyright (C) 2022, Google LLC.
* Author: Rae Moar <rmoar@google.com>
*/
#ifndef _KUNIT_VISIBILITY_H
#define _KUNIT_VISIBILITY_H
#if IS_ENABLED(CONFIG_KUNIT)
/**
* VISIBLE_IF_KUNIT - A macro that sets symbols to be static if
* CONFIG_KUNIT is not enabled. Otherwise if CONFIG_KUNIT is enabled
* there is no change to the symbol definition.
*/
#define VISIBLE_IF_KUNIT
/**
* EXPORT_SYMBOL_IF_KUNIT(symbol) - Exports symbol into
* EXPORTED_FOR_KUNIT_TESTING namespace only if CONFIG_KUNIT is
* enabled. Must use MODULE_IMPORT_NS(EXPORTED_FOR_KUNIT_TESTING)
* in test file in order to use symbols.
*/
#define EXPORT_SYMBOL_IF_KUNIT(symbol) EXPORT_SYMBOL_NS(symbol, \
EXPORTED_FOR_KUNIT_TESTING)
#else
#define VISIBLE_IF_KUNIT static
#define EXPORT_SYMBOL_IF_KUNIT(symbol)
#endif
#endif /* _KUNIT_VISIBILITY_H */
......@@ -127,13 +127,15 @@ void kunit_binary_assert_format(const struct kunit_assert *assert,
binary_assert->text->right_text);
if (!is_literal(stream->test, binary_assert->text->left_text,
binary_assert->left_value, stream->gfp))
string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s == %lld\n",
string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s == %lld (0x%llx)\n",
binary_assert->text->left_text,
binary_assert->left_value,
binary_assert->left_value);
if (!is_literal(stream->test, binary_assert->text->right_text,
binary_assert->right_value, stream->gfp))
string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s == %lld",
string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s == %lld (0x%llx)",
binary_assert->text->right_text,
binary_assert->right_value,
binary_assert->right_value);
kunit_assert_print_msg(message, stream);
}
......@@ -204,3 +206,59 @@ void kunit_binary_str_assert_format(const struct kunit_assert *assert,
kunit_assert_print_msg(message, stream);
}
EXPORT_SYMBOL_GPL(kunit_binary_str_assert_format);
/* Adds a hexdump of a buffer to a string_stream comparing it with
* a second buffer. The different bytes are marked with <>.
*/
static void kunit_assert_hexdump(struct string_stream *stream,
const void *buf,
const void *compared_buf,
const size_t len)
{
size_t i;
const u8 *buf1 = buf;
const u8 *buf2 = compared_buf;
string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT);
for (i = 0; i < len; ++i) {
if (!(i % 16) && i)
string_stream_add(stream, "\n" KUNIT_SUBSUBTEST_INDENT);
if (buf1[i] != buf2[i])
string_stream_add(stream, "<%02x>", buf1[i]);
else
string_stream_add(stream, " %02x ", buf1[i]);
}
}
void kunit_mem_assert_format(const struct kunit_assert *assert,
const struct va_format *message,
struct string_stream *stream)
{
struct kunit_mem_assert *mem_assert;
mem_assert = container_of(assert, struct kunit_mem_assert,
assert);
string_stream_add(stream,
KUNIT_SUBTEST_INDENT "Expected %s %s %s, but\n",
mem_assert->text->left_text,
mem_assert->text->operation,
mem_assert->text->right_text);
string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s ==\n",
mem_assert->text->left_text);
kunit_assert_hexdump(stream, mem_assert->left_value,
mem_assert->right_value, mem_assert->size);
string_stream_add(stream, "\n");
string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s ==\n",
mem_assert->text->right_text);
kunit_assert_hexdump(stream, mem_assert->right_value,
mem_assert->left_value, mem_assert->size);
kunit_assert_print_msg(message, stream);
}
EXPORT_SYMBOL_GPL(kunit_mem_assert_format);
......@@ -63,7 +63,7 @@ static int debugfs_print_results(struct seq_file *seq, void *v)
kunit_suite_for_each_test_case(suite, test_case)
debugfs_print_result(seq, suite, test_case);
seq_printf(seq, "%s %d - %s\n",
seq_printf(seq, "%s %d %s\n",
kunit_status_to_ok_not_ok(success), 1, suite->name);
return 0;
}
......
......@@ -166,7 +166,7 @@ static void kunit_exec_run_tests(struct suite_set *suite_set)
{
size_t num_suites = suite_set->end - suite_set->start;
pr_info("TAP version 14\n");
pr_info("KTAP version 1\n");
pr_info("1..%zu\n", num_suites);
__kunit_test_suites_init(suite_set->start, num_suites);
......@@ -177,8 +177,8 @@ static void kunit_exec_list_tests(struct suite_set *suite_set)
struct kunit_suite * const *suites;
struct kunit_case *test_case;
/* Hack: print a tap header so kunit.py can find the start of KUnit output. */
pr_info("TAP version 14\n");
/* Hack: print a ktap header so kunit.py can find the start of KUnit output. */
pr_info("KTAP version 1\n");
for (suites = suite_set->start; suites < suite_set->end; suites++)
kunit_suite_for_each_test_case((*suites), test_case) {
......
......@@ -86,6 +86,9 @@ static void example_mark_skipped_test(struct kunit *test)
*/
static void example_all_expect_macros_test(struct kunit *test)
{
const u32 array1[] = { 0x0F, 0xFF };
const u32 array2[] = { 0x1F, 0xFF };
/* Boolean assertions */
KUNIT_EXPECT_TRUE(test, true);
KUNIT_EXPECT_FALSE(test, false);
......@@ -109,6 +112,10 @@ static void example_all_expect_macros_test(struct kunit *test)
KUNIT_EXPECT_STREQ(test, "hi", "hi");
KUNIT_EXPECT_STRNEQ(test, "hi", "bye");
/* Memory block assertions */
KUNIT_EXPECT_MEMEQ(test, array1, array1, sizeof(array1));
KUNIT_EXPECT_MEMNEQ(test, array1, array2, sizeof(array1));
/*
* There are also ASSERT variants of all of the above that abort test
* execution if they fail. Useful for memory allocations, etc.
......
......@@ -131,11 +131,6 @@ bool string_stream_is_empty(struct string_stream *stream)
return list_empty(&stream->fragments);
}
struct string_stream_alloc_context {
struct kunit *test;
gfp_t gfp;
};
struct string_stream *alloc_string_stream(struct kunit *test, gfp_t gfp)
{
struct string_stream *stream;
......
......@@ -20,6 +20,8 @@
#include "string-stream.h"
#include "try-catch-impl.h"
DEFINE_STATIC_KEY_FALSE(kunit_running);
#if IS_BUILTIN(CONFIG_KUNIT)
/*
* Fail the current test and print an error message to the log.
......@@ -149,6 +151,7 @@ EXPORT_SYMBOL_GPL(kunit_suite_num_test_cases);
static void kunit_print_suite_start(struct kunit_suite *suite)
{
kunit_log(KERN_INFO, suite, KUNIT_SUBTEST_INDENT "KTAP version 1\n");
kunit_log(KERN_INFO, suite, KUNIT_SUBTEST_INDENT "# Subtest: %s",
suite->name);
kunit_log(KERN_INFO, suite, KUNIT_SUBTEST_INDENT "1..%zd",
......@@ -175,13 +178,13 @@ static void kunit_print_ok_not_ok(void *test_or_suite,
* representation.
*/
if (suite)
pr_info("%s %zd - %s%s%s\n",
pr_info("%s %zd %s%s%s\n",
kunit_status_to_ok_not_ok(status),
test_number, description, directive_header,
(status == KUNIT_SKIPPED) ? directive : "");
else
kunit_log(KERN_INFO, test,
KUNIT_SUBTEST_INDENT "%s %zd - %s%s%s",
KUNIT_SUBTEST_INDENT "%s %zd %s%s%s",
kunit_status_to_ok_not_ok(status),
test_number, description, directive_header,
(status == KUNIT_SKIPPED) ? directive : "");
......@@ -542,6 +545,8 @@ int kunit_run_tests(struct kunit_suite *suite)
/* Get initial param. */
param_desc[0] = '\0';
test.param_value = test_case->generate_params(NULL, param_desc);
kunit_log(KERN_INFO, &test, KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT
"KTAP version 1\n");
kunit_log(KERN_INFO, &test, KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT
"# Subtest: %s", test_case->name);
......@@ -555,7 +560,7 @@ int kunit_run_tests(struct kunit_suite *suite)
kunit_log(KERN_INFO, &test,
KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT
"%s %d - %s",
"%s %d %s",
kunit_status_to_ok_not_ok(test.status),
test.param_index + 1, param_desc);
......@@ -612,10 +617,14 @@ int __kunit_test_suites_init(struct kunit_suite * const * const suites, int num_
return 0;
}
static_branch_inc(&kunit_running);
for (i = 0; i < num_suites; i++) {
kunit_init_suite(suites[i]);
kunit_run_tests(suites[i]);
}
static_branch_dec(&kunit_running);
return 0;
}
EXPORT_SYMBOL_GPL(__kunit_test_suites_init);
......
// SPDX-License-Identifier: GPL-2.0
#include <kunit/test.h>
#include <kunit/test-bug.h>
#include <linux/mm.h>
#include <linux/slab.h>
#include <linux/module.h>
......
......@@ -39,6 +39,7 @@
#include <linux/memcontrol.h>
#include <linux/random.h>
#include <kunit/test.h>
#include <kunit/test-bug.h>
#include <linux/sort.h>
#include <linux/debugfs.h>
......@@ -618,7 +619,7 @@ static bool slab_add_kunit_errors(void)
{
struct kunit_resource *resource;
if (likely(!current->kunit_test))
if (!kunit_get_current_test())
return false;
resource = kunit_find_named_resource(current->kunit_test, "slab_errors");
......
......@@ -71,11 +71,11 @@ static void dev_addr_test_basic(struct kunit *test)
memset(addr, 2, sizeof(addr));
eth_hw_addr_set(netdev, addr);
KUNIT_EXPECT_EQ(test, 0, memcmp(netdev->dev_addr, addr, sizeof(addr)));
KUNIT_EXPECT_MEMEQ(test, netdev->dev_addr, addr, sizeof(addr));
memset(addr, 3, sizeof(addr));
dev_addr_set(netdev, addr);
KUNIT_EXPECT_EQ(test, 0, memcmp(netdev->dev_addr, addr, sizeof(addr)));
KUNIT_EXPECT_MEMEQ(test, netdev->dev_addr, addr, sizeof(addr));
}
static void dev_addr_test_sync_one(struct kunit *test)
......
......@@ -106,8 +106,8 @@ config SECURITY_APPARMOR_PARANOID_LOAD
Disabling the check will speed up policy loads.
config SECURITY_APPARMOR_KUNIT_TEST
bool "Build KUnit tests for policy_unpack.c" if !KUNIT_ALL_TESTS
depends on KUNIT=y && SECURITY_APPARMOR
tristate "Build KUnit tests for policy_unpack.c" if !KUNIT_ALL_TESTS
depends on KUNIT && SECURITY_APPARMOR
default KUNIT_ALL_TESTS
help
This builds the AppArmor KUnit tests.
......
......@@ -8,6 +8,9 @@ apparmor-y := apparmorfs.o audit.o capability.o task.o ipc.o lib.o match.o \
resource.o secid.o file.o policy_ns.o label.o mount.o net.o
apparmor-$(CONFIG_SECURITY_APPARMOR_HASH) += crypto.o
obj-$(CONFIG_SECURITY_APPARMOR_KUNIT_TEST) += apparmor_policy_unpack_test.o
apparmor_policy_unpack_test-objs += policy_unpack_test.o
clean-files := capability_names.h rlim_names.h net_names.h
# Build a lower case string table of address family names
......
......@@ -48,6 +48,43 @@ enum {
AAFS_LOADDATA_NDENTS /* count of entries */
};
/*
* The AppArmor interface treats data as a type byte followed by the
* actual data. The interface has the notion of a named entry
* which has a name (AA_NAME typecode followed by name string) followed by
* the entries typecode and data. Named types allow for optional
* elements and extensions to be added and tested for without breaking
* backwards compatibility.
*/
enum aa_code {
AA_U8,
AA_U16,
AA_U32,
AA_U64,
AA_NAME, /* same as string except it is items name */
AA_STRING,
AA_BLOB,
AA_STRUCT,
AA_STRUCTEND,
AA_LIST,
AA_LISTEND,
AA_ARRAY,
AA_ARRAYEND,
};
/*
* aa_ext is the read of the buffer containing the serialized profile. The
* data is copied into a kernel buffer in apparmorfs and then handed off to
* the unpack routines.
*/
struct aa_ext {
void *start;
void *end;
void *pos; /* pointer to current position in the buffer */
u32 version;
};
/*
* struct aa_loaddata - buffer of policy raw_data set
*
......@@ -126,4 +163,17 @@ static inline void aa_put_loaddata(struct aa_loaddata *data)
kref_put(&data->count, aa_loaddata_kref);
}
#if IS_ENABLED(CONFIG_KUNIT)
bool aa_inbounds(struct aa_ext *e, size_t size);
size_t aa_unpack_u16_chunk(struct aa_ext *e, char **chunk);
bool aa_unpack_X(struct aa_ext *e, enum aa_code code);
bool aa_unpack_nameX(struct aa_ext *e, enum aa_code code, const char *name);
bool aa_unpack_u32(struct aa_ext *e, u32 *data, const char *name);
bool aa_unpack_u64(struct aa_ext *e, u64 *data, const char *name);
size_t aa_unpack_array(struct aa_ext *e, const char *name);
size_t aa_unpack_blob(struct aa_ext *e, char **blob, const char *name);
int aa_unpack_str(struct aa_ext *e, const char **string, const char *name);
int aa_unpack_strdup(struct aa_ext *e, char **string, const char *name);
#endif
#endif /* __POLICY_INTERFACE_H */
This diff is collapsed.
This diff is collapsed.
......@@ -192,28 +192,30 @@ def _map_to_overall_status(test_status: kunit_parser.TestStatus) -> KunitStatus:
def parse_tests(request: KunitParseRequest, metadata: kunit_json.Metadata, input_data: Iterable[str]) -> Tuple[KunitResult, kunit_parser.Test]:
parse_start = time.time()
test_result = kunit_parser.Test()
if request.raw_output:
# Treat unparsed results as one passing test.
test_result.status = kunit_parser.TestStatus.SUCCESS
test_result.counts.passed = 1
fake_test = kunit_parser.Test()
fake_test.status = kunit_parser.TestStatus.SUCCESS
fake_test.counts.passed = 1
output: Iterable[str] = input_data
if request.raw_output == 'all':
pass
elif request.raw_output == 'kunit':
output = kunit_parser.extract_tap_lines(output, lstrip=False)
output = kunit_parser.extract_tap_lines(output)
for line in output:
print(line.rstrip())
parse_time = time.time() - parse_start
return KunitResult(KunitStatus.SUCCESS, parse_time), fake_test
else:
test_result = kunit_parser.parse_run_tests(input_data)
parse_end = time.time()
# Actually parse the test results.
test = kunit_parser.parse_run_tests(input_data)
parse_time = time.time() - parse_start
if request.json:
json_str = kunit_json.get_json_result(
test=test_result,
test=test,
metadata=metadata)
if request.json == 'stdout':
print(json_str)
......@@ -223,10 +225,10 @@ def parse_tests(request: KunitParseRequest, metadata: kunit_json.Metadata, input
stdout.print_with_timestamp("Test results stored in %s" %
os.path.abspath(request.json))
if test_result.status != kunit_parser.TestStatus.SUCCESS:
return KunitResult(KunitStatus.TEST_FAILURE, parse_end - parse_start), test_result
if test.status != kunit_parser.TestStatus.SUCCESS:
return KunitResult(KunitStatus.TEST_FAILURE, parse_time), test
return KunitResult(KunitStatus.SUCCESS, parse_end - parse_start), test_result
return KunitResult(KunitStatus.SUCCESS, parse_time), test
def run_tests(linux: kunit_kernel.LinuxSourceTree,
request: KunitRequest) -> KunitResult:
......@@ -359,14 +361,14 @@ def add_exec_opts(parser) -> None:
choices=['suite', 'test'])
def add_parse_opts(parser) -> None:
parser.add_argument('--raw_output', help='If set don\'t format output from kernel. '
'If set to --raw_output=kunit, filters to just KUnit output.',
parser.add_argument('--raw_output', help='If set don\'t parse output from kernel. '
'By default, filters to just KUnit output. Use '
'--raw_output=all to show everything',
type=str, nargs='?', const='all', default=None, choices=['all', 'kunit'])
parser.add_argument('--json',
nargs='?',
help='Stores test results in a JSON, and either '
'prints to stdout or saves to file if a '
'filename is specified',
help='Prints parsed test results as JSON to stdout or a file if '
'a filename is specified. Does nothing if --raw_output is set.',
type=str, const='stdout', default=None, metavar='FILE')
......
This diff is collapsed.
This diff is collapsed.
KTAP version 1
1..1
KTAP version 1
1..3
ok 1 case_1
ok 2 case_2
ok 3 case_3
ok 1 suite
KTAP version 1
1..1
KTAP version 1
# Subtest: suite
1..1
ok 1 test
ok 1 suite
\ No newline at end of file
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment