1 .. SPDX-License-Identifier: GPL-2.0
7 The purpose of this document is to describe what KUnit is, how it works, how it
8 is intended to be used, and all the concepts and terminology that are needed to
9 understand it. This guide assumes a working knowledge of the Linux kernel and
10 some basic knowledge of testing.
12 For a high level introduction to KUnit, including setting up KUnit for your
13 project, see :doc:`start`.
15 Organization of this document
16 =============================
18 This document is organized into two main sections: Testing and Isolating
19 Behavior. The first covers what unit tests are and how to use KUnit to write
20 them. The second covers how to use KUnit to isolate code and make it possible
21 to unit test code that was otherwise un-unit-testable.
29 "K" is short for "kernel" so "KUnit" is the "(Linux) Kernel Unit Testing
30 Framework." KUnit is intended first and foremost for writing unit tests; it is
31 general enough that it can be used to write integration tests; however, this is
32 a secondary goal. KUnit has no ambition of being the only testing framework for
33 the kernel; for example, it does not intend to be an end-to-end testing
39 A `unit test <https://martinfowler.com/bliki/UnitTest.html>`_ is a test that
40 tests code at the smallest possible scope, a *unit* of code. In the C
41 programming language that's a function.
43 Unit tests should be written for all the publicly exposed functions in a
44 compilation unit; so that is all the functions that are exported in either a
45 *class* (defined below) or all functions which are **not** static.
53 The fundamental unit in KUnit is the test case. A test case is a function with
54 the signature ``void (*)(struct kunit *test)``. It calls a function to be tested
55 and then sets *expectations* for what should happen. For example:
59 void example_test_success(struct kunit *test)
63 void example_test_failure(struct kunit *test)
65 KUNIT_FAIL(test, "This test never passes.");
68 In the above example ``example_test_success`` always passes because it does
69 nothing; no expectations are set, so all expectations pass. On the other hand
70 ``example_test_failure`` always fails because it calls ``KUNIT_FAIL``, which is
71 a special expectation that logs a message and causes the test case to fail.
75 An *expectation* is a way to specify that you expect a piece of code to do
76 something in a test. An expectation is called like a function. A test is made
77 by setting expectations about the behavior of a piece of code under test; when
78 one or more of the expectations fail, the test case fails and information about
79 the failure is logged. For example:
83 void add_test_basic(struct kunit *test)
85 KUNIT_EXPECT_EQ(test, 1, add(1, 0));
86 KUNIT_EXPECT_EQ(test, 2, add(1, 1));
89 In the above example ``add_test_basic`` makes a number of assertions about the
90 behavior of a function called ``add``; the first parameter is always of type
91 ``struct kunit *``, which contains information about the current test context;
92 the second parameter, in this case, is what the value is expected to be; the
93 last value is what the value actually is. If ``add`` passes all of these
94 expectations, the test case, ``add_test_basic`` will pass; if any one of these
95 expectations fail, the test case will fail.
97 It is important to understand that a test case *fails* when any expectation is
98 violated; however, the test will continue running, potentially trying other
99 expectations until the test case ends or is otherwise terminated. This is as
100 opposed to *assertions* which are discussed later.
102 To learn about more expectations supported by KUnit, see :doc:`api/test`.
105 A single test case should be pretty short, pretty easy to understand,
106 focused on a single behavior.
108 For example, if we wanted to properly test the add function above, we would
109 create additional tests cases which would each test a different property that an
110 add function should have like this:
114 void add_test_basic(struct kunit *test)
116 KUNIT_EXPECT_EQ(test, 1, add(1, 0));
117 KUNIT_EXPECT_EQ(test, 2, add(1, 1));
120 void add_test_negative(struct kunit *test)
122 KUNIT_EXPECT_EQ(test, 0, add(-1, 1));
125 void add_test_max(struct kunit *test)
127 KUNIT_EXPECT_EQ(test, INT_MAX, add(0, INT_MAX));
128 KUNIT_EXPECT_EQ(test, -1, add(INT_MAX, INT_MIN));
131 void add_test_overflow(struct kunit *test)
133 KUNIT_EXPECT_EQ(test, INT_MIN, add(INT_MAX, 1));
136 Notice how it is immediately obvious what all the properties that we are testing
142 KUnit also has the concept of an *assertion*. An assertion is just like an
143 expectation except the assertion immediately terminates the test case if it is
150 static void mock_test_do_expect_default_return(struct kunit *test)
152 struct mock_test_context *ctx = test->priv;
153 struct mock *mock = ctx->mock;
154 int param0 = 5, param1 = -5;
155 const char *two_param_types[] = {"int", "int"};
156 const void *two_params[] = {¶m0, ¶m1};
159 ret = mock->do_expect(mock,
160 "test_printk", test_printk,
161 two_param_types, two_params,
162 ARRAY_SIZE(two_params));
163 KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ret);
164 KUNIT_EXPECT_EQ(test, -4, *((int *) ret));
167 In this example, the method under test should return a pointer to a value, so
168 if the pointer returned by the method is null or an errno, we don't want to
169 bother continuing the test since the following expectation could crash the test
170 case. `ASSERT_NOT_ERR_OR_NULL(...)` allows us to bail out of the test case if
171 the appropriate conditions have not been satisfied to complete the test.
176 Now obviously one unit test isn't very helpful; the power comes from having
177 many test cases covering all of a unit's behaviors. Consequently it is common
178 to have many *similar* tests; in order to reduce duplication in these closely
179 related tests most unit testing frameworks - including KUnit - provide the
180 concept of a *test suite*. A *test suite* is just a collection of test cases
181 for a unit of code with a set up function that gets invoked before every test
182 case and then a tear down function that gets invoked after every test case
189 static struct kunit_case example_test_cases[] = {
190 KUNIT_CASE(example_test_foo),
191 KUNIT_CASE(example_test_bar),
192 KUNIT_CASE(example_test_baz),
196 static struct kunit_suite example_test_suite = {
198 .init = example_test_init,
199 .exit = example_test_exit,
200 .test_cases = example_test_cases,
202 kunit_test_suite(example_test_suite);
204 In the above example the test suite, ``example_test_suite``, would run the test
205 cases ``example_test_foo``, ``example_test_bar``, and ``example_test_baz``,
206 each would have ``example_test_init`` called immediately before it and would
207 have ``example_test_exit`` called immediately after it.
208 ``kunit_test_suite(example_test_suite)`` registers the test suite with the
209 KUnit test framework.
212 A test case will only be run if it is associated with a test suite.
214 ``kunit_test_suite(...)`` is a macro which tells the linker to put the specified
215 test suite in a special linker section so that it can be run by KUnit either
216 after late_init, or when the test module is loaded (depending on whether the
217 test was built in or not).
219 For more information on these types of things see the :doc:`api/test`.
224 The most important aspect of unit testing that other forms of testing do not
225 provide is the ability to limit the amount of code under test to a single unit.
226 In practice, this is only possible by being able to control what code gets run
227 when the unit under test calls a function and this is usually accomplished
228 through some sort of indirection where a function is exposed as part of an API
229 such that the definition of that function can be changed without affecting the
230 rest of the code base. In the kernel this primarily comes from two constructs,
231 classes, structs that contain function pointers that are provided by the
232 implementer, and architecture specific functions which have definitions selected
238 Classes are not a construct that is built into the C programming language;
239 however, it is an easily derived concept. Accordingly, pretty much every project
240 that does not use a standardized object oriented library (like GNOME's GObject)
241 has their own slightly different way of doing object oriented programming; the
242 Linux kernel is no exception.
244 The central concept in kernel object oriented programming is the class. In the
245 kernel, a *class* is a struct that contains function pointers. This creates a
246 contract between *implementers* and *users* since it forces them to use the
247 same function signature without having to call the function directly. In order
248 for it to truly be a class, the function pointers must specify that a pointer
249 to the class, known as a *class handle*, be one of the parameters; this makes
250 it possible for the member functions (also known as *methods*) to have access
251 to member variables (more commonly known as *fields*) allowing the same
252 implementation to have multiple *instances*.
254 Typically a class can be *overridden* by *child classes* by embedding the
255 *parent class* in the child class. Then when a method provided by the child
256 class is called, the child implementation knows that the pointer passed to it is
257 of a parent contained within the child; because of this, the child can compute
258 the pointer to itself because the pointer to the parent is always a fixed offset
259 from the pointer to the child; this offset is the offset of the parent contained
260 in the child struct. For example:
265 int (*area)(struct shape *this);
274 int rectangle_area(struct shape *this)
276 struct rectangle *self = container_of(this, struct shape, parent);
278 return self->length * self->width;
281 void rectangle_new(struct rectangle *self, int length, int width)
283 self->parent.area = rectangle_area;
284 self->length = length;
288 In this example (as in most kernel code) the operation of computing the pointer
289 to the child from the pointer to the parent is done by ``container_of``.
294 In order to unit test a piece of code that calls a method in a class, the
295 behavior of the method must be controllable, otherwise the test ceases to be a
296 unit test and becomes an integration test.
298 A fake just provides an implementation of a piece of code that is different than
299 what runs in a production instance, but behaves identically from the standpoint
300 of the callers; this is usually done to replace a dependency that is hard to
301 deal with, or is slow.
303 A good example for this might be implementing a fake EEPROM that just stores the
304 "contents" in an internal buffer. For example, let's assume we have a class that
305 represents an EEPROM:
310 ssize_t (*read)(struct eeprom *this, size_t offset, char *buffer, size_t count);
311 ssize_t (*write)(struct eeprom *this, size_t offset, const char *buffer, size_t count);
314 And we want to test some code that buffers writes to the EEPROM:
318 struct eeprom_buffer {
319 ssize_t (*write)(struct eeprom_buffer *this, const char *buffer, size_t count);
320 int flush(struct eeprom_buffer *this);
321 size_t flush_count; /* Flushes when buffer exceeds flush_count. */
324 struct eeprom_buffer *new_eeprom_buffer(struct eeprom *eeprom);
325 void destroy_eeprom_buffer(struct eeprom *eeprom);
327 We can easily test this code by *faking out* the underlying EEPROM:
332 struct eeprom parent;
333 char contents[FAKE_EEPROM_CONTENTS_SIZE];
336 ssize_t fake_eeprom_read(struct eeprom *parent, size_t offset, char *buffer, size_t count)
338 struct fake_eeprom *this = container_of(parent, struct fake_eeprom, parent);
340 count = min(count, FAKE_EEPROM_CONTENTS_SIZE - offset);
341 memcpy(buffer, this->contents + offset, count);
346 ssize_t fake_eeprom_write(struct eeprom *parent, size_t offset, const char *buffer, size_t count)
348 struct fake_eeprom *this = container_of(parent, struct fake_eeprom, parent);
350 count = min(count, FAKE_EEPROM_CONTENTS_SIZE - offset);
351 memcpy(this->contents + offset, buffer, count);
356 void fake_eeprom_init(struct fake_eeprom *this)
358 this->parent.read = fake_eeprom_read;
359 this->parent.write = fake_eeprom_write;
360 memset(this->contents, 0, FAKE_EEPROM_CONTENTS_SIZE);
363 We can now use it to test ``struct eeprom_buffer``:
367 struct eeprom_buffer_test {
368 struct fake_eeprom *fake_eeprom;
369 struct eeprom_buffer *eeprom_buffer;
372 static void eeprom_buffer_test_does_not_write_until_flush(struct kunit *test)
374 struct eeprom_buffer_test *ctx = test->priv;
375 struct eeprom_buffer *eeprom_buffer = ctx->eeprom_buffer;
376 struct fake_eeprom *fake_eeprom = ctx->fake_eeprom;
377 char buffer[] = {0xff};
379 eeprom_buffer->flush_count = SIZE_MAX;
381 eeprom_buffer->write(eeprom_buffer, buffer, 1);
382 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[0], 0);
384 eeprom_buffer->write(eeprom_buffer, buffer, 1);
385 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[1], 0);
387 eeprom_buffer->flush(eeprom_buffer);
388 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[0], 0xff);
389 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[1], 0xff);
392 static void eeprom_buffer_test_flushes_after_flush_count_met(struct kunit *test)
394 struct eeprom_buffer_test *ctx = test->priv;
395 struct eeprom_buffer *eeprom_buffer = ctx->eeprom_buffer;
396 struct fake_eeprom *fake_eeprom = ctx->fake_eeprom;
397 char buffer[] = {0xff};
399 eeprom_buffer->flush_count = 2;
401 eeprom_buffer->write(eeprom_buffer, buffer, 1);
402 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[0], 0);
404 eeprom_buffer->write(eeprom_buffer, buffer, 1);
405 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[0], 0xff);
406 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[1], 0xff);
409 static void eeprom_buffer_test_flushes_increments_of_flush_count(struct kunit *test)
411 struct eeprom_buffer_test *ctx = test->priv;
412 struct eeprom_buffer *eeprom_buffer = ctx->eeprom_buffer;
413 struct fake_eeprom *fake_eeprom = ctx->fake_eeprom;
414 char buffer[] = {0xff, 0xff};
416 eeprom_buffer->flush_count = 2;
418 eeprom_buffer->write(eeprom_buffer, buffer, 1);
419 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[0], 0);
421 eeprom_buffer->write(eeprom_buffer, buffer, 2);
422 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[0], 0xff);
423 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[1], 0xff);
424 /* Should have only flushed the first two bytes. */
425 KUNIT_EXPECT_EQ(test, fake_eeprom->contents[2], 0);
428 static int eeprom_buffer_test_init(struct kunit *test)
430 struct eeprom_buffer_test *ctx;
432 ctx = kunit_kzalloc(test, sizeof(*ctx), GFP_KERNEL);
433 KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ctx);
435 ctx->fake_eeprom = kunit_kzalloc(test, sizeof(*ctx->fake_eeprom), GFP_KERNEL);
436 KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ctx->fake_eeprom);
437 fake_eeprom_init(ctx->fake_eeprom);
439 ctx->eeprom_buffer = new_eeprom_buffer(&ctx->fake_eeprom->parent);
440 KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ctx->eeprom_buffer);
447 static void eeprom_buffer_test_exit(struct kunit *test)
449 struct eeprom_buffer_test *ctx = test->priv;
451 destroy_eeprom_buffer(ctx->eeprom_buffer);
454 .. _kunit-on-non-uml:
456 KUnit on non-UML architectures
457 ==============================
459 By default KUnit uses UML as a way to provide dependencies for code under test.
460 Under most circumstances KUnit's usage of UML should be treated as an
461 implementation detail of how KUnit works under the hood. Nevertheless, there
462 are instances where being able to run architecture specific code or test
463 against real hardware is desirable. For these reasons KUnit supports running on
466 Running existing KUnit tests on non-UML architectures
467 -----------------------------------------------------
469 There are some special considerations when running existing KUnit tests on
470 non-UML architectures:
472 * Hardware may not be deterministic, so a test that always passes or fails
473 when run under UML may not always do so on real hardware.
474 * Hardware and VM environments may not be hermetic. KUnit tries its best to
475 provide a hermetic environment to run tests; however, it cannot manage state
476 that it doesn't know about outside of the kernel. Consequently, tests that
477 may be hermetic on UML may not be hermetic on other architectures.
478 * Some features and tooling may not be supported outside of UML.
479 * Hardware and VMs are slower than UML.
481 None of these are reasons not to run your KUnit tests on real hardware; they are
482 only things to be aware of when doing so.
484 The biggest impediment will likely be that certain KUnit features and
485 infrastructure may not support your target environment. For example, at this
486 time the KUnit Wrapper (``tools/testing/kunit/kunit.py``) does not work outside
487 of UML. Unfortunately, there is no way around this. Using UML (or even just a
488 particular architecture) allows us to make a lot of assumptions that make it
489 possible to do things which might otherwise be impossible.
491 Nevertheless, all core KUnit framework features are fully supported on all
492 architectures, and using them is straightforward: all you need to do is to take
493 your kunitconfig, your Kconfig options for the tests you would like to run, and
494 merge them into whatever config your are using for your platform. That's it!
496 For example, let's say you have the following kunitconfig:
501 CONFIG_KUNIT_EXAMPLE_TEST=y
503 If you wanted to run this test on an x86 VM, you might add the following config
504 options to your ``.config``:
509 CONFIG_KUNIT_EXAMPLE_TEST=y
511 CONFIG_SERIAL_8250_CONSOLE=y
513 All these new options do is enable support for a common serial console needed
516 Next, you could build a kernel with these tests as follows:
521 make ARCH=x86 olddefconfig
524 Once you have built a kernel, you could run it on QEMU as follows:
528 qemu-system-x86_64 -enable-kvm \
530 -kernel arch/x86_64/boot/bzImage \
531 -append 'console=ttyS0' \
534 Interspersed in the kernel logs you might see the following:
541 # example_simple_test: initializing
542 ok 1 - example_simple_test
545 Congratulations, you just ran a KUnit test on the x86 architecture!
547 In a similar manner, kunit and kunit tests can also be built as modules,
548 so if you wanted to run tests in this way you might add the following config
549 options to your ``.config``:
554 CONFIG_KUNIT_EXAMPLE_TEST=m
556 Once the kernel is built and installed, a simple
560 modprobe example-test
562 ...will run the tests.
564 Writing new tests for other architectures
565 -----------------------------------------
567 The first thing you must do is ask yourself whether it is necessary to write a
568 KUnit test for a specific architecture, and then whether it is necessary to
569 write that test for a particular piece of hardware. In general, writing a test
570 that depends on having access to a particular piece of hardware or software (not
571 included in the Linux source repo) should be avoided at all costs.
573 Even if you only ever plan on running your KUnit test on your hardware
574 configuration, other people may want to run your tests and may not have access
575 to your hardware. If you write your test to run on UML, then anyone can run your
576 tests without knowing anything about your particular setup, and you can still
577 run your tests on your hardware setup just by compiling for your architecture.
580 Always prefer tests that run on UML to tests that only run under a particular
581 architecture, and always prefer tests that run under QEMU or another easy
582 (and monetarily free) to obtain software environment to a specific piece of
585 Nevertheless, there are still valid reasons to write an architecture or hardware
586 specific test: for example, you might want to test some code that really belongs
587 in ``arch/some-arch/*``. Even so, try your best to write the test so that it
588 does not depend on physical hardware: if some of your test cases don't need the
589 hardware, only require the hardware for tests that actually need it.
591 Now that you have narrowed down exactly what bits are hardware specific, the
592 actual procedure for writing and running the tests is pretty much the same as
593 writing normal KUnit tests. One special caveat is that you have to reset
594 hardware state in between test cases; if this is not possible, you may only be
595 able to run one test case per invocation.
597 .. TODO(brendanhiggins@google.com): Add an actual example of an architecture
598 dependent KUnit test.
600 KUnit debugfs representation
601 ============================
602 When kunit test suites are initialized, they create an associated directory
603 in ``/sys/kernel/debug/kunit/<test-suite>``. The directory contains one file
605 - results: "cat results" displays results of each test case and the results
606 of the entire suite for the last test run.
608 The debugfs representation is primarily of use when kunit test suites are
609 run in a native environment, either as modules or builtin. Having a way
610 to display results like this is valuable as otherwise results can be
611 intermixed with other events in dmesg output. The maximum size of each
612 results file is KUNIT_LOG_SIZE bytes (defined in ``include/kunit/test.h``).