selftests/net: packetdrill: report benign debug flakes as xfail
authorWillem de Bruijn <willemb@google.com>
Fri, 3 Jan 2025 11:31:14 +0000 (06:31 -0500)
committerPaolo Abeni <pabeni@redhat.com>
Tue, 7 Jan 2025 10:23:37 +0000 (11:23 +0100)
commit912d6f6697251b0024e56ed24b7873b4800822e7
tree3aa1cf7cf03f8043565f626bff8e931bc4480e83
parent51cfbed198ca4aa140babde816387db0e71f09e7
selftests/net: packetdrill: report benign debug flakes as xfail

A few recently added packetdrill tests that are known time sensitive
(e.g., because testing timestamping) occasionally fail in debug mode:
https://netdev.bots.linux.dev/contest.html?executor=vmksft-packetdrill-dbg

These failures are well understood. Correctness of the tests is
verified in non-debug mode. Continue running in debug mode also, to
keep coverage with debug instrumentation.

But, only in debug mode, mark these tests with well understood
timing issues as XFAIL (known failing) rather than FAIL when failing.

Introduce an allow list xfail_list with known cases.

Expand the ktap infrastructure with XFAIL support.

Fixes: eab35989cc37 ("selftests/net: packetdrill: import tcp/fast_recovery, tcp/nagle, tcp/timestamping")
Reported-by: Jakub Kicinski <kuba@kernel.org>
Closes: https://lore.kernel.org/netdev/20241218100013.0c698629@kernel.org/
Signed-off-by: Willem de Bruijn <willemb@google.com>
Link: https://patch.msgid.link/20250103113142.129251-1-willemdebruijn.kernel@gmail.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
tools/testing/selftests/kselftest/ktap_helpers.sh
tools/testing/selftests/net/packetdrill/ksft_runner.sh