bpf: Invoke __bpf_prog_exit_sleepable_recur() on recursion in kern_sys_bpf().
authorSebastian Andrzej Siewior <bigeasy@linutronix.de>
Wed, 30 Aug 2023 08:04:04 +0000 (10:04 +0200)
committerDaniel Borkmann <daniel@iogearbox.net>
Wed, 6 Sep 2023 08:39:31 +0000 (10:39 +0200)
commit7645629f7dc88cd777f98970134bf1a54c8d77e3
treedde5b29cdd03e8b33e1e902e4b7d1f538786d17b
parenta454d84ee20baf7bd7be90721b9821f73c7d23d9
bpf: Invoke __bpf_prog_exit_sleepable_recur() on recursion in kern_sys_bpf().

If __bpf_prog_enter_sleepable_recur() detects recursion then it returns
0 without undoing rcu_read_lock_trace(), migrate_disable() or
decrementing the recursion counter. This is fine in the JIT case because
the JIT code will jump in the 0 case to the end and invoke the matching
exit trampoline (__bpf_prog_exit_sleepable_recur()).

This is not the case in kern_sys_bpf() which returns directly to the
caller with an error code.

Add __bpf_prog_exit_sleepable_recur() as clean up in the recursion case.

Fixes: b1d18a7574d0d ("bpf: Extend sys_bpf commands for bpf_syscall programs.")
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/bpf/20230830080405.251926-2-bigeasy@linutronix.de
kernel/bpf/syscall.c