io_uring/kbuf: prune deferred locked cache when tearing down

We used to just use our page list for final teardown, which would ensure
that we got all the buffers, even the ones that were not on the normal
cached list. But while moving to slab for the io_buffers, we know only
prune this list, not the deferred locked list that we have. This can
cause a leak of memory, if the workload ends up using the intermediate
locked list.

Fix this by always pruning both lists when tearing down.

Fixes: b3a4dbc89d40 ("io_uring/kbuf: Use slab for struct io_buffer objects")
Signed-off-by: Jens Axboe <axboe@kernel.dk>

diff --git a/io_uring/kbuf.c b/io_uring/kbuf.c
index 325ca7f..39d15a2 100644 (file)
--- a/io_uring/kbuf.c
+++ b/io_uring/kbuf.c
@@ -306,6 +306,14 @@ void io_destroy_buffers(struct io_ring_ctx *ctx)
                kfree(bl);
        }
 
+       /*
+        * Move deferred locked entries to cache before pruning
+        */
+       spin_lock(&ctx->completion_lock);
+       if (!list_empty(&ctx->io_buffers_comp))
+               list_splice_init(&ctx->io_buffers_comp, &ctx->io_buffers_cache);
+       spin_unlock(&ctx->completion_lock);
+
        list_for_each_safe(item, tmp, &ctx->io_buffers_cache) {
                buf = list_entry(item, struct io_buffer, list);
                kmem_cache_free(io_buf_cachep, buf);