udp: use a separate rx queue for packet reception
authorPaolo Abeni <pabeni@redhat.com>
Tue, 16 May 2017 09:20:14 +0000 (11:20 +0200)
committerDavid S. Miller <davem@davemloft.net>
Tue, 16 May 2017 19:41:29 +0000 (15:41 -0400)
commit2276f58ac5890e58d2b6a48b95493faff7347e3a
treec77ce6fd0b942dfddb145e1ad6dbb2fb1942d849
parent65101aeca52241a05e66f23c96eb896c9412718d
udp: use a separate rx queue for packet reception

under udp flood the sk_receive_queue spinlock is heavily contended.
This patch try to reduce the contention on such lock adding a
second receive queue to the udp sockets; recvmsg() looks first
in such queue and, only if empty, tries to fetch the data from
sk_receive_queue. The latter is spliced into the newly added
queue every time the receive path has to acquire the
sk_receive_queue lock.

The accounting of forward allocated memory is still protected with
the sk_receive_queue lock, so udp_rmem_release() needs to acquire
both locks when the forward deficit is flushed.

On specific scenarios we can end up acquiring and releasing the
sk_receive_queue lock multiple times; that will be covered by
the next patch

Suggested-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Acked-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
include/linux/udp.h
include/net/udp.h
include/net/udplite.h
net/ipv4/udp.c
net/ipv6/udp.c