IB/mlx4: Follow mirror sequence of device add during device removal