drm/amd/display: fix shift-out-of-bounds in CalculateVMAndRowBytes
authorAlex Hung <alex.hung@amd.com>
Wed, 11 Jan 2023 16:54:11 +0000 (09:54 -0700)
committerAlex Deucher <alexander.deucher@amd.com>
Thu, 2 Mar 2023 03:43:19 +0000 (22:43 -0500)
commit031f196d1b1b6d5dfcb0533b431e3ab1750e6189
tree6597cb76323e6cddb0cc2d6328febeefe428ee4f
parent01a18aa309aec12461fb5e6aecb76f8b33810658
drm/amd/display: fix shift-out-of-bounds in CalculateVMAndRowBytes

[WHY]
When PTEBufferSizeInRequests is zero, UBSAN reports the following
warning because dml_log2 returns an unexpected negative value:

  shift exponent 4294966273 is too large for 32-bit type 'int'

[HOW]

In the case PTEBufferSizeInRequests is zero, skip the dml_log2() and
assign the result directly.

Reviewed-by: Jun Lei <Jun.Lei@amd.com>
Acked-by: Qingqing Zhuo <qingqing.zhuo@amd.com>
Signed-off-by: Alex Hung <alex.hung@amd.com>
Tested-by: Daniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
drivers/gpu/drm/amd/display/dc/dml/dcn30/display_mode_vba_30.c