vfio/mlx5: fix possible overflow in tracking max message size

MLX cap pg_track_log_max_msg_size consists of 5 bits, value of which is
used as power of 2 for max_msg_size. This can lead to multiplication
overflow between max_msg_size (u32) and integer constant, and afterwards
incorrect value is being written to rq_size.

Fix this issue by extending integer constant to u64 type.

Found by Linux Verification Center (linuxtesting.org) with SVACE.

Suggested-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Artem Sadovnikov <a.sadovnikov@ispras.ru>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Link: https://lore.kernel.org/r/20250701144017.2410-2-a.sadovnikov@ispras.ru
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>

diff --git a/drivers/vfio/pci/mlx5/cmd.c b/drivers/vfio/pci/mlx5/cmd.c
index 5b919a0b25247cdc2dae09cfcfa92bf9bd0aef2b..a92b095b90f6a9864b78505a24d16e08ad792262 100644 (file)
--- a/drivers/vfio/pci/mlx5/cmd.c
+++ b/drivers/vfio/pci/mlx5/cmd.c
@@ -1523,8 +1523,8 @@ int mlx5vf_start_page_tracker(struct vfio_device *vdev,
        log_max_msg_size = MLX5_CAP_ADV_VIRTUALIZATION(mdev, pg_track_log_max_msg_size);
        max_msg_size = (1ULL << log_max_msg_size);
        /* The RQ must hold at least 4 WQEs/messages for successful QP creation */
-       if (rq_size < 4 * max_msg_size)
-               rq_size = 4 * max_msg_size;
+       if (rq_size < 4ULL * max_msg_size)
+               rq_size = 4ULL * max_msg_size;
 
        memset(tracker, 0, sizeof(*tracker));
        tracker->uar = mlx5_get_uars_page(mdev);