media: v4l2-dv-timings: prevent possible overflow in v4l2_detect_gtf()

commit 3edd1fc48d2c045e8259561797c89fe78f01717e upstream.

In v4l2_detect_gtf(), it seems safer to cast the 32-bit image_width
variable to the 64-bit type u64 before multiplying to avoid
a possible overflow. The resulting object code even seems to
look better, at least on x86_64.

Found by Linux Verification Center (linuxtesting.org) with Svace.

[Sergey: rewrote the patch subject/descripition]

Fixes: c9bc9f50753d ("[media] v4l2-dv-timings: fix overflow in gtf timings calculation")
Cc: stable@vger.kernel.org
Signed-off-by: Karina Yankevich <k.yankevich@omp.ru>
Signed-off-by: Sergey Shtylyov <s.shtylyov@omp.ru>
Signed-off-by: Hans Verkuil <hverkuil@xs4all.nl>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

diff --git a/drivers/media/v4l2-core/v4l2-dv-timings.c b/drivers/media/v4l2-core/v4l2-dv-timings.c
index 2cf5dcee0ce800085c359e69aa85ee6928785609..4d05873892c168031824e7fe08d75ae10f39af64 100644 (file)
--- a/drivers/media/v4l2-core/v4l2-dv-timings.c
+++ b/drivers/media/v4l2-core/v4l2-dv-timings.c
@@ -764,7 +764,7 @@ bool v4l2_detect_gtf(unsigned int frame_height,
                u64 num;
                u32 den;
 
-               num = ((image_width * GTF_D_C_PRIME * (u64)hfreq) -
+               num = (((u64)image_width * GTF_D_C_PRIME * hfreq) -
                      ((u64)image_width * GTF_D_M_PRIME * 1000));
                den = (hfreq * (100 - GTF_D_C_PRIME) + GTF_D_M_PRIME * 1000) *
                      (2 * GTF_CELL_GRAN);
@@ -774,7 +774,7 @@ bool v4l2_detect_gtf(unsigned int frame_height,
                u64 num;
                u32 den;
 
-               num = ((image_width * GTF_S_C_PRIME * (u64)hfreq) -
+               num = (((u64)image_width * GTF_S_C_PRIME * hfreq) -
                      ((u64)image_width * GTF_S_M_PRIME * 1000));
                den = (hfreq * (100 - GTF_S_C_PRIME) + GTF_S_M_PRIME * 1000) *
                      (2 * GTF_CELL_GRAN);