media: atomisp: Avoid picking too big sensor resolution

atomisp_try_fmt() is limiting the width of the requested resolution to 1920
before calling the sensor's try_fmt() method. But it is not limiting
the height. In case of the old mode-list based t4ka3 driver which has
a mode list of:

736x496
896x736
1936x1096
3280x2464

This results in 3280x2464 being selected when try_fmt is called
with a requested resolution of 3280x2464, which is not supported because
its width > 1920 .

Fix this by also limiting the height when in preview mode.

Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Link: https://lore.kernel.org/r/20241211173516.350779-1-hdegoede@redhat.com
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

diff --git a/drivers/staging/media/atomisp/pci/atomisp_cmd.c b/drivers/staging/media/atomisp/pci/atomisp_cmd.c
index 8feb627ddcca66571a24383521855f1b024d6486..ea2099d2897f907cddee6d69bb9a1fc5f5e47285 100644 (file)
--- a/drivers/staging/media/atomisp/pci/atomisp_cmd.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_cmd.c
@@ -3784,9 +3784,14 @@ int atomisp_try_fmt(struct atomisp_device *isp, struct v4l2_pix_format *f,
                        return -EINVAL;
        }
 
-       /* The preview pipeline does not support width > 1920 */
-       if (asd->run_mode->val == ATOMISP_RUN_MODE_PREVIEW)
-               f->width = min_t(u32, f->width, 1920);
+       /*
+        * The preview pipeline does not support width > 1920. Also limit height
+        * to avoid sensor drivers still picking a too wide resolution.
+        */
+       if (asd->run_mode->val == ATOMISP_RUN_MODE_PREVIEW) {
+               f->width = min(f->width, 1920U);
+               f->height = min(f->height, 1440U);
+       }
 
        /*
         * atomisp_set_fmt() will set the sensor resolution to the requested