• Hans Verkuil's avatar
    [media] v4l2_plane_pix_format: use __u32 bytesperline instead of __u16 · cc7d2dfb
    Hans Verkuil authored
    While running v4l2-compliance tests on vivid I suddenly got errors due to
    a call to vmalloc_user with size 0 from vb2.
    
    Digging deeper into the cause I discovered that this was due to the fact that
    struct v4l2_plane_pix_format defines bytesperline as a __u16 instead of a __u32.
    
    The test I was running selected a format of 4 * 4096 by 4 * 2048 with a 32
    bit pixelformat.
    
    So bytesperline was 4 * 4 * 4096 = 65536, which becomes 0 in a __u16. And
    bytesperline * height is suddenly 0 as well. While the vivid driver may be
    a virtual driver, it is to be expected that this limit will be hit for real
    hardware as well in the near future: 8k deep-color video will already reach
    it.
    
    The solution is to change the type to __u32. The only drivers besides vivid
    that use the multiplanar API are little-endian ARM and SH platforms (exynos,
    ti-vpe, vsp1), so this is safe.
    Signed-off-by: default avatarHans Verkuil <hans.verkuil@cisco.com>
    Signed-off-by: default avatarMauro Carvalho Chehab <mchehab@osg.samsung.com>
    cc7d2dfb
pixfmt.xml 75 KB