Commit 7851496a authored by Sinclair Yeh's avatar Sinclair Yeh

drm/vmwgfx: Fix order of operation

mode->hdisplay * (var->bits_per_pixel + 7) gets evaluated before
the division, potentially making the pitch larger than it should
be.

Since the original intention is to do a div-round-up, just use
the macro instead.
Signed-off-by: default avatarSinclair Yeh <syeh@vmware.com>
Reviewed-by: default avatarThomas Hellstrom <thellstrom@vmware.com>
parent e02e5884
...@@ -573,9 +573,9 @@ static int vmw_fb_set_par(struct fb_info *info) ...@@ -573,9 +573,9 @@ static int vmw_fb_set_par(struct fb_info *info)
mode = old_mode; mode = old_mode;
old_mode = NULL; old_mode = NULL;
} else if (!vmw_kms_validate_mode_vram(vmw_priv, } else if (!vmw_kms_validate_mode_vram(vmw_priv,
mode->hdisplay * mode->hdisplay *
(var->bits_per_pixel + 7) / 8, DIV_ROUND_UP(var->bits_per_pixel, 8),
mode->vdisplay)) { mode->vdisplay)) {
drm_mode_destroy(vmw_priv->dev, mode); drm_mode_destroy(vmw_priv->dev, mode);
return -EINVAL; return -EINVAL;
} }
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment