drm/amd/display: Prevent cursor hotspot overflow for RV overlay planes

[ Upstream commit 6752bea8 ]

[Why]
The actual position for the cursor on the screen is essentially:

x_out = x - x_plane - x_hotspot
y_out = y - y_plane - y_hotspot

The register values for cursor position and cursor hotspot need to be
greater than zero when programmed, but we also need to subtract off
the plane position to display the cursor at the correct position.

Since we don't want x or y to be less than zero, we add the plane
position as a positive value to x_hotspot or y_hotspot. However, what
this doesn't take into account is that the hotspot registers are limited
by the maximum cursor size.

On DCN10 the cursor hotspot regitsers are masked to 0xFF, so they have
a maximum value of 0-255. Values greater this will wrap, causing the
cursor to display in the wrong position.

In practice this means that for sufficiently large plane positions, the
cursor will be drawn twice on the screen, and can cause screen flashes
or p-state WARNS depending on what the wrapped value is.

So we need a way to remove the value from x_plane and y_plane without
exceeding the maximum cursor size.

[How]
Subtract as much as x_plane/y_plane as possible from x and y and place
the remainder in the cursor hotspot register.

The value for x_hotspot and y_hotspot can still wrap around but it
won't happen in a case where the cursor is actually enabled.

The cursor plane needs to intersect at least one pixel of the plane's
rectangle to be enabled, so the cursor position + hotspot provided by
userspace must always be strictly less than the maximum cursor size for
the cursor to actually be enabled.
Signed-off-by: Nicholas Kazlauskas <nicholas.kazlauskas@amd.com>
Reviewed-by: Sun peng Li <Sunpeng.Li@amd.com>
Acked-by: Bhawanpreet Lakha <Bhawanpreet.Lakha@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>

drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c

@@ -2692,9 +2692,15 @@ static void dcn10_set_cursor_position(struct pipe_ctx *pipe_ctx)
 		.rotation = pipe_ctx->plane_state->rotation,
 		.mirror = pipe_ctx->plane_state->horizontal_mirror
 	};
-
-	pos_cpy.x_hotspot += pipe_ctx->plane_state->dst_rect.x;
-	pos_cpy.y_hotspot += pipe_ctx->plane_state->dst_rect.y;
+	uint32_t x_plane = pipe_ctx->plane_state->dst_rect.x;
+	uint32_t y_plane = pipe_ctx->plane_state->dst_rect.y;
+	uint32_t x_offset = min(x_plane, pos_cpy.x);
+	uint32_t y_offset = min(y_plane, pos_cpy.y);
+
+	pos_cpy.x -= x_offset;
+	pos_cpy.y -= y_offset;
+	pos_cpy.x_hotspot += (x_plane - x_offset);
+	pos_cpy.y_hotspot += (y_plane - y_offset);
 
 	if (pipe_ctx->plane_state->address.type
 			== PLN_ADDR_TYPE_VIDEO_PROGRESSIVE)