We could pin BOs into invisible VRAM otherwise.

v2: make logic more readable as suggested by Michel

Cc: stable@vger.kernel.org
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com> (v1)
Reviewed-by: Rex Zhu <Rex.Zhu@amd.com> (v1)
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>