When the DRM device is torn down and the connector is removed, make sure
to detach the panel to make sure there are no dangling pointers.
Signed-off-by: Thierry Reding <treding@nvidia.com>