Several drivers have the same (and incorrect) code in their
_remove() handler.

Coalesce this into a shared function.
Signed-off-by: Ian Molton <ian@mnementh.co.uk>
Reviewed-by: Sebastian Reichel <sebastian.reichel@collabora.co.uk>
Signed-off-by: Marcel Holtmann <marcel@holtmann.org>