Otherwise the PWM stops working before the PWM core and its consumers
are aware the device is going away.
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com>