When using the irqchip helper inside the gpiolib, make sure
the IRQs are unmapped/disposed before the irqdomain is removed
as part of removing the gpiochip.
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>