It's unnecessary to check the irqs_queued value as
it always needs to queue one if irq isn't enabled.
Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>