usb: phy: ab8500-usb: fix unbalanced clock and regulator disable warnings

To prevent clock and regulator frameworks from complaining, only disable
the host or peripheral phy if they were enabled.
Reported-by: Sakethram Bommisetti <sakethram.bommisetti@stericsson.com>
Signed-off-by: Mian Yousaf Kaukab <mian.yousaf.kaukab@stericsson.com>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Fabio Baltieri <fabio.baltieri@linaro.org>
Signed-off-by: Felipe Balbi <balbi@ti.com>

drivers/usb/phy/phy-ab8500-usb.c

@@ -888,8 +888,10 @@ static int ab8500_usb_remove(struct platform_device *pdev)
 
 	usb_remove_phy(&ab->phy);
 
-	ab8500_usb_host_phy_dis(ab);
-	ab8500_usb_peri_phy_dis(ab);
+	if (ab->mode == USB_HOST)
+		ab8500_usb_host_phy_dis(ab);
+	else if (ab->mode == USB_PERIPHERAL)
+		ab8500_usb_peri_phy_dis(ab);
 
 	platform_set_drvdata(pdev, NULL);