If device is configured to work only in HOST or DEVICE mode, there is
no point in initializing both subdrivers. This patch also fixes
resource leakage if host subdriver fails to initialize.
Acked-by: John Youn <johnyoun@synopsys.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Felipe Balbi <balbi@ti.com>