There are external pulls on these lines and enabling the
internal pulls can cause issue. This is because the internal
pulls are parallel with the external pulls. So let's clear
the internal I2C pulls.
Signed-off-by: Tony Lindgren <tony@atomide.com>