USB is always little endian, but this driver
could run on non little endian cpus. Let's
be carefull with that.
Signed-off-by: Felipe Balbi <balbi@ti.com>