This will allow us to only allocate memory when
we actually need.
Signed-off-by: Felipe Balbi <balbi@ti.com>