The channel memory base is channel-specific, add it to the channel
structure in preparation for support of multiple channels per device.
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>