Like for hw_params, hw_free should not be called on codec dai for
which the current stream is invalid.

Fixes: cde79035 ("ASoC: Handle multiple codecs with split playback / capture")
Signed-off-by: Jerome Brunet <jbrunet@baylibre.com>
Signed-off-by: Mark Brown <broonie@kernel.org>