Initially just use the helper and thus krefs when we set and clear the
port values.
Signed-off-by: Alan Cox <alan@linux.intel.com>