When we add non-deflink pointers, we need to remove the
link[0] pointer to deflink in case link[0] is not valid
afterwards. Also, we need to add that back when there
are no more valid links. Reorg the code to fix that.
Signed-off-by: Johannes Berg <johannes.berg@intel.com>