Otherwise, a xfrm policy with sport/dport being set cannot be matched.
Signed-off-by: Martynas Pumputis <martynas@weave.works>
Signed-off-by: David S. Miller <davem@davemloft.net>