another followup for 4c2c057d.

there are six possible cases:
--port can be set or not.
--address can be set, not set, or set but without a port number

The correct behavior is:
1 both --port and --address have a port number
  - use it if it's the same, otherwise an error
2 only --port has the number (--address isn't set)
  - use the value from --port
3 only --port has the number (--address is set, but has no port)
  - use the value from --port
4 --port is unset, --address has the port number
  - use the value from --address
5 --port is unset, --address has no port number
  - use the value from --address, that is, port is empty string
6 --port is unset, --address is unset
  - port is unset (an error somewhere later)

case 5 wasn't handled correctly