MINOR: connection: use conn->{src,dst} instead of &conn->addr.{from,to}

MINOR: connection: use conn->{src,dst} instead of &conn->addr.{from,to}

This is in preparation for the switch to dynamic address allocation,
let's migrate the code using the old fields to the pointers instead.
Note that no extra check was added for now, the purpose is only to
get the code to use the pointers and still work.

In the proxy protocol message handling we make sure the addresses are
properly allocated before declaring them unset.