eap: tweak conversation tracking to avoid breaking pinfo horribly

Looks like conversation_set_conv_addr_port_endpoints() from 66b441f3 is designed with this use case in mind.

This should resolve issue #18622 (closed)

(cherry picked from commit a8b16d74)