Running libp2p through a load balancer

I am new to libp2p I am trying to get libp2p hosts to talk to each other via a single load balancer or a reverse proxy , e.g. nginx. I have tried using TCP and Websockets. My question is while it works, if I dump the connected peers details, both hosts see the other host as the load balancers IP address, is this expected behavior?