Getting error "connected to wrong peer"

I have written a test that verifies that peer discovery via Kad DHT is working. The test works in the local and docker env but fails in Travis CI or Github Actions? Any idea what could be wrong here?