I am building a blockchain prototype (github.com/0xpolygon/polygon-sdk) with libp2p as the network stack. I use libp2p to establish the connection and then I run grpc on top of the stream as the communication protocol.
However, I need to do some custom handshake after the connection is established to exchange important metadata and do some validations (i.e. both clients are on the same chain). This is a pre requisite before any other grpc streams can be started.
Is there any support for this kind of behaviour or are there any best practices you would recommend to implement this?
So far this are the two options that I am considering:
After each Notifieer.Connected message do a handshake stream. If the handshake fails remove the peer from the host. However, at any time during this period the peers could connect to any of the other streams. Besides, the auth lifecycle would be all over the place. Ideally, I would like something that succeeds or fails in the Connect call.
Use a custom security transport that embeds Noise as a pre-process and after that performs the handshake with the raw net.Conn interface.
There are several solutions I can think of, depending on the security tradeoff you want:
If you don’t mind libp2p protocols running before the connection is authenticated (e.g. libp2p identify, pubsub, DHT, etc.), you can implement a custom protocol that subscribes to PeerConnectednessChanged events on the eventbus, initiates the challenge by opening a stream, and kills the connection if the challenge fails.
All other application-specific protocols would wait until this protocol has authenticated the peer.
You would need to implement a “session manager” component, that the authentication protocol reports to, and that other protocols gate on to know when they are authorised to service a peer.
You could send custom events on the eventbus to signal the state changes.
If you don’t mind the handshake running and the peer ID of the responder being revealed to the other party, prior to your app-specific authentication, you could use a ConnectionGater and implement a hook for InterceptUpgraded.
The problem is that you get a network.Conn and not a mux.MuxedConn, so I’m not sure how the other peer would accept the stream.
Note that these are raw streams, not managed by the libp2p host. I haven’t seen anybody using libp2p in this manner, so not sure if it would work.
I followed the first approach and everything seems to be working fine. However, I have another issue now.
I have a custom “dial manager” to limit the number of connected peers. It dials new peers from a queue when there are less than MaxPeers. The dial queue is filled with values from PeersAdded callback from the dht module. The aforementioned handshake protocol is called every time a new peer is connected.
However, it seems like the Dht module also makes Connect calls to other nodes (I would guess this connections are done to exchange kademlia peers). These calls are also piped into the handshake protocol. Thus, peers not dialed with the dial manager/queue are included as valid peers.
Are you referring to https://github.com/libp2p/go-libp2p-kbucket/blob/b90e3fed3255e131058ac337a19beb2ad85da43f/table.go#L52 or something else? This function is called when peers are added to the routing table, however as part of standard operation we may connect to peers or receive connections from peers who are not suitable for membership in our routing table (e.g. the peer is behind a relay, the bucket they would go into is full, they are DHT clients and not servers, etc.)