How to limit gossip pubsub peer discovery

Users and Developers go
1

Hi there! New to libp2p, liking what I’ve been able to get done so far.

I’m trying to build a peer-to-peer application using the Gossip pub/sub implementation. While the basics seems to be working, the gossip implementation seems to be connecting to a lot of unrelated nodes before anything has been published or subscribed, and I’m trying to understand why.

After getting the Host set up (details at end of this post), I create a local DHT client:

	kademliaDHT, err := dht.New(*ctx, host, opts.Mode(dht.ModeAuto))
	if err != nil {
		panic(err)
	}
	if err = kademliaDHT.Bootstrap(*ctx); err != nil {
		panic(err)
	}

Then my node advertises itself and looks for peers:

	routingDiscovery := discovery.NewRoutingDiscovery(kademliaDHT)
	discovery.Advertise(*ctx, routingDiscovery, "my-application")

	peerChan, err := routingDiscovery.FindPeers(*ctx, "my-application")
	if err != nil {
		panic(err)
	}

	for peer := range peerChan {
		if peer.ID == host.ID() {
			continue
		}
		logger.Info("Found peer:", peer)
	}

So far that works as expected; I see the other peers I expect (1-2 nodes depending on what I’m running). Next, I set up the pubsub layer this way:

	ps, err := pubsub.NewGossipSub(*ctx, host)

My node soon shows a lot of logs like this, with far more connections than known peers:

2020-06-12T19:03:40.828-0400    WARN     pubsub  go-libp2p-pubsub@v0.3.2/comm.go:74      opening new stream to peer: protocol not supportedQmUiEquVAe53THnGLJCGjb1Z4i4j8SqNNHSKYaHzPH4k1p
2020-06-12T19:03:41.175-0400    WARN     pubsub  go-libp2p-pubsub@v0.3.2/comm.go:74      opening new stream to peer: protocol not supportedQmNixNuseNf5LNbVkWpoen1kHi6EPGvDXtPij8tXUwSXgM
2020-06-12T19:03:41.769-0400    WARN     pubsub  go-libp2p-pubsub@v0.3.2/comm.go:74      opening new stream to peer: protocol not supportedQmXKVJ4Y3nXg5ZcnvH8kDcrRwX68ZFdayL29WZSHR8ap8e
2020-06-12T19:03:43.187-0400    WARN     pubsub  go-libp2p-pubsub@v0.3.2/comm.go:74      opening new stream to peer: protocol not supported12D3KooWPSw2pWRaRBE67TSdDzMeLTVrwXunjeBjCm9BiSefomyt

It seems like the gossip implementation is connecting to lots of other hosts, I guess discovered through the dht. For this application, which is bandwidth sensitive, I’d like to avoid connections to unrelated nodes (especially since many of them seem to be failing anyway). I’d love to understand this part of the protocol stack a bit better.

Question: How do I limit Gossip pubsub to interrogating/using only to those peers that have advertised themselves as part of my application?

thank you!

Appendix

Here’s how I set up the host on each peer:

	host, err := libp2p.New(
		context.Background(),
		libp2p.ListenAddrs(listenAddr),
		libp2p.Identity(*prvKey),
		libp2p.Security(libp2ptls.ID, libp2ptls.New),
		libp2p.DefaultTransports,
		libp2p.ConnectionManager(connmgr.NewConnManager(
			10,          // Lowwater
			50,          // HighWater,
			time.Minute, // GracePeriod
		)),
		libp2p.NATPortMap(),
		libp2p.Routing(func(h host.Host) (routing.PeerRouting, error) {
			idht, err := dht.New(*ctx, h)
			return idht, err
		}),
		libp2p.EnableAutoRelay(),
	)
2

You should be able to define a private network dor your app, so your nodes will not talk to any node outside of it. I think regular nodes won’t see you so they shouldn’t try to contact you. Use it only if you don’t plan to fetch data from the main network, of course (or find a mechanism to bring the needed data into your private network somehow). You will need to use one of your own node as a bootstrap node of course, as private network nodes can’t talk to the main network nodes, including default hardcoded bootstrap nodes.

3

Hey thanks for the tip!

Hmm, that seems like it could work to keep the network secure and private. Is that the best/only solution here because GossipSub is bound to the host’s PeerStore, which is going to pick up lots of other potential peers because of the bootstrap dht connection…?

What I’m trying to get my head around is why/where in the abstractions GossipSub is picking up all of these other things its connecting to. It seems like it would be a common enough use case – “don’t bother gossiping with nodes that know nothing about our app” – so I’m trying to figure out where my mental model (or code!) is busted…

4

No, by running a private network for your app, they will run a parallel, independant network. You will bootstrap your app nodes from one (or several) of your own nodes, and they will connect to each other, build a DHT containing only your app nodes, and never know there is another libp2p network in existence (the bigger, main, default network).

I think this doc articles does a good job at explaining how PubSub works.

In a lot of case where libp2p is running on a desktop computer or even a server, you may run sevral apps on top of libp2p. When you do, you want to run only one libp2p node that will serve several apps to save ressources. The libp2p node represents a user, not an instance of an app. In that case, you can subscibe to different PubSub topics relevant to the apps, for example. In this design, a node tends to talk mainly with peers from the same apps (because they tend to keep connections with nodes following the same topics), but is aware of the broader network.
In your case, you want to save resources and run only one app, so you can isolate your nodes from other libp2p nodes and make them run in their own independent network.

5

Sorry, was still speaking about the config I originally shared: Using the default bootstrap nodes and “public” dht as in my snippet, Gossip PubSub is discovering lots of peers. I’m not providing any pubsub.WithDiscovery(..) option, so it seems like Gossip is discovering peers from the host peerstore.

Thank you, yes, gave that a few reads - a very helpful doc. I’m trying to tie it to the go implementation. In this section:

Potential methods for discovering peers include:

  • Distributed hash tables
  • Local network broadcasts
  • Exchanging peer lists with existing peers
  • Centralized trackers or rendezvous points
  • Lists of bootstrap peers

In the libp2p gossip implementation, would these alternative gossip peer discovery mechanisms be achieved by provided WithDiscovery and my own implementation?

Thank you! That’s a very helpful clarification, and helps explain some of the interfaces and idioms I’m seeing; I’ve been treating lib p2p as “embedded” for a single app.

Thanks again for taking the time to point me in the right direction! Much appreciated.

1 Like
6

Gossipsub is not discovering peers from the peerstore; what you observe is the DHT opening connections as it is bootstrapping and querying.

7

Hm. I guess I am thrown off by these log lines arising from go-libp2p-pubsub.

It seems to be part of the handleNewPeer internal callback. This seems to be triggered because NewPubSub registers itself as a Notifiee with the host network.

Specifically:

h.Network().Notify((*PubSubNotif)(ps))

… seems to rigidly bind the PubSub stack to learn about any and all peers that become registered with the host. Which I understand the use of the DHT does.

Would being able to inject a different Notifee as an option be out of the question here, as an improvement to NewPubSub?

In the broader design of libp2p, is there any reason my gossip instance should not be able to bring its own peer connected (and disconnected, for that matter) handlers…?