Minimal example for AutoNAT+AutoRelay with PNET

Hello!

I’m slowly losing my mind and getting more gray hair…

What is the bl*dy minimal setup for a working private net with AutoNAT and AutoRelay?

Scenario (all crafted with Linux namespaces and connectivity is verified):

  • 8 public nodes (6 “seeds” willing to be AutoRelay and 2 “regulars” not willing to be AR) in 8 different public networks
  • 4 private nodes (4 “leafs”) behind 4 different NATs.

No 2 nodes are in same network. Worst-case-scenario so to speak.

Global settings

routerDHT = func(h host.Host) (routing.PeerRouting, error) {
     return kaddht.New(
                  context.Background(), h,
                  kaddht.BootstrapPeers(SEED_NODES),
            )
}
opts := []libp2p.Option{
			libp2p.UserAgent(MY_NAME),
			libp2p.Identity(NODE_PRIV_KEY),
			libp2p.PrivateNetwork(NET_KEY),
			libp2p.Ping(true),
			libp2p.Transport(tcp.NewTCPTransport),
			libp2p.Security(noise.ID, noise.New),
			libp2p.DefaultMuxers,
			libp2p.DefaultPeerstore,
			libp2p.Routing(routerDHT),
			libp2p.ListenAddrStrings(fmt.Sprintf("/ip4/0.0.0.0/tcp/%d",NODE_PORT))
  ...

Additional seed settings

...
			libp2p.EnableRelayService(),
			libp2p.EnableNATService(),
			libp2p.ForceReachabilityPublic(),
			libp2p.EnableRelay(),
}

Additional “regular” settings

...
			libp2p.ForceReachabilityPublic(),
			libp2p.EnableRelay(),
}

Additional “leaf” settings

...
			libp2p.EnableRelay(),
		     	libp2p.EnableAutoRelay(
				autorelay.WithMinCandidates(3),
				autorelay.WithMaxCandidates(10),
				autorelay.WithNumRelays(2),
				autorelay.WithBootDelay(10*time.Second),
			),
		)
}

Roll the network

p2pHost, err := libp2p.NewWithoutDefaults(opts...)

What does work

  • “seeds” and “regulars” do mesh correctly.
  • “leafs” do connect to seeds correctly
  • DHT seems to be filled with correct data (extensive debug dumps)
  • Second overlay DHT (not mentioned above) works correctly (it maps user readable names to node IDS)

What doesn’t work

  • Any attempt to set “leaf” private (like setting libp2p.ForceReachabilityPrivate()) ends in crash:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xc0a071]

goroutine 81 [running]:
github.com/libp2p/go-libp2p/p2p/host/autorelay.(*relayFinder).findNodes(0xc000122340, {0x132b830?, 0xc000a4a140})
        /home/pawel/Private/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/autorelay/relay_finder.go:209 +0x51
github.com/libp2p/go-libp2p/p2p/host/autorelay.(*relayFinder).background.func2()
        /home/pawel/Private/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/autorelay/relay_finder.go:112 +0x66
created by github.com/libp2p/go-libp2p/p2p/host/autorelay.(*relayFinder).background
        /home/pawel/Private/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/autorelay/relay_finder.go:110 +0x192
  • I am unable to make “leafs” perform NAT discovery to notice they are behind NAT
  • I am unable to make “leafs” perform advertisement (to build relay circuit)
  • As a result I am unable to make “leafs” register in relay service
  • Ultimately I am not able to connect between “leafs”.

I am totally out of ideas… I tried every conceivable combination of services running in any of the 3 classes of nodes. The two possibly relevant articles (“Content Routing” and “Peer Routing”) of the official site are “This article is coming soon!”.

Please help!


P.S. As I’m using PNET, is it advisable to get rid of libp2p.Security?

@PKraszewski
I’m having the same problem. Did you manage to get it to work?
The fact that there is virtually no documentation on this topic (and no examples) certainly does not help. It would appear that even a minimal setup fails with the same runtime error.

I’ll share this with the team and see if someone can help out @PKraszewski @Raffy

The fact that there is virtually no documentation on this topic (and no examples) certainly does not help

Totally understand the frustration and want to say that I hear you. Good news is that we’re in the process of adding missing documentation, examples, and guides right now. Right now there are issues created to add articles on AutoNat and AutoRelay you can follow: Enhance AutoNAT doc · Issue #171 · libp2p/docs · GitHub
Add AutoRelay to Circuit Relay doc · Issue #172 · libp2p/docs · GitHub

We’re hoping to get these roll these out. Our newly formed docs team has been really hard at work.

Can you contribute the demo’s github repository URL?

I managed a workaround. Unfortunately I can’t share it as it is, I must do a cleanroom implementation I’d happily share.

Basically I entirely dumped PNET and partially rewritten QUIC module to work with mutual requirement of valid certificates (checked against CA and CRL). Additionally, p2p private key is pulled directly from TLS private key, so essentially your EDDSA PKCS12 file is your P2P identity.

Currently (closed project with no intended interop with other networks) I just “stole” QUIC protocol ID, but after sharing it, it would be advisable to reserve a new ID (“PROTECTED_QUIC”/“PRIVATE QUIC” or something), as my “standard” is incompatible with the regular p2p QUIC.