LibP2P on the Polkadot Network

I’m looking to build a DHT crawler for Polkadot.

A few month ago, I chatted with one of the devs, Pierre, and he mentioned that I might have to set the user-agent to a string that contains “substrate”. I’ve tried to connect to the network but i’m not getting a response. I’m using a boot node gotten from the Polkadot Source: https://github.com/paritytech/polkadot/search?q=Qm&unscoped_q=Qm

Has anyone got a connection working to the Polkadot network through LibP2P? Has anything changed, maybe some new network ID?

Thanks

I don’t know why it doesn’t show up in your GitHub search (maybe because the file is too large), but Polkadot’s bootnodes are listed here: https://raw.githubusercontent.com/paritytech/polkadot/master/service/res/alexander.json
(and for future reference, chain specs will be here)

Whatever happens, the connection initiation consists of secio and mplex/yamux, and that should be totally compatible with go-libp2p and js-libp2p.

Afterwards, however, you node will get disconnected if:

  • It doesn’t support ping or identify.
  • The response from identify doesn’t contain the string “substrate” (to prevent IPFS nodes from connecting).

Thanks Pierre,

Do you know of any existing code that uses LibP2P (either NodeJs or GoLang) that is connecting to Substrate? I’m able to connect to IPFS but still unable to connect to the substrate network. It looks like that some of the libraries like (NodeJs multiaddr) might be built for IPFS only.

I was able to ping the IPFS bootnodes but when I just try to ping (using the standard setup provided by libp2p tutorial) the Substrate bootnodes, I get either one of two errors:

When pinging /ip4/40.117.153.33/tcp/30363/p2p/QmPiGU1jwL9UDw2FMyMQFr9FdpF9hURKxkfy6PWw6aLsur
The response is:

pinging remote peer at  /ip4/40.117.153.33/tcp/30363/ipfs/QmPiGU1jwL9UDw2FMyMQFr9FdpF9hURKxkfy6PWw6aLsur
error pinging:  { Error: No available transports to dial peer QmPiGU1jwL9UDw2FMyMQFr9FdpF9hURKxkfy6PWw6aLsur!

When Pinging: /ip4/104.211.48.247/tcp/30363/ipfs/QmYT3p4qGj1jwb7hDx1A6cDzAPtaHp3VR34vmw5BsXXB8D
Response:

pinging remote peer at  /ip4/104.211.48.247/tcp/30363/ipfs/QmYT3p4qGj1jwb7hDx1A6cDzAPtaHp3VR34vmw5BsXXB8D
error pinging:  Error: dialed to the wrong peer, Ids do not match

My config:

    transport: [
      TCP,
      WebSockets
    ],
    connEncryption: [
      SECIO
    ],
    streamMuxer: [
      Multiplex
    ]
  }

Looks like the multiaddr library is converting all p2p protocol into ipfs not sure if that’s intentional and if that’ll work for the substrate network.

If you can point me to some reference code (go/node) for connecting to Substrate using libP2P that would be very helpful.

Thanks

Commented over at https://github.com/multiformats/js-multiaddr/issues/93#issuecomment-514549901. Multiaddr shouldn’t be an issue. It looks like the BootNodes from the config aren’t available as TCP is getting an ECONNREFUSED error.

  libp2p:tcp:dial Connecting to 30363 40.117.153.33 +21s
  libp2p:switch:dialer:queue:error TCP:work { Error: connect ECONNREFUSED 40.117.153.33:30363
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1106:14)
  errno: 'ECONNREFUSED',
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: '40.117.153.33',
  port: 30363 } +11s

It looks like the Alexander bootnodes are now down, indeed. We’re preparing to launch Kusama soon, so the devops team has probably stopped the old bootnodes.

In the meanwhile you should be able to try to connect to Flaming Fir, the Substrate testnet: https://raw.githubusercontent.com/paritytech/substrate/8c1ed555bfb2bcbe5abe4e901c3a6a1796b273ca/node/cli/res/flaming-fir.json
However, it is not unlikely that we kill it soon as well!

Thanks Pierre,

I’m trying the new bootnodes that you mentioned and I’m getting a lot of :

pinging remote peer at  /ip4/35.246.224.91/tcp/30334/ws/ipfs/QmSk5HQbn6LhUwDiNMseVUjuRYhEtYj4aUZ6WfWoGURpdV
error pinging:  Error: dialed to the wrong peer, Ids do not match

Jacob mentioned that this is most likely: node has connected but SECIO failed due to the actual ID is not matching the ID that was passed in. Any clue on why this would be the case?

Thanks

Oh, I suspect that what happens is that the public key’s format is secp256k1, which the Go and Js implementations don’t support.

We’re switching to ed25519, which Go and Js might support?
If you spawn your own Substrate (or Polkadot) node, its network key will be ed25519 by default.

Hey Pierre,

Just to follow up on this, do you know roughly when the GoLang or JS implementations will be able to support this? We are working on one of the grant projects and we have finished 2/3 milestones and we need to get the DHT crawler working for the final step to complete the project

Thanks.

I am not involved in the Go or JS implementations.
As for Polkadot, we should be deploying Kusama soon, which will use ED25519 keys.