Js-libp2p "peer and content routing" / kad-dht example

Hi. I am learning to use js-libp2p. The new docs are a big improvement but I am a little confused because many things are automatic and I’m trying to figure out “how automatic” they are.

I have a web app that integrates the “libp2p in the browser” example and uses libp2p-bootstrap to get a list of peers. I am trying to leverage that up to my web app being able to search a Kademlia DHT for providers for hashes. It would be acceptable or even desirable if the DHT I am searching consists exclusively of instances of my app. I have a basic understanding of Kademlia itself (I have previously implemented it in C++).

I am looking at the peer-and-content-routing example in 2.9.2:

Here is what is confusing me.

  • I can run this example in node.js in two Terminal windows simultaneously. The two nodes find each other. How are they finding each other? Is the hardcoded 0.0.0.0 address simply finding “all js-libp2p instances on the local machine”?

  • How would I attempt, in a web app, to connect to a Kademlia DHT on the general internet? Can libp2p-bootstrap be used for this? Is it sufficient to simply load libp2p-bootstrap and libp2p-kad-dht at once or are there extra steps?

  • Say I have libp2p-bootstrap or libp2p-kad-dht working in a web app. Say my configured "transport"s are “libp2p-websockets” and “libp2p-webrtc-star”. Say I open two browsers on separate computers A and B running this web app. Say one or both of A and B are behind NAT such as a consumer router. Will A be able to open a connection to B? Are there additional steps I must do to make A<->B connections possible? Also, are A<->B connections exclusively possible using the libp2p-webrtc-star transport (the page for libp2p-webrtc-star implies some STUN/TURN-like things are happening)? If so, is there any reason in the web app to include the libp2p-websockets transport at all?

  • Why does this example create three nodes? Is it so that the example network consisting of two node.js processes will have six nodes and therefore a nontrivial search space? Or if you were connecting to a global/internet DHT, would you tend to create multiple nodes so that you have multiple probes into the Kademlia hash space?

Hi @mcc

In here, you mean search only on the peers you are connected to via bootstrap?

The listen: ['/ip4/0.0.0.0/tcp/0'] is the addresses that the configured transport (TCP) should listen on. 0.0.0.0 means all IPv4 addresses on the local machine. For example, if a host has two IP addresses, 192.168.1.1 and 10.0.0.1, and a server running on the host listens on 0.0.0.0, it will be reachable at both IPs. The 0 port means to choose a random available port. If you change this example to listen: ['/ip4/127.0.0.1/tcp/0']you can see that it will work, but the found addresses will only be that address.
Whit the above in mind, all the 3 spinned nodes will be listening for connections on the local machine addresses. Then in the example, we connect A to B and B to C. This way, B knows about both A and C, but the others only know about B. Then, A will do a DHT query to find Peer C addresses. It will query the peers it is connected to running the DHT protocol (according to the DHT query algorithm). So, A asks B (only known peer) if it knows C. As B knows C, it will just return its addresses to A. If it didn’t know, it would return the closest peers it knew to the requested peer.

So long story short, the bootstrap nodes are the entry point in the network as a node needs to know peers, in order to find other peers easily. In this case, the bootstrap nodes should be running the DHT protocol and should know a lot of peers. Once the bootstrap nodes are connected and the DHT protocol kicks in, it will use its built in discovery protocol (random walk) to find peers once in a while. This is as simple as generating a random peer ID and try to find it in the network. The goal is not even to find the computed peer ID, but to go through the process of asking known nodes if they node that Peer. As a consequence, known nodes will answer with the closest peers they know and you will get to know new peers with this process.
Please note that there are known problems and the js DHT in the browser is not stable at all in the moment. Also, go DHT had significant improvements over the last year and we will be working on porting these changes to js and improve the general state of the DHT across different environments soon. If you are testing it, you should use its client mode and rely on go nodes to make DHT queries for now.

In webrtc-star, if a direct connect fails, particularly due to NAT traversal or firewalls, WebRTC ICE uses a relay TURN server. In other words, ICE will first use STUN with UDP to directly connect peers and, if that fails, will fall back to a TURN relay server.
The main reason to support multiple transport protocols is to be able to speak with more peers. It certainly depends on what the use case. For example, go does not support webrtc at the moment. As a consequence, for a browser peer to talk with a go node, it must use websockets.

I think I answered to this above. The reason is to create a simple topology where we can rely on DHT algorithms. Any DHT node needs to know any other node so that it can query it.


FYI, another solution we highly recommend for peer/content routing is delegated routing. You can see it here. It will rely on more capable nodes in the network to do DHT queries on your behalf. js-ipfs currently uses this approach with some public delegate nodes. This is the best solution for content/peer routing while we do not work on the DHT improvements mentioned above.

Thank you for the explanations!

So long story short, the bootstrap nodes are the entry point in the network as a node needs to know peers, in order to find other peers easily. In this case, the bootstrap nodes should be running the DHT protocol and should know a lot of peers.

Okay. So to be clear— your suggestion is I should create dedicated bootstrap nodes for my application, and that way all the peers my bootstrap node[s] suggest will be already participating in my application? That makes sense.

Right now I’m using the bootstrap nodes from https://github.com/libp2p/js-libp2p/blob/v0.29.2/examples/libp2p-in-the-browser/index.js – the libp2p-in-the-browser example. If I wanted to connect to a pre-existing DHT without running a node— I guess my only option would be to connect to the libp2p-in-the-browser nodes and random walk until I find one that responds to KadDHT, then get more KadDHT from that? Correct?

Then I guess my questions are:

  1. If I have the Bootstrap and KadDHT modules enabled at once, and I connect to a new peer found via Bootstrap (that is: I receive a peer:connect message on the connection), and the new peer happens to also have the KadDHT module, will libp2p-kad-dht automatically activate and start doing DHT things? Or do I need to manually activate it?

  2. I see I can use .handle() on a libp2p node to register the ability to handle incoming protocol connections, and .dialProtocol to open outgoing connections; and I see if the protocol is not handled by the other side, then dialProtocol will throw a ERR_UNSUPPORTED_PROTOCOL error. Is there a way to ensure I only peer with peers that support a particular protocol? (I assume I can wait until I get a peer:connect, attempt to dialProtocol, and then disconnect if I get ERR_UNSUPPORTED_PROTOCOL, but I also assume at that point nothing stops from me finding the same peer I don’t want to talk to again and again via gossip. So disconnection is not enough, I also need a way to “poison” the peer’s ID so that libp2p does not put it in the discovered list or open any new connections to it).

Thanks again.

Yes, per “It would be acceptable or even desirable if the DHT I am searching consists exclusively of instances of my app.”, you want to not be connected to the public libp2p network. So, you should use your own bootstrap nodes and webrtc star server, so that your application peers only discover each other.

The DHT needs to have at least one entry in its routing table to start. This way, at least one bootstrap node should be running the DHT protocol. Once you connect to the bootstrap nodes, the identify protocol will kick in and peers will exchange the protocols they support. With that, a DHT protocol stream is created between both parties and they are now part of an overlay network. After this, if you have the random walk discovery enabled in the config, your node will periodically try to find other nodes via the DHT peer it already knows. Once new peers are discovered, the node might connect to them until the connection manager gets to its maximum threshold.

Yes! So, grabbing the above explanation, the flow is:

  • Connection established
  • Identify protocol starts and peers exchange their known protocols
  • Peer Protocols are stored in the PeerStore’s ProtoBook
  • Libp2p topologies (DHT, pubsub are some of the topologies we have) will verify if the new “discovered” protocols for a peer include their protocols. If they have, the topology onConnect handler is triggered.
  • The protocol implements the onConnect logic. While this is not mandatory to happen, both DHT and pubsub open streams on their protocol to these peers and they become part of the DHT/pubsub topology automatically.

You can use the libp2p topology to help with this. You can take a look on the DHT network. This does not solve the disconnect+poison.
So, with the latest comment you might need to explore making autoDial to be false and manually trigger dials (if you use your own network with own bootstraps and no connectivity to the public network this should not be necessary, right?). With this, when peers are discovered (you can listen on the discovery event), you can manually dial them if you don’t have any information for them in the ProtoBook (be aware that peer protocols might also change over time) you can dial them and listen for the events on protocols change. When the protocol event is triggered, if your protocol is not there, you can “poison” the peer (use the metadataBook within the peerStore for this). So, you can add logic in your decision to dial on discovery the metadataBook check for poison.

FYI in this discovery+connection flows, we will be working on improving the developer experience for this soon libp2p/js-libp2p#744. You can follow it and also provide your input according to your experience while working on this. We want to get rid of the autoDial and to enable libp2p to become more intelligent on the peers it should connect to. This should include essential peers like rendezvous servers, relays, protocol peers (according to the topology configuration) and distance.

Thank you again. So I am trying to follow these instructions…

After reading the js-libp2p-websockets README I made a very simple standalone script bounce.js:

// Based on peer-and-content-routing 1.js
/* eslint-disable no-console */
'use strict'

const Libp2p = require('libp2p')
const Websockets = require('libp2p-websockets')
//const WebRTCStar = require('libp2p-webrtc-star')
const Mplex = require('libp2p-mplex')
const { NOISE } = require('libp2p-noise')
const KadDHT = require('libp2p-kad-dht')

const delay = require('delay')

const createNode = async () => {
  const node = await Libp2p.create({
    addresses: {
      listen: ['/ip4/0.0.0.0/tcp/10001/ws']
    },
    modules: {
      transport: [Websockets],
      streamMuxer: [Mplex],
      connEncryption: [NOISE],
      dht: KadDHT
    },
    config: {
      dht: {
        enabled: true
      }
    }
  })

  await node.start()
  return node
}

let node1
;(async () => {
  node1 = await createNode()
  console.log("Self is", node1.peerId.toB58String())
})();

I then set my web app to have only /ip4/127.0.0.1/tcp/10001/ws in the bootstrap list. Though this web app works fine connecting to the public IPs in the libp2p-in-the-browser script, it does not manage to discover the bounce.js peer. ("/ip4/127.0.0.1/tcp/10001" also did not work). Probing I find port 10001 is open on localhost, so the problem does not seem to be at the bounce.js end. Am I missing something?

A couple related questions about webrtc-star…

  1. Reading the js-libp2p-webrtc-star README it’s not so obvious how to set oneself up as a WStar listener. They give the example of creating a WStar object manually with WStar({ wrtc: require('wrtc') }), but how do I feed that into Libp2p.create? Can I get Libp2p.create to create the WStar object for me given the wrtc object?

  2. In webrtc-star , if a direct connect fails, particularly due to NAT traversal or firewalls, WebRTC ICE uses a relay TURN server. In other words, ICE will first use STUN with UDP to directly connect peers and, if that fails, will fall back to a TURN relay server.

    Okay. So would it be correct to say that (for my ‘application’ p2p network) the utility of hosting a webrtc “bootstrap” node on the public internet (assuming I already have the websocket “bootstrap” node working) would be only to allow nodes that can connect via webrtc-star but not websockets to enter the network? Webrtc-star peers do not help with initiating connections to other webrtc-star peers (except by gossiping about their existence)? (If this is correct it differs from the impression the comment in the libp2p-in-the-browser example initially gave me.)

    Are nodes able to discover websocket nodes through webrtc-star peers and vice versa? That is, do nodes gossip about the multiaddrs of peers with different connection types?

As a piece of feedback— if every libp2p app effectively has to host its own bootstrap server, that does seem to reduce the usefulness of libp2p. Once I have gone to the bother of setting up a host to run the bootstrap node, for many applications it would have been about as simple to just set up a traditional server for hosting the data. (Setting up a traditional server might even be easier because it could potentially be done inside application hosts such as Apache.) On the other hand, if it were somehow possible for participants in an app to find each other through public gateways such as the bootstrap.libp2p․io servers, this would make libp2p much more useful. Is this something that the “rendezvous” spec (referenced in the issue 744 you link) could possibly help with? A related question— if I understand everything correctly, the bootstrap.libp2p․io and wrtc-star1…dwebops.pub nodes listed in the libp2p-in-the-browser examples could never be used as the basis of any app, because the only thing those nodes can definitely do is gossip about other peers to discover— it seems that there’s no guarantee that any node in that bootstrap list (or among the peers you discover from the bootstrap nodes) will ever support any specific “protocol”, not even the libp2p “official” protocols like kad-dht. What exactly is the purpose of hosting those bootstrap.libp2p․io peers, then? Do they exist only for the sample-code apps like libp2p-in-the-browser to work? Have I missed something?

Thanks for your patience with my many questions.

Two possibilities here:

  • Does your web app have the websockets transport in? If you can show me the config might help.
  • Are you running the web app on localhost as well?

I recommend that you use the debug logs to try to gather more information. You can do it in the browser console by:

  • localStorage.debug = ‘libp2p*’
  • and you can increase the specificity of it, maybe in this case see the transport level logs: localStorage.debug = ‘libp2p:websockets*’

If I understand your point correctly, webrtc-star signal server is not a bootstrap node. Bootstrap nodes are other libp2p nodes, while the signal server is a server that enables the network to overcome some challenges that otherwise it would not be able to at this moment.

webrtc-star comes with built in discovery, but what this means? The star server will keep connections with peers that listen for connections through it. This way, every time a new node “joins”, the star server will emit signals to the already connected nodes, to inform them of this new peer, which they can reach through this same server. So, no websocket discovery at this point. However, once they establish a connection and the identify protocol starts, both peers will exchange their protocols and multiaddrs. With this, a peer could get to know websockets addresses from the other peer, if it has them. For further connections, it might use this instead of the webrtc-star address.

The main advantages of using the webrtc-star are the built in discovery and NAT traversal support.

I am not sure if I entirely answered all your questions, feel free to redo them.

It does not need to. You can be connected to the public network instead, using the default bootstrap nodes.

Yes totally! I am implementing the rendezvous protocol in JS, which should land for the next release: https://github.com/libp2p/js-libp2p/issues/655

While there is no guarantee, in theory the bootstrap list should contain regular libp2p nodes with its protocols enabled. The purpose of the bootstrap nodes is not more that showing you a way into the network, which will enable you to discover and connect to other peers that are meaningful to you. We are heading towards a flow like:

  • node starts for the first time
  • node connects to its bootstrap nodes (this should include the public bootstrap nodes, rendezvous server nodes, relay nodes)
  • node will try to discover its closest peers on the network leveraging the connected nodes and connect to them
  • node will try to find its important nodes on the network via service discovery (https://github.com/libp2p/js-libp2p/issues/768), which will use rendezvous/content routing
  • node will try to find important pubsub peers (shared topic subscriptions)
  • node will connect to all the important peers discovered
  • node will disconnect from the bootstrap nodes once it already has “enough” peers for its subsystems
  • if a node stops and starts again, it will have all its discovered nodes persisted and it can try to connect back to them. It will only leverage the bootstrap nodes again if needed. The Peerstore stores peers’ protocols and metadata which will enable us to figure out what peers to connect.

This is the path we are going, but most of this is still being worked on. The next 2 releases should probably support most of this. You can read a bit more about this on:

I think I understand what was confusing me before about webrtc-star, thanks. I will test more with setting up my bounce.js tonight.

Just to triple check…

This one step relies on future features, not currently supported features, correct? So with 2.9.2, either you must get lucky and find an “important” node through chance, or run app-specific bootstrap+signal nodes which will ensure that all nodes are “important”. Correct?

Yes. It depends on what features you use and how you use them. For example, using webrtc-star for a given dapp, will make all peers to discover and connect to each other out of the box. This way, if they use pubsub all the peers will be connected and everything will just work.
If you are talking about network querying with DHT for example, or not shared bootstrap nodes, than yes.
This is the case in JS. go-libp2p is more advanced by the way.

1 Like

So, I’m sorry, I tried both localStorage.debug = 'libp2p*' and localStorage.debug = 'libp2p:websockets*' and neither one did anything. I tried these lines both in my bundle.js right before require()ing the libp2p and calling libp2p.create(), and I tried them in the console before starting the libp2p node. They did nothing at all, there are no additional messages in the console. Am I missing something?

Here is my browser-bundle.ts.

// Adapted from index.js, libp2p-js "libp2p-in-the-browser" example, git tag v0.29.0

declare let require:any

//localStorage.debug = 'libp2p:websockets*'
localStorage.debug = 'libp2p*'

const Libp2p = require('libp2p')
const Websockets = require('libp2p-websockets')
const WebRTCStar = require('libp2p-webrtc-star')
const NOISE = require('libp2p-noise')
const Mplex = require('libp2p-mplex')
const Bootstrap = require('libp2p-bootstrap')
const KadDHT = require('libp2p-kad-dht')

// Starting peers
const bootstrapList = [
  '/ip4/127.0.0.1/tcp/10001/ws'
]

// Used for inbound connections when NATed
// libp2p-in-the-browser comment claims these are "added to our multiaddrs list"
const signalingList = [
  '/dns4/wrtc-star1.par.dwebops.pub/tcp/443/wss/p2p-webrtc-star',
  '/dns4/wrtc-star2.sjc.dwebops.pub/tcp/443/wss/p2p-webrtc-star'
]

const Node:any = Libp2p.create({
  addresses: {
    //listen: signalingList
  },
  modules: {
    transport: [Websockets, WebRTCStar],
    connEncryption: [NOISE],
    streamMuxer: [Mplex],
    peerDiscovery: [Bootstrap],
    dht: KadDHT,
  },
  config: {
    peerDiscovery: {
      bootstrap: {
        enabled: true,
        list: bootstrapList
      }
    }
  }
})

// Note export is a promise
export { Node }

Can go-libp2p be run in a browser (for example with wasm?)

Oh… this is very weird. So I have two test apps. One is using 0.29.0 from npm and the other is using 0.29.2 from npm. The 0.29.0 app IMMEDIATELY picks up the localStorage.debug and is giving me an enormous amount of console spew. The 0.29.2 app (the one I want to debug…) does not. Is this surprising?

EDIT: Upgraded my 2.29.2 test app to 0.29.3, cleared all cache, still no console prints.

I believe there were some experiments, not sure if go or rust. But, I would not recommend that at this point.

Is this surprising?

Yes, could you try in another browser just to double check? We simply use npm debug package for the logs, nothing should have changed

For what I saw on your code, I would not recommend to use that as you are. I recommend you simply use the browser console for this:

  • Open a tab, and open the console
  • Do the localStorage command on the console
  • Get in your web app / Refresh

Thanks, I tried it in Firefox and worked now.

Now that I can see the libp2p debug messages, I notice this important looking error:

libp2p:bootstrap:error Invalid multiaddr +0ms

Just as feedback, this error message would probably be more helpful if it included the multiaddr it doesn’t like. But it doesn’t matter because I only have one multiaddr in my bootstrap list in these tests.

The multiaddr it’s failing on is:

/ip4/127.0.0.1/tcp/10001/ws

I also saw it fail on

/dns4/THE.DOMAIN.NAME.OF.MY.VPS.com/tcp/10001/ws

I’m sorry to be asking all these basic questions. But, are either of these multiaddrs incorrectly formed (given I’m trying to connect to the standalone bounce.js in this post run via node)?

Ok good! Strange that you are having issues on other browsers though. I usually use { Chrome, Brave, Firefox } and never found issues with the debug.

Yes! The error is quite bad though. The problem is that it must be a valid P2P address. This means that it must have a peerId: /dns4/THE.DOMAIN.NAME.OF.MY.VPS.com/tcp/10001/ws/p2p/YOUR_PEER_ID.

When you dial to a given address, you must know its ID. This is the only way we actually have guarantees on who you are talking to.

Would you like to improve the error on https://github.com/libp2p/js-libp2p-bootstrap/blob/master/src/index.js#L58 ? I think it makes sense to log the multiaddr as you suggested and perhaps mentioning it must include the PeerId of the target.

Maybe something associated with the Chrome profile needs to be cleared or deleted. I will experiment.

Hm, OK. But I notice that every time I run my server script it has a different PeerID. I guess I need to save a PeerID object and store it in a file or something. There’s no existing example code for that, right?

My standalone bounce server is starting to seem a bit complicated. Possibly once I have it working with the peer ID storage I will submit it to js-libp2p as an example.

I will attempt a PR but since I am still learning the requirements I may miss some things…

How do you run your bounder node?
My recommendation is to create a PeerId with its CLI like peer-id --type ed25519 --bits 2048 and save it in JSON. After this, you create a PeerId to provide to Libp2p.create({ peerId }) and the peerId is created by PeerId.createFromJSON

Just with node bounce.js. Thanks for the suggestions

So, I got a little further this time.

Here is my bounce.js now: https://gist.github.com/mcclure/c05ffc31329cfd80b40b04b9a9618bbe

I don’t get a successful peer connection. It does attempt a connection however. I get these in the developer console with debug = libp2p*:

The main error seems to be: “Failed to upgrade outbound connection +0ms Error: protocol selection failed”

My browser-bundle.js is still unchanged from my earlier comment #9. I’m not trying to use -star yet. The only thing I changed is the bootstrap list now contains

/ip4/127.0.0.1/tcp/10001/ws/p2p/12D3KooWGsdpzkAqD7vqpPdNjSxQWy66mSZTrmnYZTVnyjUhJCcR

I’m not trying yet to create a custom protocol or dial() or anything.

Any advice?

Not sure where the issue is. Somehow, the crypto protocol is not well set. Just ran your bouncer code and set the address in the bootstrap addresses from the browser example and it worked:

libp2p:upgrader Starting the outbound connection upgrade +0ms
common.js:111 libp2p:upgrader selecting outbound crypto protocol +0ms ["/noise"]
common.js:111 libp2p:upgrader encrypting outbound connection to {"id":"12D3KooWHvP1HJGRDkieJtT9RoZY12m2J4jiPrWhVwyUHWXFevij","pubKey":"CAESIHhn7S7T30TeAuukS6bzLFPp6CKz/08dBjZSG7DzOimQ"} +9ms
common.js:114 libp2p:noise Stage 0 - Initiator starting to send first message. +0ms
common.js:114 libp2p:noise Stage 0 - Initiator finished sending first message. +6ms
common.js:114 libp2p:noise Stage 1 - Initiator waiting to receive first message from responder... +1ms
common.js:114 libp2p:noise Stage 1 - Initiator received the message. +71ms
common.js:114 libp2p:noise Initiator going to check remote's signature... +0ms
common.js:114 libp2p:noise All good with the signature! +34ms
common.js:114 libp2p:noise Stage 2 - Initiator sending third handshake message. +0ms
common.js:114 libp2p:noise Stage 2 - Initiator sent message with signed payload. +25ms

I noticed you had const NOISE = require('libp2p-noise') in Js-libp2p "peer and content routing" / kad-dht example

Could be that the problem, since it should be const { Noise } = require('libp2p-noise'). Let me know

oh, this fixed it! :open_mouth: thanks

I am going to try to get WebRTC-star working next, if it doesn’t work I will check back here…

1 Like

So, WebRTC-Star didn’t seem to work…

Here’s what I did.

In my bounce.js, I added const wsPort = 10001, wssPort = 10002 at the top, and changed my listen to

    listen: [
        `/ip4/0.0.0.0/tcp/${wsPort}/ws`,
        `/ip4/0.0.0.0/tcp/${wssPort}/wss`
      ]

Meanwhile, I changed my browserbundle to

const familyIp = "ip4/127.0.0.1", wsPort = 10001, wssPort = 10002, publicKey = "12D3KooWGsdpzkAqD7vqpPdNjSxQWy66mSZTrmnYZTVnyjUhJCcR"

// Starting peers
const bootstrapList = [
  `/${familyIp}/tcp/${wsPort}/ws/p2p/${publicKey}`
]

// Used for inbound connections when NATed
// libp2p-in-the-browser comment claims these are "added to our multiaddrs list"
const signalingList = [
  `/${familyIp}/tcp/${wssPort}/wss/p2p-webrtc-star`
]

const Node:any = Libp2p.create({
  addresses: {
    listen: signalingList
  },
  modules: {
    transport: [Websockets, WebRTCStar],
    connEncryption: [NOISE],
    streamMuxer: [Mplex],
    peerDiscovery: [Bootstrap],
    dht: KadDHT,
  },
  config: {
    peerDiscovery: {
      bootstrap: {
        enabled: true,
        list: bootstrapList
      }
    }
  }
})

So I am assuming I can run a websockets transport and a webrtc-star signaler in the same script, as long as I do it on two different ports. I did not put a public key in the signaling server multiaddr because I tried that and it immediately threw an error saying that [keyname] is not a protocol.

Anyway, the above did not work. Here is the libp2p* debug:

The main error seems to be: Error: Transport (WebRTCStar) could not listen on any available address. (By comparison if I replace my 127.0.0.1 server with '/dns4/wrtc-star1.par.dwebops.pub/tcp/443/wss/p2p-webrtc-star', I instead get many messages like dialing dwebops, connection opened to dwebops etc.)

Any suggestions? I can post my full up to date browser-bundle and bounce.js text if it helps.