Js-libp2p "peer and content routing" / kad-dht example

[vasco-santos]

Hi @mcc

@mcc

It would be acceptable or even desirable if the DHT I am searching consists exclusively of instances of my app.

In here, you mean search only on the peers you are connected to via bootstrap?

@mcc

I can run this example in node.js in two Terminal windows simultaneously. The two nodes find each other. How are they finding each other? Is the hardcoded 0.0.0.0 address simply finding “all js-libp2p instances on the local machine”?

The listen: ['/ip4/0.0.0.0/tcp/0'] is the addresses that the configured transport (TCP) should listen on. 0.0.0.0 means all IPv4 addresses on the local machine. For example, if a host has two IP addresses, 192.168.1.1 and 10.0.0.1, and a server running on the host listens on 0.0.0.0, it will be reachable at both IPs. The 0 port means to choose a random available port. If you change this example to listen: ['/ip4/127.0.0.1/tcp/0']you can see that it will work, but the found addresses will only be that address.
Whit the above in mind, all the 3 spinned nodes will be listening for connections on the local machine addresses. Then in the example, we connect A to B and B to C. This way, B knows about both A and C, but the others only know about B. Then, A will do a DHT query to find Peer C addresses. It will query the peers it is connected to running the DHT protocol (according to the DHT query algorithm). So, A asks B (only known peer) if it knows C. As B knows C, it will just return its addresses to A. If it didn’t know, it would return the closest peers it knew to the requested peer.

@mcc

How would I attempt, in a web app, to connect to a Kademlia DHT on the general internet? Can libp2p-bootstrap be used for this? Is it sufficient to simply load libp2p-bootstrap and libp2p-kad-dht at once or are there extra steps?

So long story short, the bootstrap nodes are the entry point in the network as a node needs to know peers, in order to find other peers easily. In this case, the bootstrap nodes should be running the DHT protocol and should know a lot of peers. Once the bootstrap nodes are connected and the DHT protocol kicks in, it will use its built in discovery protocol (random walk) to find peers once in a while. This is as simple as generating a random peer ID and try to find it in the network. The goal is not even to find the computed peer ID, but to go through the process of asking known nodes if they node that Peer. As a consequence, known nodes will answer with the closest peers they know and you will get to know new peers with this process.
Please note that there are known problems and the js DHT in the browser is not stable at all in the moment. Also, go DHT had significant improvements over the last year and we will be working on porting these changes to js and improve the general state of the DHT across different environments soon. If you are testing it, you should use its client mode and rely on go nodes to make DHT queries for now.

@mcc

Say I have libp2p-bootstrap or libp2p-kad-dht working in a web app. Say my configured "transport"s are “libp2p-websockets” and “libp2p-webrtc-star”. Say I open two browsers on separate computers A and B running this web app. Say one or both of A and B are behind NAT such as a consumer router. Will A be able to open a connection to B? Are there additional steps I must do to make A<->B connections possible? Also, are A<->B connections exclusively possible using the libp2p-webrtc-star transport (the page for libp2p-webrtc-star implies some STUN/TURN-like things are happening)? If so, is there any reason in the web app to include the libp2p-websockets transport at all?

In webrtc-star, if a direct connect fails, particularly due to NAT traversal or firewalls, WebRTC ICE uses a relay TURN server. In other words, ICE will first use STUN with UDP to directly connect peers and, if that fails, will fall back to a TURN relay server.
The main reason to support multiple transport protocols is to be able to speak with more peers. It certainly depends on what the use case. For example, go does not support webrtc at the moment. As a consequence, for a browser peer to talk with a go node, it must use websockets.

@mcc

Why does this example create three nodes? Is it so that the example network consisting of two node.js processes will have six nodes and therefore a nontrivial search space? Or if you were connecting to a global/internet DHT, would you tend to create multiple nodes so that you have multiple probes into the Kademlia hash space?

I think I answered to this above. The reason is to create a simple topology where we can rely on DHT algorithms. Any DHT node needs to know any other node so that it can query it.

---

FYI, another solution we highly recommend for peer/content routing is delegated routing. You can see it here. It will rely on more capable nodes in the network to do DHT queries on your behalf. js-ipfs currently uses this approach with some public delegate nodes. This is the best solution for content/peer routing while we do not work on the DHT improvements mentioned above.