Js-libp2p "peer and content routing" / kad-dht example

So, as far as I understand, you are not running your own star signal server, is that correct? If that is the case, you should spin up your own: https://github.com/libp2p/js-libp2p-webrtc-star/blob/master/DEPLOYMENT.md . The dns ones you pointed out are these signal servers hosted by the Libp2p Project for people to play with.

You can also do something like https://suda.pl/free-webrtc-star-heroku/

Next year we should add the capability for distributed signalling via libp2p relays. When we support that, you should be able to leverage a set of libp2p nodes for this. Meanwhile, this is the feasible approach.

Hm, OK, I think I was confused. I was trying to create a star signal server. I thought I could turn the bounce.js node into a star signal server by just telling it to act as a wss node and passing it wrtc. I guess that’s not how it works.

I find these instructions a little confusing. If I need to use docker then I need to use docker!, but there seems to be a “sig-server” directory in js-libp2p-webrtc star. Is the docker image just a container for this “sig-server” node script? It seems weird to bring up docker when I could just run the sig-server directly in node. What does the docker image in the DEPLOYMENT.md provide that just running node sig-server/bin.js does not? Can I use my local SSL certificates with bin.js?

One more question about the -star server— earlier you said

Is there a way when running the -star signaling server to set up the server to volunteer for STUN peer connection, but not volunteer to relay actual content from peer to peer?

Thanks again for taking the time to answer my questions.

Where did got this confusion? Any docs hinted you that?

You can test it locally just running the bin. But for production, you will need to setup the SSL certificates, which you cannot do at this point in libp2p. You also do not need to use docker, but you would need to setup your nginx reverse proxy and use the bin.js with it. I think the docker setup is easier and faster, but you can move on from it after, if you like. Heroku is a nice temporary solution for experimenting .

After the connection is established, the peers will directly communicate over webrtc. So, the star server will not relay the actual content.

Thanks again for the help. I think I now have connection between two NATed nodes using a custom bootstrap + signaling server.

Two more weird questions…

  • Is there a way for a node to know what name another node knows it by? For example, if I have an app that in the bootstrap list is told to connect to /dns4/sub1.mydomain.com/tcp/9111/ws/p2p/SOMEKEY, does the bootstrap node know that the other node thinks of it as sub1.mydomain.com? (I am thinking of doing something hacky involving multiple subdomains pointing to the same IP.)

  • Is there a way within libp2p, with either a node or the signaling server, to track libp2p’s data usage? That is, how much data up/down libp2p has used. I would like to measure how much bandwidth the bounce/signaling server are using. (I may be able to track this on a per-process basis without assistance from libp2p, I haven’t looked into this yet.)

I’m trying to figure out where I got this wrong. I think I probably just made a bad assumption that there was a symmetry between the “signaling server” and a normal node, an assumption that was easier to make because there is symmetry between a websockets bootstrap server and a normal node. Maybe it would help to add an additional sentence to the “Rendezvous server (aka Signaling server)” section of the js-libp2p-webrtc-star README such as “This signaling server is different from a normal peer, and is required for libp2p-webrtc-star to work”. (This is already suggested by the existing text but since learning to use libp2p involves introduction to many new concepts at once, maybe it helps to be extra explicit.)

It would also probably help to have an explicit document explaining how to replace the public servers used by the samples with your own servers. Right now for example the libp2p-in-the-browser example links only to the js-libp2p-webrtc-star/DEPLOYMENT.md, which is vague on a lot of points and does not cover matters such as how to make a websockets bootstrap node. Once I get everything I have learned in this thread up and working I am intending to attempt to write a guide explaining what I did and how, so I can submit that upstream.

Following up on this.

So this is my current logic in the bounce server.

  // Create node with peer
  node1 = await createNode(peerId)
  console.log("Self is", node1.peerId.toB58String())

  if (commander.protocol) { // See note at top
    const tempBuffer = new Uint8Array([0]) // Single zero byte, indicating a 0-length message, indicating ACK
    const protocolString = `/{commander.protocol}`

    node1.handle([protocolString], ({ protocol, stream }) => {
      (async () => {
        await stream.write(tempBuffer, 1)
      })()
    })

    node1.connectionManager.on('peer:connect', async (connection) => {
console.log("TRACE-1")
      try {
        await node1.dialProtocol(connection.remotePeer, protocolString)
console.log("TRACE-2")
      } catch (e) {
        console.log("Connection failed:", e)
        node1.hangUp(connection.remotePeer)
        node1.peerStore.addressBook.delete(connection.remotePeer)
      }
console.log("TRACE-3")
    })
  }

My goal is that every peer that tries to talk to the bounce server, before the bounce server adds that peer to the gossip list, it should screen it to ensure a dial with the protocol is successful.

(You are correct that if the bounce server is specific to the application then probably no one will connect to the bounce server unless they are running the application, but, I have seen strange things happen on P2P networks.)

In my implementation, however, what I really did was wait for a connection to be opened, and when it opens I dial the protocol and see if that raises an exception or not. If an exception is raised I close the connection and remove the peer from the peerStore.

In my testing, what happens if I run a non-compliant node is the node connects, the bounce server prints TRACE-1, there is a pause of several seconds, and then the bounce server prints the error. Unless node is flushing messages strangely, it seems that an unsuccessful dial does not fail immediately but times out. When the noncompliant node is disconnected, it seems to immediately try to reconnect, it tries several times but it seems to back off exponentially.

autoDial is false on the bounce server (autoDial defaults to false, right? this is unclear to me in the documentation) but this doesn’t matter because the point of the bounce server is nodes connect to it.

So this is probably “good enough” but it does not quite meet my goal because I think I am still potentially gossiping about noncompliant nodes. I think (?) that in the several seconds between a noncompliant node connecting and me realizing it is noncompliant and disconnecting it, I will probably still gossip about that node to anyone I’m talking to in the meanwhile. That’s mostly harmless but I am curious.

What determines whether a node is gossiped about? Is it just anything in the peerStore? What controls when things are added to the peerStore?

Looking at the peerStore config and peerStore methods I don’t find any good hints. It appears something automatic is happening but I cannot find where it is documented. What should I look at here?

Nice, happy to hear!

We have this notion of observedAddresses, which I think is what you are looking for: https://github.com/libp2p/js-libp2p/blob/v0.29.4/src/identify/index.js#L217 However, this is not yet fully implemented. The blocker is that we need to have multiaddr confidence in place, in order to properly track what multiaddrs are more likely to work.

For now, you probably will have to do your own protocol:

You would basically have a protocol (let’s say a one message each chat), where once peers connect they would send each other the known multiaddrs of the other one (getting them from peerStore.addressBook.get)

Yes! You can look into:

Thanks, that would be super nice! :slight_smile: I agree with you, I started production guides https://github.com/libp2p/js-libp2p/tree/0.30.x/doc/production but they still have a long way to go. It is important for us to have users that try it out and help with this type of documentation.

autoDial is true by default at the moment. That will change in a couple of releases.

I think you can simplify this to:

if (commander.protocol) { // See note at top
    const tempBuffer = new Uint8Array([0]) // Single zero byte, indicating a 0-length message, indicating ACK
    const protocolString = `/{commander.protocol}`

    node1.handle([protocolString], ({ protocol, stream }) => {
      (async () => {
        await stream.write(tempBuffer, 1)
      })()
    })

    node1.peerStore.on('change:protocols', ({ peerId, protocols}) => {
      if (!protocols.include(protocolString)) {
        node1.hangUp(peerId)
        node1.peerStore.addressBook.delete(peerId)
      }
    })
  }

When two peers connect, they will run identify and exchange protocols. This will always be triggered and you kick it out.

I need to double check what might be happening, as I believe it should not take too long to fail the dialProtocol.

Yes, that currently happens as the other peer will likely have autoDial set to true. We used to have a deny list, but we do not support it since we removed libp2p-switch. There are two things that we need to do in this context:

It depends on the subsystems enabled. In gossipsub peer exchange, this will not likely happen unless you exchange pubsub messages with the peer. In the DHT side of things, it will happen if DHT queries get to that ID on the algortihm.
It would probably be nice if libp2p could offer a customFilterFunction that would filter out peers that could be shared. With this, users could just add on peer:connect metadata to the PeerStore stating that the peer is being “inspected”. The customFilterFunction would remove these peers. Once the protocol for the peer is validated, the metadata would be removed. We will also work on connection tags, which could probably be also used in this context if we provide such function to the user.

Nothing controls things being added to the peerStore, and I think that is correct. We could just have the filter I described above. Once we know something new from a peer (addresses, protocols) it is added by libp2p to the PeerStore. You can see these flows in https://github.com/libp2p/js-libp2p/tree/master/src/peer-store

1 Like