Gossip questions

Implementers and Contributors
1

Hey guys!

We are now implementing Libp2p for JVM. The first phase is targeted for using in Ethereum 2.0. The key protocol for Ethereum is gossip. Thus I have some questions regarding it:

  1. Is it true that peers are added to fanout map when (and only when) your peer is publishing to some topic but not subscribed?
    I.e. if every peer in the network is subscribed to a topic-1 then fanout maps of all peers are empty?

  2. gossips are emitted to fanout and peers in a mesh only?

  3. If the above is true (I’m relaying on the Go implementation), then if we have a network where all peers a subscribed to a single topic then fanout and gossip features have no effect and the message propagation relies on meshes only?

@raul advised me to call for @vyzo to this topic as gossip author

Thanks a lot!

1 Like
3

Hi there,

Some quick answers:

  1. yes

  2. gossip is emitted to peers not in the fanout or the mesh set; ie it is emitted to other known peers

  3. not true; gossip is emitted to peers outside the mesh; this allows you to overcome overlay pathologies and oppurtunistically jump hops.

It also allows for regrafting the overlay, which is a key feature of the next iteration, episub.

hope that helps.

– vyzo

1 Like
4

Seems there is a minor bug in the go implementation, gossip may be emitted to fanout peers as well; this should be fixed.

– vyzo

5

@vyzo Thanks for prompt response! This now makes much more sense :+1:

6

Hey @vyzo!

Replaying here my question from GitHub (https://github.com/libp2p/go-libp2p-pubsub/issues/133#issuecomment-520509632):

Did I get it correctly, that pubsub communication between 2 peers is kind of ‘half-duplex’ in terms of streams? I.e. each peer opens a write-only stream and 2 -side communication requires 2 streams?
If this is correct then can you please explain the rationale behind this solution?

Thank you!

7

yes, this is correct. It’s a historical design decision for the pubsub implementation, as it is the simplest way to proceed.

8

as it is the simplest way to proceed.

So what was the major problem with using a single stream? It seems pretty natural.
Looks like two-stream solution causes more problems

9

Gossipsub is designed as an “ambient” service. This means that as we connect to peers in the network, we interrogate if they support gossip by opening a stream. This approach has nice properties (including a level of randomness contributing to attack resistance), but it also means that each node needs to probe its peer by opening and owning a dedicated stream.

Alternatively, we could:

  1. wait for identify to finish, but this won’t work well in future monoplexed scenarios.
  2. run a role assignment heuristic (e.g. highest peer id wins and is in charge of opening the canonical full-duplex stream), but this introduces a timing dependency for the passive party.
  3. consolidate both streams into a single one after the fact, but that’s complex and has no benefit.

Really, keeping two half-duplex streams where each party owns one simplifies the design a lot.

1 Like
10

@raul Thanks for detailed explanation!
Are there any other protocols which uses the same scheme?
I believe there are p2p protocols which uses a single full-duplex streams. How do they solve role-assignment then?

11

Also why not just use the rule, that only the dialing peer opens a full-duplex stream?

12

The stream architecture of a protocol is dependent on the requirements of the protocol itself. libp2p should be flexible enough to accommodate any style. Empirically speaking, there are (at least):

  1. RPC-based protocols that rely on single-use streams (one per request).
  2. Protocols that better suited to long-lived streams (e.g. overlay networks) where only one party has an active interest, and the one remains passive (e.g. unidirectional flows of data). The active peer may open multiple streams against the passive peer. An example is the circuit relay protocol.
  3. Protocols that are inherently bi-directional, with both peers playing an active role. In this case, it’s common to have each peer own a stream.

The issue is that you don’t know if the dialing peer is or not a gossipsub peer. Therefore, as the responder of the dial, gossipsub would need to park the candidate, introduce a timeout, deal with this transitory state in mesh maintenance, etc. Just like resolving the conflict based on peer ID, it’s a deterministic rule, but it leads to unnecessary complexity. And streams are cheap.

13

Why not use the following algorithm:

  • A dialer supporting pubsub always actively opens a pubsub stream (if remote listener supports it)
  • Listener just waits and passively creates a pubsub stream on remote request
  • pubsub router takes a peer into account only when a pubsub stream is opened. No candidates, timeouts and transitory states

If connecting peer doesn’t open pubsub stream then it doesn’t support it and the pubsub router doesn’t even know about this peer.

@raul no intention to redesign libp2p pubsub, just want to understand the background

14

The whole point of half-duplex streams is that it doesn’t need any algorithm, and is thus much simpler to implement and more robust.

But really, what would you gain by having duplex streams? Streams are cheap and they reuse the same connection.