Filtering nodes that we gossip to

Hey guys!

I want to know whether there is any way of deciding at the application level which peers we want to forward or gossip a message to? This is related to TopicValidator which delivers to the application but does not forward to the network

Use case: We essentially have a network that is split into two separate sides, with a specific set of nodes (called “staked ANs”) act as the gatekeepers between one side of the network and the other.

I want to allow messages on certain topics to pass from the right side of the network to the left side, and I want NO messages to pass from the left to the right.

Originally, we were thinking we could accomplish this using Topic Validators, as discussed in the post above. However, there are two concerns here.

  • Will this impact how nodes on the left side score the staked ANs? They would notice that the staked AN’s are dropping all their messages, and hence would decrease their score right?
  • How can we control which topics to allow messages to pass from the right to the left? For topics we don’t want messages to pass through, we need some way of only forwarding / gossipping them back to the right side of the network (we also cannot just drop them entirely once we receive them or else nodes on the right side will decrease our score).

Abstractly, these two concerns are the same:

We need a mechanism where, given a message, we can filter out nodes that we don’t want to forward or gossip about the message to.

A more subtle point: To be fully secure, we probably also want some mechanism so that if the nodes on the left side of the network sends an IWANT request for a message that we received but that was never forwarded / gossiped to that side of the network (and hence is a message the left side of the network should not know about), we ignore / act as if that message doesn’t exist.

This is an interesting use case, we could potentially allow for a user filter in gossip node selection. Can you open an issue so that we discuss there and develop a solution?

Okay, I will create an issue. Something related to this that I wanted to ask, is about the peer information sharing.

In the above scenario, we also want to ensure that peer address information of nodes on the right side of the network do not get revealed to nodes on the left. However, in the protocol the peers will tell each other about peers that they know about, right? And if so, is there some way for me to say “for peers among a given peer ID set, I should never share their information with peers outside of that set”?

Only if PX is enabled, which is not by default. We can probably attach a user filter there too.

If I interpreted correctly, you are saying they will not tell other peers about information from their own peerstore by default, correct?

I should also mention that we are using DHT on the left side of the network for peer discovery. Does that change anything? For example, could a node on the left side of the network find out about peers of the staked AN’s that are on the right side of the network via DHT queries or anything like that?

Also #1153, #1154, and #1155 (I wasn’t allowed to post more than 2 links)

Yes, pubsub doesn’t tell anything about it’s peers unless configured with PX, in which case it will return some peers duing prune.

If you are using random walks with the DHT, you’ll likely find peers on both sides of the fence.


So I think Filter for nodes that we gossip to · Issue #1152 · libp2p/go-libp2p · GitHub is the most urgent for us right now, may I start working on a PR for this?

that would be great, go for it!

1 Like

Do you have any references to the code for a definitive yes or no? It wasn’t entirely clear to me when I tried to look through it.

Working on it now :slight_smile:

Hey @vyzo, so I’ve thought about our usecase a little more, and I realize that the staked AN’s in the middle are basically just acting as relayers who relay messages unidirectionally from the right side to the left side of the network.

Once the messages reach the left side, the nodes on the left can gossip those messages among themselves all they want, but what we don’t want is for new messages to go from the left to the right.

What would really be simpler here, both conceptually and in implementation, is a mechanism that allows us to relay the message to a different topic.

There are two obstacles to this approach though:

  • We could just relay it by calling Topic.Publish, but that will fill in our own peer ID in the From field. This means that if two different staked ANs relay the same message from the right side of the network to the leftside, they will become two distinct messages on the left side, causing unnecessary message duplication.
  • Maybe we could call PubSubRouter.Publish instead, but as far as I’m aware this interface isn’t actually available to our application code. For example the gossipsub constructor wraps the creation of the router, and the router is not a public field. Further more, we cannot just construct our own router since the fields we need to set on it are private: go-libp2p-pubsub/gossipsub.go at master · libp2p/go-libp2p-pubsub · GitHub

Do you think it would make sense to make the PubSubRouter directly accessible from application code? Would there be any side effects if I directly published a message to the router?

I think you can more simply accomplish the same thing by using two different topics; the middlemen subscribe to both and forward between topics as needed.

For the distinct messages issue, you can use a custom message id function that uses the hash instead of the default.

1 Like

Ah, I was not aware a custom message ID function was allowed.

@vygo back to the peer exchange question:

So in the default, non PX, non DHT gossipsub, how does a peer find out about new peers they can connect to?

Originally my understanding was that peers tell each others about other peers they know about, but you said this isn’t the case.

But then, does that mean that as a new peer joining the network I will never learn about other peers besides the ones I already knew prior to joining the network?

Or do you just mean that this only happens if you provide a discovery object to the pubsub?

Secondly, I was wondering if it is possible to completely ignore certain peers on a given topic?

Meaning, I will not forward or gossip about any messages I hear on that channel to them. As far as I’m concerned, they’re not subscribed to the channel at all, even if they tell me they are.

I saw the Subscription Filters, but @vishal tried it out and it doesn’t seem to do what we thought it would do,

I observed that while the subscription filter does ignore the subscription request from a peer for a topic, it does not stop the peer from gossiping the message for that topic.

  1. In my little test, A, B and C are 3 nodes, A connects to B and C, and all three subscribe to two topics - 1 & 2.
  2. A and B use a subscription filter which filter C from sending subscription request for topic 2.
  3. When C does send a message on topic 2, A and B both get the message.

I am guessing this is as-designed since ?

@vyzo Any updates on this?

Are we just not using the Subscription Filter correctly?

If the subscription filter dropd a subscription, then the peer wont be considered in the topic and it shouldnt emit gossip.

You have somewhat involved requirements, maybe best to get on a call and figure things out.

That would be super helpful, how could we set this up?

I’ll have to get back from vacation first, but we can zoom. You can email me at vyzo at protocol ai

1 Like