Is there a way by which a new node that just joined can ask for all the messages it has missed for a particular topic?
I understand that the gossipsub will provide those to the node eventually using the IHAVE and IWANT exchanges but I am concerned about the delay in getting the node populated with all the messages it has missed with that approach.
In a p2p network, you don’t want nodes to store past messages by default. That can expose significant vectors for DoS. In the past, we have discussed storing the last message and serving it upon a new GRAFT (i.e. a peer that joins the mesh for a topic).
One of such behaviours is what you call LWW (last writer wins), which is similar to the “last image caching” subscription recovery policy of ActiveMQ, where you retain only the newest message so that new subscribers (or ones that missed the message) can fetch it.
from PubSub & GossipSub configuration proposal · Issue #175 · libp2p/go-libp2p-pubsub · GitHub
Are you really looking for a sync procedure, @vishal?
@vishal can you give greater detail about what you’re trying to accomplish? Are you trying to get all the nodes in a topic to have the same state?
If so you may want to check out https://github.com/libp2p/go-libp2p-pubsub-router, it is a Key-Value store based on pubsub that allows nodes that join the pubsub channel to quickly get up to speed. You may either be able to use that code, or build something similar for your use case based off of it.
Thanks @adin and @raul for your replies. The use case that I am trying to address is that if a node just joins, he would like to come up to speed with the earlier messages and converge on the same state as the other nodes. I will look into the links that you have shared. Thank you ! cheers.
Just to close the loop on this one; pubsub is not a state synchronisation protocol. Nodes to not keep a historical database of messages that others can request on demand. Suggestion here is to look at how other blockchains like ETH and Bitcoin deal with state sync, which is dissociated from pubsub / immediate data dissemination. For the record, IWANT and IHAVE messages are used to gossip and request messages within a specific rolling lookback window.