I’ve got a race condition where two participants in a libp2p gossipsub network both join a topic around the same time and miss the initial messages from each other. Is there a way for a client to discover past messages published based on some retention policy (e.g. 1 minute), and then re-request copies of those messages?
I saw that gossipsub broadcasts metadata and allows nodes to pull data they need, is there someway to access that capability in the libp2p pubsub libs?
Thanks!
It is not using gossippub for messaging.
Gossippub isn’t a messaging protocol, it is a multicast one.
Gossippub would be great to sync the heads of a CRDT for example.
Create a linked list of messages, each one being an IPLD block and have each message point to the previous one.
And broadcast on gossippub each new last message CID and the last one from time to time.
If you miss a message, either you wait the next rebroadcast and then download it, or when the next message (after the one havn’t received) is sent, you download it, then realise it links to a CID you don’t have and download that one aswell.
It’s what https://berty.tech/ do for example.
EDIT:
You could also do something more classical, like publishing messages with a sequence ID, and asking for resend if you see holes in the sequence.
I’m scoping out something similar. Would it be feasible to do this without IPFS by having one of the messages be a request for a full state message? As in, I have a CRDT which can be updated either by CvRDT or CmRDT mechanisms, and I want new nodes to be immediately converged to be consistent with an existing node. From there, it can update using ops just like everyone else. Would that work with Gossipsub or do I need some other kind of network behaviour which allows direct messages?
As previously suggested, you may have to create your own protocol on top of libp2p as gossipsub would not sufficient by itself.
Waku is designed to provide protocols that enable message exchange in any environment, including environment where reliability may be an issue (e.g. browser, mobile phone).
For this use case, the Waku Store protocol is designed to allow client to retrieve missed messages from a remote node.
Happy to help if that’s a solution you are keen to explore.