I had need for distributed state over Libp2p but I could not find anything that handles this satisfactorily, so I created a a system for synchronizing CRDTs through Libp2p. This system is a little bit similar to OrbitDB but is simpler and runs on Libp2p without the need for IPFS.
The system is only designed for use in a private network where you can trust all the other nodes to follow the same protocol and trust all the nodes with read/write access. Let me know if you have any suggestions what can be done to provide a permission system so trust between the nodes is limited.
The libp2p-crdt-synchronizer module can synchronize any CRDT following the CRDT
interface from the crdt-interfaces package. The CRDT
interface defines a sync
method which defines the synchronization protocol between instances. The interface also allows for additional methods like serialize
(to allow saving state) and addBroadcaster
(to allow faster synchronization over Libp2p pubsub).
Another thing to note is the same CRDT type can have different protocols for sync, currently they are required to be treated as separate CRDTs (in regards to synchronization) but perhaps there is a way to refactor the synchronization protocols so that multiple can be added to a single CRDT implementation.
I have created a bunch of basic CRDT implementations in the crdts package that will work with libp2p-crdt-synchronizer module, making it easy to get started with it. Most of these implementations use CBOR encoding so they work with a variety of data types, if you want to create a version of one of these CRDTs to use a specific data type you might want to create protobuf definitions for each type which would clearly specify how it’s protocol works, I have done this in crdt-protocols but have not implemented any of them due to the work needed to get each type working.
The following code should help you to get started quickly:
import { createCRDTSynchronizer } from "@organicdesign/libp2p-crdt-synchronizer";
import { createGCounter } from "@organicdesign/crdts";
const synchronizer = createCRDTSynchronizer()(libp2p);
synchronizer.start();
const counter = createGCounter({ id: libp2p.peerId.toBytes() });
synchronizer.setCRDT("my-counter", counter);
If you give this a try, please let me know how it goes, there is a lot of room for improvement in this system and I am hoping some of you would have suggestions on how it could be done better.
Here is a list of the npm modules that make up this system:
- @organicdesign/libp2p-crdt-synchronizer The Libp2p CRDT synchronizer module.
- @organicdesign/crdts A group of basic CRDT implementations.
- @organicdesign/crdt-interfaces Interfaces for creating compatible CRDTs.
-
@organicdesign/crdt-tests Basic tests for implementations of CRDTs following the
crdt-interfaces
interfaces. - @organicdesign/crdt-protocols JS implementations of crdt-protocols.