Sloppy hashing and self-organizing clusters

Figured I’d drop in the establishing paper for CoralCDN, Sloppy hashing and self-organizing clusters. Coral sought to optimize DHTs by helping peers discover other peers in their network proximity. Furthermore, by introducing a notion of “sloppy hashing” and capacity for peers, Coral distributed loads more evenly across a cluster by leaving a breadcrumb trail of copies of a piece of data along the lookup path it used to find a peer closest to a piece of data.

We hope to integrate Coral’s clusters as one instance of what I’m currently thinking of as “DHT overlay networks”, a mechanism through which peers can opt in to specific subsets of the data being stored in our universal DHT.

I noticed @jhiesey had a note about Coral in the new DHT proposal (discussion here).

@bigs I’m curious to hear more about the overlay networks - is this a kind of namespace within the global DHT that peers can opt into? Possibly clustered together for efficiency using Coral?

Generally, yes, it’s namespacing! The hope is that Coral “clusters” could just be namespaces with semantics attached, but a more general overlay system wouldn’t have to know about those semantics—just the Coral implementation.