Implementing custom protocols with rust-libp2p

Users and Developers rust
1

I am trying to develop a custom protocol implementation in rust-libp2p, and I’m finding it to be a hard slog. There are large complex traits, NetworkBehaviour and ProtocolsHandler, which force the implementations to be stateful and pollable. I have looked at actively maitained codebases that are using libp2p::swarm, such as libp2p’s built-in protocol implementations, substrate, and lighthouse, and most of them seem to do similar things with quite a lot of boilerplate code that does not look very composable or reusable.

Here are some of the specific issues I have been grappling with:

  • There is a NetworkBehaviour proc macro to help implement aggregate behaviours. But it’s insufficiently well documented, especially the customizations with #[behaviour(...)], and the generated associated types are not straightforward: the protocol handler and its InEvent, the Error type, become unwieldy aggregates with no clearly specified way to use them explicitly or access the components.

  • The pattern is to have non-async methods injecting events into the stateful object, and a polling method to make progress with possible use of async APIs. This means every implementation has to maintain some sort of state to store events for subsequent processing in the poll, but there is currently little in the way of implementation utilities to help organize this, outside of some limited use cases like RequestResponse or OneShotHandler.

  • There does not seem to be much support for stream-oriented protocols. The ProtocolsHandler API relegates substream management to the implementation, and seems from the code that the keep-alive management has to keep track of the substreams that are in use by the application. It would be more convenient if the connection would be automatically kept alive as long as there are substreams, even if ProtocolsHandler::connection_keep_alive returns KeepAlive::No, or perhaps a newly introduced KeepAlive::Substreams case value.

  • The poll methods seem to be an odd fit for what many behaviour/handler implementations really do. In most of the protocol implementations, Poll::Pending is returned when the implementation has nothing to do, without scheduling a wakeup through the Context. This means that the caller of poll needs to back up the Pending case with polling another source that uses the waker correctly, in order to avoid task hangups. This may be what libp2p does internally, but the break with the usual polling convention feels troublesome: is there a reliable way to know when to re-poll a behaviour or a protocol handler that does not arrange wakeups by itself, outside of some dumb periodic polling?

  • Some of the types (e.g. errors) are heavy with generic parameters that make their usage cumbersome.

Are there any ideas or plans afoot to make this part of libp2p easier to use?