Logging capabilities and parsing engines for libp2p nodes

Users and Developers
1

There seems to had been multiple efforts for detecting and parsing libp2p nodes activity.

Moving forward, what would be the best future-compatible way to,

  • Process nodes information (e.g. DEBUG=libp2p* flag)
  • Parse nodes information (AFAIK there’s no standard format)
  • Consume this information (e.g. web server via REST/gRPC).

As I’m looking to put together something to visualise an existing network that uses libp2p, it would be great that any effort in that direction can be used by the community moving forward and do not get stuck in a single GitHub repository.

1 Like
2

The long term goal here is to standardize around the work on Phantom Drift, which is currently set up to be a pull based websocket endpoint. This will enable us to create observers for logging, or live for observations. Most of the work we’ve been doing on the JS side of things has been to improve our ability to collect the data needed for this, so we haven’t started on the endpoint itself.

I’m not sure what the state of the specification is for the data stream for this, I’ll sync up with @raul and @vasco-santos on this. Ideally we’d be able to get at least a draft of that out so that the community can help with integrating that into the various implementations.

1 Like
3

https://github.com/libp2p/observer-toolkit has the current working spec. The protocol is described in https://github.com/libp2p/observer-toolkit/blob/7cd2e46/docs/introspection-data-emitting-protocol.md and the protobuf is described in https://github.com/libp2p/observer-toolkit/blob/e063228/packages/proto/lib/introspection.proto. The work is pretty close, we just had some competing priorities come up that have temporarily paused completing the Go efforts.

From here we should get the specification moved to the libp2p/specs repo and get that merged as a Working Draft. It should be in a state that work can proceed on other implementations.

Any efforts for introspection should really be focused on pushing this work forward. We should then be able to leverage the existing UI that has been created for the initial Widgets to create additional Widgets/components to expand on the visualizations.

4

Perfect! That’s incredible useful. My guess was that indeed considering there’s already a spec and already some existing protobufs it would be better to push in that direction rather than having to implement another format.

Let me know if we can help in any way to push the introspection specs as a working draft somehow.