How to send a file from A to B?

[EricR]

I have a small network, including bootstrap nodes created with go-libp2p, and a pubsub is setup and working. I can send small messages to other peers by making them subscribe to topics with their own peer ID. This works in my implementation. As I understand, transferring whole files in this way would be very inefficient, though. If I understood this correctly, I also cannot assume that all peers in the network will be connected - when the network grows larger, messages will be routed without direct connection. Right?

So, suppose A knows B’s peer ID, i.e., what host.ID() returns, and vice versa. Could someone explain to me how I would exchange files between A and B, e.g. for A to send a file to B? How do I make A connect to B? How do I send the file from A to B as efficiently as the network allows? Should I exchange more information via the pubsub first? Which file transport method would you recommend? If I have a direct stream connection like in the Ping example, does this mean I can send arbitrary long data over the stream (e.g. 5 GB)? Or is it better to chunk the data?

Lots of questions. I’d be happy about any hints, advice, or additional tutorials.

[EricR]

Since I haven’t gotten a reply in 11 days I suppose nobody here knows how to transmit a file from one peer to another or can give advice on how to best achieve this functionality using libp2p. Before I implement this functionality on top of the stream handler myself, I have to ask: Is file transfer kind of deprecated or unwanted at this time when using libp2p, e.g. for network performance reasons or because existing servers would be overused?

I don’t want to implement something that is undesirable in the long run (similar to e.g. how torrenting over Tor is deprecated by the Tor team).

[technicallyty]

I have actually began trying to make something quite similar. I’m using js-libp2p. I would take a look at this: https://github.com/libp2p/js-libp2p/blob/master/examples/chat/src/stream.js

This is their chat example, but I think we just need to change the content of what is being sent. At the end of the day it’s just a stream of bytes. In the example it’s just text, but surely we can hack it out a bit and get it to send arbitrary files.

As for the proper transport, TCP is probably fine. There are lots of examples already online (not libp2p) of general file transfers using TCP.

[alexsunny123]

@technicallyty

This is their chat example, but I think we just need to change the content of what is being sent. At the end of the day it’s just a stream of bytes. In the example it’s just text, but surely we can hack it out a bit and get it to send arbitrary files.

As for the proper transport, TCP is probably fine. There are lots of examples already online (not libp2p) of general file transfers using TCP.

thanks for the awesome information.

[alexsunny123]

@alexsunny123

@technicallyty

This is their chat example, but I think we just need to change the content of what is being sent. At the end of the day it’s just a stream of bytes. In the example it’s just text, but surely we can hack it out a bit and get it to send arbitrary files.

As for the proper transport, TCP is probably fine. There are lots of examples already online (not libp2p) of general file transfers using TCP.

https://ometv.onl https://chatroulette.top
thanks for the awesome information.

thanks my issue has been fixed.

[fungs]

Hi, is there any outcome so far? I’d be in particular interested in establishing a tiny ad-hoc network and send large data from A to B as a raw file stream. However, I’m totally unfamiliar with libp2p. Would that be inefficient or unintended in some way?

[zeisler]

We are using libp2p at Files.com for a on-perm file agent. It’s very good at file transfers. It’s faster than most solutions for what enterprises are using for this, which is typically sftp/ftp. We are using quic between the agent and the relay and then standard http to download files on the the end client.