Given that the hosted signaling servers (such as ws-star.discovery.libp2p.io) are not recommended for production apps, what sort of specifications are used/recommended for the signaling servers ?
A few months ago there was a conversation between @vasco-santos, @jacobheun and myself about documenting deploying singaling stars in https://github.com/libp2p/js-libp2p-webrtc-star/issues/238 which was I believe put as a draft in https://github.com/libp2p/js-libp2p/pull/718. I’m unaware of the status, but I know that there’s enough information there to quickly deploy one, and the docker image (libp2p/js-libp2p-webrtc-star) is available for use.
Thanks for pointing this @jjperezaguinaga
I had to switch my priorities, but I hope to get back to these docs. Meanwhile, I think the PR content should help
@jjperezaguinaga Thank you for the resources. I will have to continue to mess with them/go through them again as in my initial tests, despite the computers both being able to visit the website that they set up (at <ip/host>:9090), they were unable to connect. No errors or anything either.
After messing around with it some more, using libp2p-websocket-star-rendezvous seems to have worked.
@ls5 what versions of libp2p are you running? FYI libp2p-websocket-star-rendezvous is deprecated and we would recommend using libp2p-webrtc-star. What is your use case that needs it?
@vasco-santos Ahh sorry, I probably should have mentioned that in my previous post. For Libp2p, I’m currently using 0.26.2 (I believe this is about one year old as that’s when I started this project). A few months back, I did try to upgrade to 0.27.8, but found it to be unstable. Because of this, I was forced to go back to 0.26.2.
As my project currently stands, for better or for worse, it currently treats libp2p as a black box allowing me to send messages between peers. Because of this, I decided to prioritize development of my application/its communication protocol on a version of libp2p which I knew to be stable. And then at some point down the road (probably somewhat soon), I’ll try to update again and see if things are more stable now. To be clear, I haven’t made this project public yet, so I’m not floating around deprecated copies of code.
While I had initially tried using libp2p-webrtc-star, I was having issues dialing my peers with it. However, I do plan to try it again.
To be honest, I am not the most up to date with the change logs at the moment, but I’m happy to discuss this further.
Hey @ls5
Sorry for the late response, but I missed this notification. Would you be able to update? Now with all the improvements that landed until 0.29 and 0.30 nearby I highly recommend it.
If you need any help to updated, let me know and you can also check https://github.com/libp2p/js-libp2p/tree/master/doc/migrations
@vasco-santos I’m trying to find the change logs to see whats new. My main issue is still the reliability of the versions post 0.26.6, and the time it takes to test a different version. I would like to update, and I can try updating soon-ish (possibly part of a bit of a restructure to my code). However, I cannot say for sure if I will be able to–and this really comes down to how stable the new versions are. By chance, do you know if stability issues have been addressed/what the major improvements are?
What are the stability issues you found out at that time? We have done a tone of progress over time and we highly recommend that no one is in versions prior to 0.26. That codebase is way less performant and was entirely rewritten from callbacks+pullstreams into promise+async iterators with a lot of new features. Moreover, we deprecated the connection encryption protocol secio and only noise should be used.
@vasco-santos I found that 0.27.8 had a much lower throughput than 0.26.2. This lower throughput made peers prone to disconnect without warning or error (similar to what would happen if you try to shove too much data through a TCP connection). 0.27.8’s max throughput also fluctuated wildly (which made it impossible to try to keep a reasonable connection with a large bandwidth). In contrast, 0.26.2’s throughput essentially matched the performance of a direct TCP/UDP connection between peers (in my tests, UDP and TCP were performing about the same).
@vasco-santos I’ve been testing out newer versions by following your examples for libp2p with webRTC. For example, https://gist.github.com/vasco-santos/eac2a3d63b1ea87414d27385aa3d0148/revisions. However, I cannot get this to work–I keep getting “Error: no WebRTC support”. Am I missing something?
@vasco-santos I found that 0.27.8 had a much lower throughput than 0.26.2. This lower throughput made peers prone to disconnect without warning or error (similar to what would happen if you try to shove too much data through a TCP connection). 0.27.8’s max throughput also fluctuated wildly (which made it impossible to try to keep a reasonable connection with a large bandwidth). In contrast, 0.26.2’s throughput essentially matched the performance of a direct TCP/UDP connection between peers (in my tests, UDP and TCP were performing about the same).
Can you create an issue in libp2p and provide us the benchmarks you ran? We will have support in Testground for js-libp2p soon and we will be able to start creating better benchmarks simulations.
@vasco-santos I’ve been testing out newer versions by following your examples for libp2p with webRTC. For example, Revisions · libp2p Basic running setup · GitHub. However, I cannot get this to work–I keep getting “Error: no WebRTC support”. Am I missing something?
Let me know the versions of your modules + node/npm versions so that I can see what is going on
I can try to create a version of the benchmark outside of my code, and create an issue. However, if you don’t mind, I’d like to test the newer version first (which I’m currently messing around with) just to make sure that it is truly a libp2p issue.
For context
For context, I first started my project using ipfs, ipfs-pubsub-room, and libp2p. I eventually found that between differences in what ipfs was designed for and what I needed (along with ipfs being able to corrupt files if peers go offline during a sync), that I needed a different solution. Because of this, I ended up building something that fit my needs on top of ipfs-pubsub-room as an easier way of accessing libp2p (and I figured I would switch to libp2p only eventually). During this time, I did look at the pubsub code (in part because I found that, in my case, I could make some tweaks to it to make it more efficient). I want to double check what I saw there to see if switching to libp2p’s streams directly might help (for example, if pubsub room is pushing the whole buffer at once).
This should be my package.json (there might be a few extra items in there from my other tests).
package.json dependencies
“babel-polyfill”: “^6.26.0”,
“express”: “^4.17.1”,
“ffmpeg-static”: “^4.2.7”,
“is-stream”: “^2.0.0”,
“it-concat”: “^1.0.2”,
“it-pipe”: “^1.1.0”,
“libp2p”: “^0.30.0”,
“libp2p-bootstrap”: “^0.12.1”,
“libp2p-mplex”: “^0.10.1”,
“libp2p-noise”: “^2.0.1”,
“libp2p-tcp”: “^0.15.2”,
“libp2p-webrtc-star”: “^0.20.6”,
“libp2p-websockets”: “^0.15.0”,
“parcel-bundler”: “^1.12.4”,
“wrtc”: “^0.4.6”
I found (just so that I could test this), that modifying libp2p-webrtc-start\src\index.js in the following way was able to “fix” it:
const TEST = require('wrtc'); //Added
class WebRTCStar
{
constructor(options = {})
{
//Lines ommitted....
if(options.wrtc) {
this.wrtc = options.wrtc;
}
else //Added
this.wrtc = TEST; //Added
}
}
This next part is a bit off topic, but I figured it would be worthwhile to ask. In libp2p-webrtc-star, I noticed that each connection corresponds to a SimplePeer (libp2p-webrtc-peer). Is it possible to access that SimplePeer/the peer.addStream(… and peer.on(‘stream’… functions?
Ok, just keep me posted. Regarding using libp2p streams directly, it depends on your use case. Pubsub basically allows you do exchange messages according to a topic, while libp2p streams directly would send messages to the other peers connected who have the same protocol stream running.
Oh I see. So, this libp2p node is running in Node.js. When in Node.js, you need to provide the webrtc implementation (in the browser it already exists. This is basically what you need to add to your libp2p configuration: https://github.com/libp2p/js-libp2p/blob/master/doc/CONFIGURATION.md#customizing-transports
We are looking into understand if and how we would support this in [Feature Request] Expose custom transport data streams · Issue #838 · libp2p/js-libp2p · GitHub. I recommend you follow that issue until we decide on how to proceed.
Will do. I’m not sure where’s best to ask this (so feel free to redirect me elsewhere). With the examples I’ve seen, we retreve messages by:
pipe(
stream,
async function (source) {
for await (const msg of source) {
console.log(msg.toString());
}
}
)
However, I’ve noticed that when any decent size of data is sent (I’ve seen it occur at ~3000 characters, but I haven’t extensively tested this), this function hangs permanently in the for await loop (despite console.log showing that the the whole buffer is eventually received). Am I missing something?
Did I miss the documentation for how to do that?
Thank you for this! I’ve temporarily exposed the lower level streams, but I’d be interested to see whats decided on.
This is a bug we need to solve. You can read more about it in Message chunking · Issue #117 · libp2p/js-libp2p-mplex · GitHub . For now you need to chunk the messages yourself to its max length or PR mplex to fix this if you like.
That is not a problem. We know that we need to improve a lot libp2p’s configuration and we are aiming to do that in a few months to improve user experience
Sorry for the slow reply. Thank you for these resources–I’ve managed to make a few tweaks and re-write my program using libp2p streams. Right now, my main issue comes down to message priority/sending two streams at the same time. In my project, I’m having my program listen to two different protocols at times, and using pipe() to send the streams. The only thing is: one of the streams needs to have its data sent immediately (its fairly small chunks of data used for the basics of the program), and I have a second protocol which I’m using to send larger blocks of data. The only thing is, once I start to pipe the large block of data on the second protocol, anything I have to pipe across the first protocol seems to have to wait for the second to finish. I don’t know if there’s some sort of mechanism that I can use for this that already exists, or if I have to find some way of pausing the second pipe.
@vasco-santos Dear Sir I think there is a problem with the command star-signal --port=9090 --host=xxx.xxx.xxx because it works just with http I tried to add --cert --key to convert it to https like I did with the command rendezvous --port=9090 --host=xxx.xxx.xxx --key=“path…privkey.pem” --cert=“path…cert.pem” . Any help or ideas why star-signal is just with http.