IPFS - test with different binaries/versions

[rchaves-veniam]

Hello,

I want to use p2plab to test and compare different versions/branches of IPFS and the underlying projects (like bitswap). I found this repo here, and have been using it and reading the code for a couple of days.

So far I'm able to create scenarios, clusters and run benchmarks, however this is all in the same version of IPFS. I can change the transport protocol (tcp,ws,quic), or to change the commit a peer is using (I was hopeful this was the commit that signalled what IPFS version to use, but it refers to the p2plab commit history).

I was hoping to use this repo to compare IPFS versions, am I missing something here?

Please correct me if this question is in the wrong place, or if I should reach you through another channel (eg: email).

Thank you.

[hinshun]

Hi @rchaves-veniam, thanks for your interest in this project!

The intent is to support that and I'd be very happy to review any PRs for that feature. For now though, you could change the go.mod to point to the components that a specific IPFS daemon consumes in a branch, and then you can point p2plab to that branch.

It is possible to compare IPFS components, but not the official go-ipfs daemons themselves. We'll need two things:

1. The builder can build any arbitrary go binary and ship that to your cluster, but we'll need some changes to make that agnostic to p2plab's repository.
2. p2plab doesn't run against the go-ipfs API directly, there's a small test harness API instead, which is implemented by the peer which is a minimal libp2p peer that can transfer IPLD DAGs. The idea is that we want full access to manipulate the components of this peer for benchmarking purposes. We can write a proxy that fulfills this test harness and calls out to the ipfs HTTP api so we can test both go-ipfs and js-ipfs.

[rchaves-veniam]

Thanks for the quick reply and apologies for my late one. I'll look further into it. As a future note, what is the best way to open a pull request? Fork and merge to this repo, or create a new branch and then merge?

[hinshun]

Fork and merge please 👍

Also, feel free to ask any questions in the thread if you get stuck.

[hinshun]

Hi @rchaves-veniam , you can now upload a binary of labapp to benchmark: #38

$ labctl build upload ./path-to-binary
{
    "ID": "585a9887-d27a-4f7f-9261-1cc40104c62e",
    "Link": "file:///home/edgarl/go/src/github.com/Netflix/p2plab/tmp/labd/uploaders/file/baeabeielxsgtgdggtlt5bld2x3sxxrdtgldyg6ljuxuoo2vvwb2rhev2t4",
    "CreatedAt": "2020-04-10T17:03:46.525812784Z",
    "UpdatedAt": "2020-04-10T17:03:46.525812784Z"
}

$ labctl node update --git-reference 585a9887-d27a-4f7f-9261-1cc40104c62e my-cluster
+----------------------+-----------+--------------------------------------+---------------------------------------------------+---------------+-----------+
|          ID          |  ADDRESS  |             GITREFERENCE             |                      LABELS                       |   CREATEDAT   | UPDATEDAT |
+----------------------+-----------+--------------------------------------+---------------------------------------------------+---------------+-----------+
| bq8adcfic6v8i1squep0 | 127.0.0.1 | 585a9887-d27a-4f7f-9261-1cc40104c62e | bq8adcfic6v8i1squep0,t2.micro,us-west-2           | 7 minutes ago | now       |
| bq8adcnic6v8i1squepg | 127.0.0.1 | 585a9887-d27a-4f7f-9261-1cc40104c62e | bq8adcnic6v8i1squepg,neighbors,t2.micro,us-west-2 | 7 minutes ago | now       |
| bq8adcnic6v8i1squeq0 | 127.0.0.1 | 585a9887-d27a-4f7f-9261-1cc40104c62e | bq8adcnic6v8i1squeq0,neighbors,t2.micro,us-west-2 | 7 minutes ago | now       |
+----------------------+-----------+--------------------------------------+---------------------------------------------------+---------------+-----------+

$ labctl benchmark create my-cluster neighbors
5:10PM INF Retrieving nodes in cluster bid=my-cluster-neighbors-1586538644142581864
5:10PM INF Resolving git references bid=my-cluster-neighbors-1586538644142581864
5:10PM INF Building p2p app(s) bid=my-cluster-neighbors-1586538644142581864 commits=["585a9887-d27a-4f7f-9261-1cc40104c62e"]

This means that you can compile a binary from anywhere as long as it fulfills the labapp API. (A fork of this repo, your local code, etc)

We still need a means of easily creating binaries of official IPFS releases though.

[rchaves-veniam]

Thank you. In the meantime I'm using my own platform with Docker to benchmark everything and run different test cases, but I'll give this a look.