Expectable IO Performance

[tonobo]

Howdy guys,

just got a question regarding the possible performance. I tried to read from a an RBD, and print them out over a common io.Pipe, but according to my tests it's much slower compared to rbd export. I tried using io.Copy with different buffers (e.g aligned to stripe size), but still far from reaching the rbd command. While digging into profile dumps, is mostly spend time in external c librarys and doing syscalls. Is it theoretically possible to reach the native rbd read/write performance, or is it the cgo tradeoff i have to deal with?

[phlogistonjohn]

Theoretically, the rbd export command should be making use of the same core libraries as go-ceph. But in our case there's the translation layers of C++ to C to Go and back while rbd export can make use of the core C++ apis directly. So I would expect some overhead. The question is how much and what can we do about it.

Would you be able to share some of the numbers with us? Including what library and system calls we do in excess of the rbd tooling? Thanks.