You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There is a fair amount of duplication between RunBuilder and WriteBufferWriter as well as RunReader and WriteBufferReader. It would be good for maintainability to extract the shared functionality.
Regarding deduplication, there are two avenues I can think of.
Deduplicating WriteBufferReader and RunReader. And my thinking is that we could create a more general KOpsReader type that can be instantiated to be either a WriteBufferReader or RunReader. We should probably do this first, because it requires fewer changes than the next avenue
Deduplicating WriteBufferWriter and RunBuilder. My thinking is that we could construct this hierarchy:
RunAcc is split up into KOpsAcc and ProbeAcc. The former takes care of building pages for key/ops and blob files. The latter takes care of building the bloom filters and fence pointer indices. RunBuilder is split in two in the same way, doing the IO for the Accumulators. The new RunBuilder is the composition of a KOPsBuilder and a ProbeBuilder. The WriteBufferBuilder is just a small wrapper around a KOpsBuilder
The text was updated successfully, but these errors were encountered:
I've been prototyping my suggestion a little bit today, and I'm not so sure it's going to work out nicely. It seems that modularising the components in the way I suggested above adds quite a bit of overhead in terms of lines of code, cognitive overhead, and performance. This overhead is arguably more sizeable than maintaining a separate write buffer reader/writers and run reader/writers. Moreover, having two separate code paths means that we can optimise better for the specific use cases
I do think we should still look into deduplicating the bits that we can, but the component refactoring I suggested above should probably be scrapped
There is a fair amount of duplication between RunBuilder and WriteBufferWriter as well as RunReader and WriteBufferReader. It would be good for maintainability to extract the shared functionality.
@jorisdral wrote in #478:
The text was updated successfully, but these errors were encountered: