You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Assuming most objects at least implement the io.ReadWriteCloser interface
And such objects can be either duplicated, distributed, or act as performance helpers
A Balancer object can be seen as an abstraction to implement a Strategy to select which objects are selected to proxy a Workload to.
USE CASE:
Retrieving data from a Data Lake built on S3 compatible storage is fast
Retrieving data from an in-memory data store (essentially a cache) is even faster
Both stores are using a Prefix based approach to provide key/value storage (they are 1:1 compatible)
An object implementing the Balancer interface would be passed a Strategy called FirstToFinish (or something like that)
FirstToFinish calls both stores, the S3 compatible and the in-memory store
Which ever store returns the data first will cancel out the request to the other one
This will always provide the quickest path to results, while also having a fallback to the slower path
PRACTICAL ABSTRACT EXAMPLE:
package main
/*S3Store is a client to a data lake built on AWS S3 (or compatible) storage.*/typeS3Storestruct {
io.ReadWriteCloser
}
// ... Read, Write, Close methods omitted./*RadixStore is an in-memory Radix Trie.*/typeRadixStorestruct {
io.ReadWriteCloser
}
// ... Read, Write, Close methods omitted./*Balancer is an interface to implement if an object should be able to balance a Workload to two or more objects implementing the io.ReadWriteCloser interface. Itself it also implements the io.ReadWriteCloser to make the usage of the interface type transparent to the direct usage of the Store types.*/typeBalancerinterface {
io.ReadWriteCloser
}
/*NewBalancer takes in a compatible struct type as a balancer Strategy and converts it to a Balancer interface type.*/funcNewBalancer(balancerStrategyBalancer) Balancer {
returnbalancerStrategy
}
/*ProtoBalancer serves as the most common implementation of a Strategy acting as a Balancer type.It just uses a random object from the pool.*/typeProtoBalancerstruct {
pool []io.ReadWriteCloserselector*rand.Rand
}
/*NewProtoBalancer constructs a Balancer Strategy that uses random pool selection.*/funcNewProtoBalancer(pool []io.ReadWriteCloser) ProtoBalancer {
returnProtoBalancer{
pool: pool,
selector: rand.New(rand.NewSource(time.Now().Unix())),
}
}
/*Close the objects in the Balancer pool and cleanup.*/func (protoProtoBalancer) Close() error {
for_, p:=rangeproto.pool {
err:=p.Close()
errnie.Handles(err)
}
}
/*Read selects a random object from the pool and proxies the workload.*/func (protoProtoBalancer) Read(p []byte) (nint, errerror) {
returnproto.pool[proto.selector.Intn(len(proto.pool))].Read(p)
}
/*Write selects a random object from the pool and proxies the workload.*/func (protoProtoBalancer) Write(p []byte) (nint, errerror) {
returnproto.pool[proto.selector.Intn(len(proto.pool))].Write(p)
}
funcmain() {
// Instantiate a new balancer type.balancer:=NewBalancer(NewProtoBalancer(S3Store{}, RadixStore{}))
// Make some test data.buf:= []byte("some test data")
// User io.Copy(dst, src) to copy the data to the balancer, which will take care of the rest.io.Copy(balancer, buf)
}
The text was updated successfully, but these errors were encountered:
Assuming most objects at least implement the io.ReadWriteCloser interface
And such objects can be either duplicated, distributed, or act as performance helpers
A Balancer object can be seen as an abstraction to implement a Strategy to select which objects are selected to proxy a Workload to.
USE CASE:
Retrieving data from a Data Lake built on S3 compatible storage is fast
Retrieving data from an in-memory data store (essentially a cache) is even faster
Both stores are using a Prefix based approach to provide key/value storage (they are 1:1 compatible)
An object implementing the Balancer interface would be passed a Strategy called FirstToFinish (or something like that)
FirstToFinish calls both stores, the S3 compatible and the in-memory store
Which ever store returns the data first will cancel out the request to the other one
This will always provide the quickest path to results, while also having a fallback to the slower path
PRACTICAL ABSTRACT EXAMPLE:
The text was updated successfully, but these errors were encountered: