Merge branch 'master' into blob-verifier-tweak

api/docs/disperser.html

@@ -1009,7 +1009,7 @@ <h3 id="disperser.Disperser">Disperser</h3>
                 <td><a href="#disperser.AuthenticatedRequest">AuthenticatedRequest</a> stream</td>
                 <td><a href="#disperser.AuthenticatedReply">AuthenticatedReply</a> stream</td>
                 <td><p>DisperseBlobAuthenticated is similar to DisperseBlob, except that it requires the
-client to authenticate itself via the AuthenticationData message. The protoco is as follows:
+client to authenticate itself via the AuthenticationData message. The protocol is as follows:
 1. The client sends a DisperseBlobAuthenticated request with the DisperseBlobRequest message
 2. The Disperser sends back a BlobAuthHeader message containing information for the client to
    verify and sign.

api/docs/disperser.md

@@ -369,7 +369,7 @@ Disperser defines the public APIs for dispersing blobs.
 | DisperseBlob | [DisperseBlobRequest](#disperser-DisperseBlobRequest) | [DisperseBlobReply](#disperser-DisperseBlobReply) | DisperseBlob accepts a single blob to be dispersed. This executes the dispersal async, i.e. it returns once the request is accepted. The client should use GetBlobStatus() API to poll the processing status of the blob.
 
 If DisperseBlob returns the following error codes: INVALID_ARGUMENT (400): request is invalid for a reason specified in the error msg. RESOURCE_EXHAUSTED (429): request is rate limited for the quorum specified in the error msg. user should retry after the specified duration. INTERNAL (500): serious error, user should NOT retry. |
-| DisperseBlobAuthenticated | [AuthenticatedRequest](#disperser-AuthenticatedRequest) stream | [AuthenticatedReply](#disperser-AuthenticatedReply) stream | DisperseBlobAuthenticated is similar to DisperseBlob, except that it requires the client to authenticate itself via the AuthenticationData message. The protoco is as follows: 1. The client sends a DisperseBlobAuthenticated request with the DisperseBlobRequest message 2. The Disperser sends back a BlobAuthHeader message containing information for the client to verify and sign. 3. The client verifies the BlobAuthHeader and sends back the signed BlobAuthHeader in an 	 AuthenticationData message. 4. The Disperser verifies the signature and returns a DisperseBlobReply message. |
+| DisperseBlobAuthenticated | [AuthenticatedRequest](#disperser-AuthenticatedRequest) stream | [AuthenticatedReply](#disperser-AuthenticatedReply) stream | DisperseBlobAuthenticated is similar to DisperseBlob, except that it requires the client to authenticate itself via the AuthenticationData message. The protocol is as follows: 1. The client sends a DisperseBlobAuthenticated request with the DisperseBlobRequest message 2. The Disperser sends back a BlobAuthHeader message containing information for the client to verify and sign. 3. The client verifies the BlobAuthHeader and sends back the signed BlobAuthHeader in an 	 AuthenticationData message. 4. The Disperser verifies the signature and returns a DisperseBlobReply message. |
 | GetBlobStatus | [BlobStatusRequest](#disperser-BlobStatusRequest) | [BlobStatusReply](#disperser-BlobStatusReply) | This API is meant to be polled for the blob status. |
 | RetrieveBlob | [RetrieveBlobRequest](#disperser-RetrieveBlobRequest) | [RetrieveBlobReply](#disperser-RetrieveBlobReply) | This retrieves the requested blob from the Disperser's backend. This is a more efficient way to retrieve blobs than directly retrieving from the DA Nodes (see detail about this approach in api/proto/retriever/retriever.proto). The blob should have been initially dispersed via this Disperser service for this API to work. |
 

api/docs/eigenda-protos.html

@@ -1891,7 +1891,7 @@ <h3 id="disperser.Disperser">Disperser</h3>
                 <td><a href="#disperser.AuthenticatedRequest">AuthenticatedRequest</a> stream</td>
                 <td><a href="#disperser.AuthenticatedReply">AuthenticatedReply</a> stream</td>
                 <td><p>DisperseBlobAuthenticated is similar to DisperseBlob, except that it requires the
-client to authenticate itself via the AuthenticationData message. The protoco is as follows:
+client to authenticate itself via the AuthenticationData message. The protocol is as follows:
 1. The client sends a DisperseBlobAuthenticated request with the DisperseBlobRequest message
 2. The Disperser sends back a BlobAuthHeader message containing information for the client to
    verify and sign.

api/docs/eigenda-protos.md

@@ -721,7 +721,7 @@ Disperser defines the public APIs for dispersing blobs.
 | DisperseBlob | [DisperseBlobRequest](#disperser-DisperseBlobRequest) | [DisperseBlobReply](#disperser-DisperseBlobReply) | DisperseBlob accepts a single blob to be dispersed. This executes the dispersal async, i.e. it returns once the request is accepted. The client should use GetBlobStatus() API to poll the processing status of the blob.
 
 If DisperseBlob returns the following error codes: INVALID_ARGUMENT (400): request is invalid for a reason specified in the error msg. RESOURCE_EXHAUSTED (429): request is rate limited for the quorum specified in the error msg. user should retry after the specified duration. INTERNAL (500): serious error, user should NOT retry. |
-| DisperseBlobAuthenticated | [AuthenticatedRequest](#disperser-AuthenticatedRequest) stream | [AuthenticatedReply](#disperser-AuthenticatedReply) stream | DisperseBlobAuthenticated is similar to DisperseBlob, except that it requires the client to authenticate itself via the AuthenticationData message. The protoco is as follows: 1. The client sends a DisperseBlobAuthenticated request with the DisperseBlobRequest message 2. The Disperser sends back a BlobAuthHeader message containing information for the client to verify and sign. 3. The client verifies the BlobAuthHeader and sends back the signed BlobAuthHeader in an 	 AuthenticationData message. 4. The Disperser verifies the signature and returns a DisperseBlobReply message. |
+| DisperseBlobAuthenticated | [AuthenticatedRequest](#disperser-AuthenticatedRequest) stream | [AuthenticatedReply](#disperser-AuthenticatedReply) stream | DisperseBlobAuthenticated is similar to DisperseBlob, except that it requires the client to authenticate itself via the AuthenticationData message. The protocol is as follows: 1. The client sends a DisperseBlobAuthenticated request with the DisperseBlobRequest message 2. The Disperser sends back a BlobAuthHeader message containing information for the client to verify and sign. 3. The client verifies the BlobAuthHeader and sends back the signed BlobAuthHeader in an 	 AuthenticationData message. 4. The Disperser verifies the signature and returns a DisperseBlobReply message. |
 | GetBlobStatus | [BlobStatusRequest](#disperser-BlobStatusRequest) | [BlobStatusReply](#disperser-BlobStatusReply) | This API is meant to be polled for the blob status. |
 | RetrieveBlob | [RetrieveBlobRequest](#disperser-RetrieveBlobRequest) | [RetrieveBlobReply](#disperser-RetrieveBlobReply) | This retrieves the requested blob from the Disperser's backend. This is a more efficient way to retrieve blobs than directly retrieving from the DA Nodes (see detail about this approach in api/proto/retriever/retriever.proto). The blob should have been initially dispersed via this Disperser service for this API to work. |
 

api/proto/disperser/disperser.proto

@@ -18,7 +18,7 @@ service Disperser {
 	rpc DisperseBlob(DisperseBlobRequest) returns (DisperseBlobReply) {}
 
 	// DisperseBlobAuthenticated is similar to DisperseBlob, except that it requires the
-	// client to authenticate itself via the AuthenticationData message. The protoco is as follows:
+	// client to authenticate itself via the AuthenticationData message. The protocol is as follows:
 	// 1. The client sends a DisperseBlobAuthenticated request with the DisperseBlobRequest message
 	// 2. The Disperser sends back a BlobAuthHeader message containing information for the client to
 	//    verify and sign.

core/eth/reader.go

@@ -786,7 +786,7 @@ func (t *Reader) GetGlobalSymbolsPerSecond(ctx context.Context) (uint64, error)
 	return globalSymbolsPerSecond.Uint64(), nil
 }
 
-func (t *Reader) GetGlobalRateBinInterval(ctx context.Context) (uint64, error) {
+func (t *Reader) GetGlobalRateBinInterval(ctx context.Context) (uint32, error) {
 	if t.bindings.PaymentVault == nil {
 		return 0, errors.New("payment vault not deployed")
 	}
@@ -796,7 +796,7 @@ func (t *Reader) GetGlobalRateBinInterval(ctx context.Context) (uint64, error) {
 	if err != nil {
 		return 0, err
 	}
-	return globalRateBinInterval.Uint64(), nil
+	return uint32(globalRateBinInterval.Uint64()), nil
 }
 
 func (t *Reader) GetMinNumSymbols(ctx context.Context) (uint32, error) {

core/meterer/meterer.go

@@ -259,23 +259,11 @@ func (m *Meterer) SymbolsCharged(numSymbols uint) uint32 {
 	return uint32(core.RoundUpDivide(uint(numSymbols), uint(m.ChainPaymentState.GetMinNumSymbols()))) * m.ChainPaymentState.GetMinNumSymbols()
 }
 
-// ValidateReservationPeriod checks if the provided bin index is valid
-func (m *Meterer) ValidateGlobalReservationPeriod(header core.PaymentMetadata) (uint32, error) {
-	// Deterministic function: local clock -> index (1second intervals)
-	currentReservationPeriod := uint32(time.Now().Unix())
-
-	// Valid bin indexes are either the current bin or the previous bin (allow this second or prev sec)
-	if header.ReservationPeriod != currentReservationPeriod && header.ReservationPeriod != (currentReservationPeriod-1) {
-		return 0, fmt.Errorf("invalid bin index for on-demand request")
-	}
-	return currentReservationPeriod, nil
-}
-
 // IncrementBinUsage increments the bin usage atomically and checks for overflow
 func (m *Meterer) IncrementGlobalBinUsage(ctx context.Context, symbolsCharged uint64) error {
-	//TODO: edit globalIndex based on bin interval in a subsequent PR
-	globalIndex := uint64(time.Now().Unix())
-	newUsage, err := m.OffchainStore.UpdateGlobalBin(ctx, globalIndex, symbolsCharged)
+	globalPeriod := GetReservationPeriod(uint64(time.Now().Unix()), m.ChainPaymentState.GetGlobalRateBinInterval())
+
+	newUsage, err := m.OffchainStore.UpdateGlobalBin(ctx, globalPeriod, symbolsCharged)
 	if err != nil {
 		return fmt.Errorf("failed to increment global bin usage: %w", err)
 	}

core/meterer/meterer_test.go

@@ -171,7 +171,7 @@ func TestMetererReservations(t *testing.T) {
 	ctx := context.Background()
 	paymentChainState.On("GetReservationWindow", testifymock.Anything).Return(uint32(1), nil)
 	paymentChainState.On("GetGlobalSymbolsPerSecond", testifymock.Anything).Return(uint64(1009), nil)
-	paymentChainState.On("GetGlobalRateBinInterval", testifymock.Anything).Return(uint64(1), nil)
+	paymentChainState.On("GetGlobalRateBinInterval", testifymock.Anything).Return(uint32(1), nil)
 	paymentChainState.On("GetMinNumSymbols", testifymock.Anything).Return(uint32(3), nil)
 
 	reservationPeriod := meterer.GetReservationPeriod(uint64(time.Now().Unix()), mt.ChainPaymentState.GetReservationWindow())

core/meterer/offchain_store.go

@@ -93,9 +93,9 @@ func (s *OffchainStore) UpdateReservationBin(ctx context.Context, accountID stri
 	return binUsageValue, nil
 }
 
-func (s *OffchainStore) UpdateGlobalBin(ctx context.Context, reservationPeriod uint64, size uint64) (uint64, error) {
+func (s *OffchainStore) UpdateGlobalBin(ctx context.Context, reservationPeriod uint32, size uint64) (uint64, error) {
 	key := map[string]types.AttributeValue{
-		"ReservationPeriod": &types.AttributeValueMemberN{Value: strconv.FormatUint(reservationPeriod, 10)},
+		"ReservationPeriod": &types.AttributeValueMemberN{Value: strconv.FormatUint(uint64(reservationPeriod), 10)},
 	}
 
 	res, err := s.dynamoClient.IncrementBy(ctx, s.globalBinTableName, key, "BinUsage", size)

core/meterer/onchain_state.go

@@ -20,7 +20,7 @@ type OnchainPayment interface {
 	GetOnDemandPaymentByAccount(ctx context.Context, accountID gethcommon.Address) (*core.OnDemandPayment, error)
 	GetOnDemandQuorumNumbers(ctx context.Context) ([]uint8, error)
 	GetGlobalSymbolsPerSecond() uint64
-	GetGlobalRateBinInterval() uint64
+	GetGlobalRateBinInterval() uint32
 	GetMinNumSymbols() uint32
 	GetPricePerSymbol() uint32
 	GetReservationWindow() uint32
@@ -42,7 +42,7 @@ type OnchainPaymentState struct {
 
 type PaymentVaultParams struct {
 	GlobalSymbolsPerSecond uint64
-	GlobalRateBinInterval  uint64
+	GlobalRateBinInterval  uint32
 	MinNumSymbols          uint32
 	PricePerSymbol         uint32
 	ReservationWindow      uint32
@@ -211,7 +211,7 @@ func (pcs *OnchainPaymentState) GetGlobalSymbolsPerSecond() uint64 {
 	return pcs.PaymentVaultParams.Load().GlobalSymbolsPerSecond
 }
 
-func (pcs *OnchainPaymentState) GetGlobalRateBinInterval() uint64 {
+func (pcs *OnchainPaymentState) GetGlobalRateBinInterval() uint32 {
 	return pcs.PaymentVaultParams.Load().GlobalRateBinInterval
 }
 

core/mock/payment_state.go

@@ -62,9 +62,9 @@ func (m *MockOnchainPaymentState) GetGlobalSymbolsPerSecond() uint64 {
 	return args.Get(0).(uint64)
 }
 
-func (m *MockOnchainPaymentState) GetGlobalRateBinInterval() uint64 {
+func (m *MockOnchainPaymentState) GetGlobalRateBinInterval() uint32 {
 	args := m.Called()
-	return args.Get(0).(uint64)
+	return args.Get(0).(uint32)
 }
 
 func (m *MockOnchainPaymentState) GetMinNumSymbols() uint32 {

disperser/apiserver/server_v2_test.go

@@ -443,6 +443,7 @@ func newTestServerV2(t *testing.T) *testComponents {
 	mockState.On("GetReservationWindow", tmock.Anything).Return(uint32(1), nil)
 	mockState.On("GetPricePerSymbol", tmock.Anything).Return(uint32(2), nil)
 	mockState.On("GetGlobalSymbolsPerSecond", tmock.Anything).Return(uint64(1009), nil)
+	mockState.On("GetGlobalRateBinInterval", tmock.Anything).Return(uint32(1), nil)
 	mockState.On("GetMinNumSymbols", tmock.Anything).Return(uint32(3), nil)
 
 	now := uint64(time.Now().Unix())

disperser/dataapi/server_test.go

@@ -1240,7 +1240,7 @@ func TestFetchDeregisteredOperatorOnline(t *testing.T) {
 }
 
 func TestFetchDeregisteredOperatorsMultipleOfflineOnline(t *testing.T) {
-	// Skipping this test as repported being flaky but could not reproduce it locally
+	// Skipping this test as reported being flaky but could not reproduce it locally
 	t.Skip("Skipping testing in CI environment")
 
 	r := setUpRouter()

test/synthetic-test/synthetic_client_test.go

@@ -297,7 +297,7 @@ func TestDisperseBlobEndToEnd(t *testing.T) {
 	// For now log....later we can define a baseline value for this
 	logger.Printf("Time to Disperse Blob %s", disperseBlobStopTime.String())
 
-	// Set Confirmation DeaLine For Confirmation of Dispersed Blob
+	// Set Confirmation Deadline For Confirmation of Dispersed Blob
 	// Update this to a minute over Batcher_Pull_Interval
 	confirmationDeadline, err := time.ParseDuration(testSuite.BatcherPullInterval)