Skip to content

Commit

Permalink
storage: in upsert StateValue allow converting finalized tombstones
Browse files Browse the repository at this point in the history
Before, we were using the wrong assumption that the new feedback UPSERT
operator could not yield finalized tombstone values, because it only
writes finalized values that "exist". But it can happen that
`ensure_decoded` puts in place a finalized tombstone when state values
consolidate to a diff of zero.

We say "can happen" above, because `merge_update` returns whether or not
a merged value can be deleted, and this returns true when the diff_sum
is zero. So in most cases the finalized tombstone would be cleaned up
right away. However, not all callers immediately (or at all) act on this
"can delete" information, which leads to finalized tombstones actually
existing in state.
  • Loading branch information
aljoscha authored and teskje committed Jan 8, 2025
1 parent daa2352 commit c7835c5
Showing 1 changed file with 4 additions and 10 deletions.
14 changes: 4 additions & 10 deletions src/storage/src/upsert/types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -391,16 +391,10 @@ impl<T: Eq, O> StateValue<T, O> {
provisional_value: (Some(provisional_value), provisional_ts, provisional_order),
})
}
StateValue::Value(Value::Tombstone(_)) => {
// This cannot happen with how the new feedback UPSERT uses
// state. There are only ever provisional tombstones, and all
// updates that are merged/consolidated into upsert state come
// from the persist input, which doesn't need tombstones.
//
// Regular, finalized tombstones are only used by the classic
// UPSERT operator when doing partial processing.
panic!("cannot turn a finalized tombstone into a provisional value")
}
StateValue::Value(Value::Tombstone(_)) => StateValue::Value(Value::ProvisionalValue {
finalized_value: None,
provisional_value: (Some(provisional_value), provisional_ts, provisional_order),
}),
StateValue::Value(Value::ProvisionalValue {
finalized_value,
provisional_value: _,
Expand Down

0 comments on commit c7835c5

Please sign in to comment.