Adjust Statistics::total_byte_size
in Filter
with a projection
#13224
Labels
enhancement
New feature or request
Is your feature request related to a problem or challenge?
@Dandandan pointed out in https://github.com/apache/datafusion/pull/13187/files#r1824330274 that when applying a projection to a filter the resulting
StatisticsCalculation
is not properly updatedHowever, I did not want to try and add that in the bugfix PR #13187 because:
total_byte_size
calculation in filter also needs to take estimated selectivity into accounttotal_byte_size
instats_projection
is also somewhat suspect as it only accounts for "fixed sized" rows but still claims the size is known preciselyDescribe the solution you'd like
Account for the projection in the filter statistics calculation somehow
Describe alternatives you've considered
https://github.com/apache/datafusion/blob/ac79ef3442e65f6197c7234da9fad964895b9101/datafusion/physical-plan/src/projection.rs#L261-L260
Additional context
No response
The text was updated successfully, but these errors were encountered: