Skip to content

Commit

Permalink
Fixed DefaultCompactionPlanner.findFilesToCompactWithLowerRatio logic
Browse files Browse the repository at this point in the history
Fixed logic to break out of loop when a set of files is found
that meets the criteria. Loop was continuing and ended up
using the lowest possible ratio above 1.1.
  • Loading branch information
dlmarion committed Jan 16, 2025
1 parent d758759 commit 0de2a93
Showing 1 changed file with 12 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -382,8 +382,13 @@ private Collection<CompactableFile> findFilesToCompactWithLowerRatio(PlanningPar
goalCompactionSize = 0;
}

// If the highRatio is 1.1 or less, then we will never get
// into the loop below.
boolean checkPerformed = false;

// Do a binary search of the compaction ratios.
while (highRatio - lowRatio > .1) {
checkPerformed = true;
double ratioToCheck = (highRatio - lowRatio) / 2 + lowRatio;

// This is continually resorting the list of files in the following call, could optimize this
Expand All @@ -399,12 +404,17 @@ private Collection<CompactableFile> findFilesToCompactWithLowerRatio(PlanningPar
lowRatio = ratioToCheck;
found = filesToCompact;
}

if (!found.isEmpty()) {
break;
}

}

if (found.isEmpty() && lowRatio == 1.0) {
if (checkPerformed && found.isEmpty() && lowRatio == 1.0) {
// in this case the data must be really skewed, operator intervention may be needed.
log.warn(
"Attempted to lower compaction ration from {} to {} for {} because there are {} files "
"Attempted to lower compaction ratio from {} to {} for {} because there are {} files "
+ "and the max tablet files is {}, however no set of files to compact were found.",
params.getRatio(), highRatio, params.getTableId(), params.getCandidates().size(),
maxTabletFiles);
Expand Down

0 comments on commit 0de2a93

Please sign in to comment.