Skip to content

Commit

Permalink
fixes intermittent failure in ScanConsistencyIT (#4292)
Browse files Browse the repository at this point in the history
The following failure was observed when running ScanConsistencyIT in the
elasticity branch.

```
java.util.concurrent.ExecutionException: java.util.NoSuchElementException
	at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
	at org.apache.accumulo.test.ScanConsistencyIT.testConcurrentScanConsistency(ScanConsistencyIT.java:186)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.util.NoSuchElementException
	at com.google.common.collect.MoreCollectors$ToOptionalState.getElement(MoreCollectors.java:163)
	at com.google.common.collect.MoreCollectors.lambda$static$1(MoreCollectors.java:75)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:582)
	at org.apache.accumulo.test.ScanConsistencyIT$TableOpsTask.call(ScanConsistencyIT.java:685)
	at org.apache.accumulo.test.ScanConsistencyIT$TableOpsTask.call(ScanConsistencyIT.java:622)
	... 4 more
```

This was caused by the test attempting to do a filter compaction when
there was currently no data to delete.  Added a check for this case in
this commit.
keith-turner authored Feb 22, 2024
1 parent f2adaeb commit 1a40aee
Showing 1 changed file with 18 additions and 15 deletions.
33 changes: 18 additions & 15 deletions test/src/main/java/org/apache/accumulo/test/ScanConsistencyIT.java
Original file line number Diff line number Diff line change
@@ -659,21 +659,24 @@ public String call() throws Exception {
// 1 in 20 chance of doing a filter compaction. This compaction will delete a data set.
var deletes = tctx.dataTracker.getDeletes();

// The row has the format <random long>:<generation>, the following gets the generations
// from the rows. Expect the generation to be the same for a set of data to delete.
String gen = deletes.stream().map(m -> new String(m.getRow(), UTF_8))
.map(row -> row.split(":")[1]).distinct().collect(MoreCollectors.onlyElement());

IteratorSetting iterSetting =
new IteratorSetting(100, "genfilter", GenerationFilter.class);
iterSetting.addOptions(Map.of("generation", gen));

// run a compaction that deletes every key with the specified generation. Must wait on the
// compaction because at the end of the test it will try to verify deleted data is not
// present. Must flush the table in case data to delete is still in memory.
tctx.client.tableOperations().compact(tctx.table, new CompactionConfig().setFlush(true)
.setWait(true).setIterators(List.of(iterSetting)));
numFilters++;
if (!deletes.isEmpty()) {
// The row has the format <random long>:<generation>, the following gets the generations
// from the rows. Expect the generation to be the same for a set of data to delete.
String gen = deletes.stream().map(m -> new String(m.getRow(), UTF_8))
.map(row -> row.split(":")[1]).distinct().collect(MoreCollectors.onlyElement());

IteratorSetting iterSetting =
new IteratorSetting(100, "genfilter", GenerationFilter.class);
iterSetting.addOptions(Map.of("generation", gen));

// run a compaction that deletes every key with the specified generation. Must wait on
// the
// compaction because at the end of the test it will try to verify deleted data is not
// present. Must flush the table in case data to delete is still in memory.
tctx.client.tableOperations().compact(tctx.table, new CompactionConfig().setFlush(true)
.setWait(true).setIterators(List.of(iterSetting)));
numFilters++;
}
}
}

0 comments on commit 1a40aee

Please sign in to comment.