BREAKING: Add cancellation support to IndexSearcher, #922

[paulirwin]

• You've read the Contributor Guide and Code of Conduct.
• You've included unit or integration tests for your change, where applicable.
• You've included inline docs for your change, where applicable.
• There's an open issue for the PR that you are making. If you'd like to propose a change, please open an issue to discuss the change or find an existing issue.

Adds basic cancellation support to IndexSearcher.

Fixes #922

Description

This adds optional CancellationToken parameters to all IndexSearcher.Search and SearchAfter methods, which is a breaking change to the API.

If an executor is provided in the constructor for multithreaded search, then the provided CancellationToken is passed to the Task.Wait(CancellationToken) method. If an executor is not provided, then it will call ThrowIfCancellationRequested() in the synchronous case at the evaluation of each leaf reader context.

This will not help cancellation for i.e. single-leaf reader contexts that might take a while to complete (that would require a much larger and messier change to add CancellationToken support to Scorer and ICollector, and benchmarking to ensure we don't hurt performance by doing so), but should at least help with many scenarios that require cancellation.

I wanted to publish this draft PR for feedback; please let me know if you like this direction or have concerns.

[NightOwl888]

Repeating this comment from #1119 (comment):

After taking a step back from this and considering that we are working on support for CancellationTokens, I wonder if we should consider removing LimitedConcurrencyLevelTaskScheduler.Shutdown() and always use CancellationToken instead?

If we do keep the LimitedConcurrencyLevelTaskScheduler.Shutdown() method, there are some things to consider:

1. Should we make LimitedConcurrencyLevelTaskScheduler public? The BCL doesn't have one and it is provided in the docs for TaskScheduler. But there is no ShutDown() on that implementation.
2. There are some tests where Shutdown() was called in Lucene that would likely perform better if we use it, since those lines were often commented out in Lucene.NET. However, using CancellationToken could be an alternative way to handle those cases.
   • https://github.com/apache/lucene/blob/releases/lucene-solr/4.8.1/lucene/core/src/test/org/apache/lucene/search/TestBooleanQuery.java#L177-L178
   • https://github.com/apache/lucene/blob/releases/lucene-solr/4.8.1/lucene/core/src/test/org/apache/lucene/search/TestSearcherManager.java#L291-L292
   • https://github.com/apache/lucene/blob/releases/lucene-solr/4.8.1/lucene/suggest/src/test/org/apache/lucene/search/spell/TestSpellChecker.java#L435-L437
   • https://github.com/apache/lucene/blob/releases/lucene-solr/4.8.1/lucene/test-framework/src/java/org/apache/lucene/index/ThreadedIndexingAndSearchingTestCase.java#L645-L646
   • https://github.com/apache/lucene/blob/releases/lucene-solr/4.8.1/lucene/core/src/test/org/apache/lucene/search/TestIndexSearcher.java#L119
   • https://github.com/apache/lucene/blob/releases/lucene-solr/4.8.1/lucene/test-framework/src/java/org/apache/lucene/util/LuceneTestCase.java#L1570

The upside of using Shutdown() is that it aligns more closely to Lucene.

The upside of using CancellationToken is that it is more fine-grained and can potentially shut down a process quicker, since it will be able to intervene inside of the running task instead of prior to queuing the task.

Allowing a user-defined TaskScheduler still seems like a good idea, but since Microsoft didn't allow them to be cancelled, it seems like we should follow the CancellationToken approach instead of trying to shoehorn a way to cancel a TaskScheduler. Using the same CancellationToken to make LimitedConcurrencyLevelTaskScheduler stop queuing new work is also something that might be worth considering.