In the greedy query planner, compute only O(n²) trees instead of O(n³)

[joka921]

First, a quick recap: For a connected component of n triples, the greedy query planner maintains a set of disjoint query execution trees. The initial set are n simple query execution trees, each consisting of one index scan. In each step, the cheapest possible join of two trees that are not yet connected, is computed. The worst case is when each pair of triples is connected, so that the induced triple graph is a clique.

In the implementation so far, a possible join is computed and its cost estimated (which involved constructing the respective tree) potentially many times, namely in each step until it is picked or no longer possible. Now, each possible join (and the respective query execution tree) is computed and evaluated exactly once and then discarded when it has been picked or is no longer possible. Since each step except the first adds exactly one tree (and throws out two trees, namely those that were joined), only a linear number of new trees have to be constructed in each step. In the code so far, this was a quadratic number in the worst case. This cost significant time for large connected components.

NOTE: The cheapest possible join in each step is still computed by a linear scan over all candidates, which in the worst case is a quadratic number in the first half of the steps. This could be improved by using a priority queues and other tricks. However, we would need to evaluate first whether these minimum computations actually cost significant time. In the code so far, the expensive part was the construction of the trees.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.86%. Comparing base (acb6633) to head (807324e).
Report is 10 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1722   +/-   ##
=======================================
  Coverage   89.86%   89.86%           
=======================================
  Files         389      389           
  Lines       37308    37334   +26     
  Branches     4204     4206    +2     
=======================================
+ Hits        33527    33552   +25     
+ Misses       2485     2484    -1     
- Partials     1296     1298    +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[hannahbast]

1-1 with Johannes, thanks a lot for the fix. We discussed a few changes

[hannahbast]

Another quick round

[hannahbast]

Awesome, I will test this end to end, write a description, and then merge this

[sparql-conformance]

Conformance check passed ✅

No test result changes.

Details: https://qlever.cs.uni-freiburg.de/sparql-conformance-ui?cur=807324e3f7d85d1ee9b4d3a17e0f3560c4751608&prev=acb6633debc7341985341aff147b5038cc8d951b

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud