-
Notifications
You must be signed in to change notification settings - Fork 11
Splitter Balancer Criteria Configuration
This criterion splits the test suite into a given number of units such that each unit has an equal or optimally equal number of tests.
- 20 tests and 1 unit => 1 unit with 20 tests.
- 18 tests and 3 units => 3 units with 6 tests.
- 39 tests and 6 units => 3 units with 6 tests and 3 units with 7 tests each.
This criterion splits the test suite into a given number of units such that each unit, when executed, takes the same or optimally same amount of time.
- 20 tests and 1 unit => 1 unit with 20 tests.
- 10 tests and 2 units with each taking 2 minutes => 2 units with 5 tests each taking 10 minutes each.
- 18 tests and 3 units, with 1 taking 13 minutes and rest taking 1 minute => 1 unit with the 13 minute test, 1 unit with 9 tests taking 9 minutes, 1 unit with 8 tests taking 8 minutes.
This criterion delegates to a criteria chain, trying criterion in order, until it exhausts the last one. This is generally the way we use ‘time based’ setup. We make a chain of ‘time based splitter’ followed by ‘count based splitter’ so if time based splitting fails, which may happen because a server(the whiteboard tlb runs against) holding artifact with test times is down or the artifact file doesn’t exist, it defaults to ‘count balancing’ which doesn’t require the ‘test suite time’ artifact. If all criterion in the chain fail, the build fails. This is preferred way of using ‘time based criterion’.
Delegating criterion requires CRITERIA_DEFAULTING_ORDER to be set to a colon(:) separated list of fully qualified names of criteria classes. For instance, a typical time followed by count criteria would look like ‘com.github.tlb.splitter.TimeBasedTestSplitterCriteria:com.github.tlb.splitter.CountBasedTestSplitterCriteria’.
The test split criteria is passed in using the environment variable TLB_CRITERIA. If this variable is not set, TLB by default uses a criteria that doesn’t do any balancing at all.