Include the task scheduling errors in schedule response #14754
Conversation
shounakmk219
requested review from
tibrewalpratik17,
swaminathanmanish and
KKcorps
shounakmk219
added
backward-incompat
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #14754 +/- ##
============================================
+ Coverage 61.75% 63.84% +2.09%
- Complexity 207 1609 +1402
============================================
Files 2436 2704 +268
Lines 133233 151008 +17775
Branches 20636 23318 +2682
============================================
+ Hits 82274 96412 +14138
- Misses 44911 47380 +2469
- Partials 6048 7216 +1168
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
pinot-controller/src/main/resources/app/pages/TaskQueueTable.tsx
Outdated
Show resolved
Hide resolved
...egration-test-base/src/test/java/org/apache/pinot/integration/tests/MinionTaskTestUtils.java
Outdated
Show resolved
Hide resolved
| assertNotNull(info.getGenerationErrors()); | ||
| assertTrue(info.getGenerationErrors().isEmpty()); | ||
| assertNotNull(info.getSchedulingErrors()); | ||
| assertTrue(info.getSchedulingErrors().isEmpty()); |
There was a problem hiding this comment.
Not sure iiuc, but errors can be non-empty too if no task is scheduled due to scheduling / generation errors right?
There was a problem hiding this comment.
The tests are using this util methods right now to assert that tasks are not being scheduled due to no work or tasks already in progress. In that flow we need to make sure that there are no errors as well.
There was a problem hiding this comment.
Hmm would suggest to move the last 4 assertions to a separate method maybe -- assertNoError kind of. But can be done in a follow-up PR as well.
Co-authored-by: Pratik Tibrewal <[email protected]>
Description
The task schedule endpoint on controller does not give the user any visibility into how the scheduling went when there is a
taskType : nullresponseTo relay more info back to the user, this PR enhances the return types of
PinotTaskManagerschedule methods to a wrapper object that contains the generation and scheduling errors along with the names of scheduled tasksIn order to not break the schedule endpoints the error info is returned as separate entry in the response map as below
Earlier response
Successful schedule
Successful schedule with no tasks to schedule or failed schedule
Improved response
Successful schedule without any errors
Successful schedule with no tasks to schedule
Failed schedule with errors during task generation
UI Changes
This PR also makes the UI change for
Schedule NowctaWhen task is scheduled
When there is nothing to schedule
When there are scheduling errors
Backward incompatible changes
The
PinotTaskManagerschedule methods have changed return types from eitherList<String>toTaskSchedulingInfoorMap<String, List<String>>toMap<String, TaskSchedulingInfo>Schedule methods used to return null as scheduled task names list incase of both scheduling failures as well as no task to schedule. Now that behaviour is changed to null in case of scheduling failures and empty list when there is no task to be scheduled.