Expected Behavior From Pipeline w/ Split Jobs?

LabKey Support Forum (Inactive)
Expected Behavior From Pipeline w/ Split Jobs? Ben Bimber  2014-07-29 10:55
Status: Closed
 
I have a pipeline job that does the following tasks:

Task 1 runs joined (ie. 1 job for all files)
Task 2 runs split (ie. creates a separate job per file).
Task 3 runs joined

I had a batch of files complete through Task 2, except 1 of 5 jobs failed. If I could get Task 3 to run, it would theoretically complete even though 1 of the previous jobs failed. I manually marked the failed job from Task 2 as completed from the pipeline grid. I was hoping this would coax Task 3 to continue, but that did not.

What's the expected behavior of a split->joined pipeline job if a job fails, and do I have any way to force Task 3 to continue even if one of the Task 2 jobs failed?

Thanks in advance.
 
 
jeckels responded:  2014-08-04 09:29
Hi Ben,

The ability to complete a job through the web UI is a low-level administrator function. It just flips the status in the database, without considering other potentially related jobs.

I don't think there's a way to force the joined job to resume with the current UI. You might be able to manually set it to ERROR via direct SQL against the database, and then retry it through the web UI.

If you expect this to happen on a regular basis, it shouldn't be too hard to make the manual process of setting a job to COMPLETE check if there are downstream jobs that can resume. Let me know if this is of interest.

Thanks,
Josh