A "pipeline-filewatcher" type job with pipeline task "Imports specimen using data file" reuses existing job record

LabKey Support Forum
A "pipeline-filewatcher" type job with pipeline task "Imports specimen using data file" reuses existing job record tstellin  2022-08-12 05:25
Status: Active
 

A month or so ago we updated our labkey jobs to use the new "pipeline-filewatcher" type job with pipeline task "Imports specimen using data file". Since doing this, we have observed a strange behavior: instead of creating a new Job record for a new "Imports specimen using data file" task, LabKey re-uses the most recent previously-existing job record. See attachment.

The attachment shows that the job was created on 2022-06-27 and last updated on 2022-08-12. When the task starts, it seems to re-use the old Job record. Every day, this record (with primary key rowId=98469) is re-used and modified to reflect that day's import. The job details page https://atlas.scharp.org/cpas/pipeline-status/HVTN/Tools and Reports/Specimen Management/details.view?rowId=98469 lists all previous runs in the log output (which I think will eventually make the page difficult to load).

Is this behavior expected? In the past, a new Job record (with a "created on" set to the timestamp the task started) would be created. We also see this behavior on our staging system (running v22.3.9).

 
 
mohara responded:  2022-08-12 09:12

Hi:
I'm transferring this to your premium support portal. Your account manager will be able to access that link and better investigate what might be happening here.
--Molly