This topic covers some general information about monitoring, maintaining, and troubleshooting the Enterprise Pipeline.
Determine Which Jobs and Tasks Are Actively Running
Each job in the pipeline is composed of one or more tasks. These tasks are assigned to run at a particular location. Locations might include the web server and one or more remotes server for RAW to mzXML conversion, and other tools. Each location may have one or more worker threads that runs the tasks. A typical installation might have the following locations that run the specified tasks:
|Location||# of threads||Tasks|
|Web Server||1||CHECK FASTA IMPORT RESULTS|
|Web Server, high priority||1||MOVE RUNS|
|Conversion server||1+||MZXML CONVERSION|
|Other remote server||1+||SEARCH ANALYSIS|
When jobs are submitted, the first task in the pipeline will be added to the queue in the WAITING (SEARCH WAITING, for example) state. As soon as there is a worker thread available, it will take the job from the queue and change the state to RUNNING. When it is done, it will put the task back on the queue in the COMPLETE state. The web server should immediately advance the job to the next task and put it back in the queue in the WAITING state.
If jobs remain in a intermediate COMPLETE state for more than a few seconds, there is something wrong and the pipeline is not properly advancing the jobs.
Similarly, if there are jobs in the WAITING state for any of the locations, and no jobs in the RUNNING state for those locations, something is wrong and the pipeline is not properly running the jobs.
Reload User Undefined
If you see an error message similar to the following:
26 Jun 2018 02:00:00,071 ERROR: The specified reload user is invalid
Consider whether any user accounts responsible for reload pipeline jobs may have been deleted, instead of deactivated
To resolve errors of this kind, locate any automated reload that may have been defined and deactivate it. Reactivate the reload with a new, valid user account.