This topic covers some general information about monitoring, maintaining, and troubleshooting the Enterprise Pipeline. Due to the high level of customization that is possible, some of the information may vary from installation to installation.
Determine Which Jobs and Tasks Are Actively Running
Each job in the pipeline is composed of one or more tasks. These tasks are assigned to run at a particular location. Locations might include the web server, cluster, remote server for RAW to mzXML conversion, etc. Each location may have one or more worker threads that runs the tasks. A typical installation might have the following locations that run the specified tasks:
| Location | # of threads | Tasks |
|---|
| Web Server | 1 | CHECK FASTA IMPORT RESULTS |
| Web Server, high priority | 1 | MOVE RUNS |
| Conversion server | 1+ | MZXML CONVERSION |
| Cluster | 1+ | SEARCH ANALYSIS |
When jobs are submitted, the first task in the pipeline will be added to the queue in the WAITING (SEARCH WAITING, for example) state. As soon as there is a worker thread available, it will take the job from the queue and change the state to RUNNING. When it is done, it will put the task back on the queue in the COMPLETE state. The web server should immediately advance the job to the next task and put it back in the queue in the WAITING state.
If jobs remain in a intermediate COMPLETE state for more than a few seconds, there is something wrong and the pipeline is not properly advancing the jobs.
Similarly, if there are jobs in the WAITING state for any of the locations, and no jobs in the RUNNING state for those locations, something is wrong and the pipeline is not properly running the jobs.
