mule error wnels2  2009-11-28 21:00
Status: Closed
 
I have added a task to the mule pipeline. It works most of the but every once in a while I get the exception below. Also, the pipeline will get stuck intermittently - restarting ActiveMQ will restart everything. Everything is running on a laptop - no globus. Do you have any suggestions on how to trouble shoot this?
Thanks,
Bill

28 Nov 2009 22:13:30,359 ERROR: class org.mule.umo.ComponentException
org.mule.umo.ComponentException: Failed to invoke org.labkey.pipeline.mule.PipelineJobRunner. Component that caused exception is: PipelineJobRunnerUMO. Message payload is of type: ActiveMQTextMessage
    at org.mule.impl.DefaultLifecycleAdapter.intercept(DefaultLifecycleAdapter.java:199)
    at org.mule.impl.InterceptorsInvoker.execute(InterceptorsInvoker.java:47)
    at org.mule.impl.model.DefaultMuleProxy.run(DefaultMuleProxy.java:470)
    at org.mule.impl.work.WorkerContext.run(WorkerContext.java:310)
    at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$CallerRunsPolicy.rejectedExecution(ThreadPoolExecutor.java:1895)
    at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:765)
    at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1288)
    at org.mule.impl.work.ScheduleWorkExecutor.doExecute(ScheduleWorkExecutor.java:39)
    at org.mule.impl.work.MuleWorkManager.executeWork(MuleWorkManager.java:277)
    at org.mule.impl.work.MuleWorkManager.scheduleWork(MuleWorkManager.java:244)
    at org.mule.impl.model.seda.SedaComponent.run(SedaComponent.java:483)
    at org.mule.impl.work.WorkerContext.run(WorkerContext.java:310)
    at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1061)
    at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:575)
    at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NullPointerException
    at java.util.Properties$LineReader.readLine(Properties.java:418)
    at java.util.Properties.load0(Properties.java:337)
    at java.util.Properties.load(Properties.java:325)
    at org.labkey.ms2.pipeline.aws.Credentials.loadProperties(Credentials.java:65)
    at org.labkey.ms2.pipeline.aws.Credentials.<init>(Credentials.java:24)
    at org.labkey.ms2.pipeline.aws.CredentialFactory.getCredentials(CredentialFactory.java:19)
    at org.labkey.ms2.pipeline.aws.S3Utilities.initS3Service(S3Utilities.java:161)
    at org.labkey.ms2.pipeline.aws.S3Utilities.getS3Utilities(S3Utilities.java:39)
    at org.labkey.ms2.pipeline.aws.S3Utilities.uploadData(S3Utilities.java:46)
    at org.labkey.ms2.pipeline.tandem.XTandemSearchTask.run(XTandemSearchTask.java:175)
    at org.labkey.api.pipeline.PipelineJob.runActiveTask(PipelineJob.java:621)
    at org.labkey.api.pipeline.PipelineJob.run(PipelineJob.java:844)
    at org.labkey.pipeline.mule.PipelineJobRunner.run(PipelineJobRunner.java:31)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.mule.impl.model.resolvers.DynamicEntryPoint.invokeMethod(DynamicEntryPoint.java:312)
    at org.mule.impl.model.resolvers.DynamicEntryPoint.invoke(DynamicEntryPoint.java:259)
    at org.mule.impl.DefaultLifecycleAdapter.intercept(DefaultLifecycleAdapter.java:193)
    ... 14 more
 
 
jeckels responded:  2009-11-30 16:10
Hi Bill,

It looks like this is happening in your new code. If you can get this to repro when running in a debugger, setting an exception breakpoint on NullPointerException will make it so that you can check out the problem when it happens.

I do have one guess as to the root problem, though. If you're causing the web server to redeploy the webapp (by deploying a new build without restarting the server, either by building it or copying in the files), I've seen problem where sometimes Tomcat is unable to load ClassLoader resources without restarting the whole Tomcat process. The only workaround that I know is to restart the server - it's annoying but typically only a problem on development machines where you're rebuilding.

Thanks,
Josh
 
wnels2 responded:  2009-12-03 11:29
Thanks for the reply.
Yea, I usually do restart tomcat after a build but I forget sometimes. I do think I found the min problem though - also development related - There were about 40 messages persisted in the job.queue. I haven't had the problem since I deleted them.