Rserve with assay transformation scripts

LabKey Support Forum (Inactive)
Rserve with assay transformation scripts Will Holtz  2016-01-28 15:31
Status: Closed
 
Hi,

I have Rserve working in a two-computer configuration. Currently this is just on my development machine (OSX) with the Rserve computer running in a linux VM. I tested that my Labkey instance can communicate with Rserve by running this simple report:

write.table(Sys.info(), file = "${tsvout:tsvfile}", sep = "\t", qmethod = "double", col.names=NA)

The output generated confirms the R code was executed on the Rserve VM.

I have several file-based modules that contain assays with data import transformation scripts. These transformation scripts are written in R and I would like for them to benefit from the Rserve performance bump. I tried to import data to an assay and got the following error:

Saving runs failed. eval failed, request status: error code: 127 Error in setwd("/foo/bar/trunk/server/localModules/HPLC/resources/assay/hplc/scripts/TransformAndValidationFiles/AssayId_4/65/") : cannot change working directory

That setwd command does not appear in my transformation script, so I’m guessing it is part of some setup done before the script. I added a path mapping to my Rserve configuration that mapped /foo/bar/trunk/server to a location within my Rserve VM. This allowed the transformation script to be found and start executing.

I then got the error:
Saving runs failed. eval failed, request status: error code: 127 Error: cannot open the connection Call sequence: 12: file(con, "r") 11: readLines("/foo/bar/trunk/server/localModules/HPLC/resources/assay/hplc/scripts/TransformAndValidationFiles/AssayId_4/65/runProperties.tsv") 10: readRunPropertiesFile() 9: eval(expr, envir, enclos) 8: eval(ei, envir) 7: withVisible(eval(ei, envir)) 6: source("/labkey/MODULEroot/localModules/HPLC/resources/assay/hplc/scripts/TransformAndValidationFiles/AssayId_4/65/HPLC_Transform.R") 5: eval(expr, envir, enclos) 4: eval(expr, pf) 3: withVisible(eval(expr, pf)) 2: evalVis(expr) 1: capture.output(source("/labkey/MODULEroot/localModules/HPLC/resources/assay/hplc/scripts/TransformAndValidationFiles/AssayId_4/65/HPLC_Transform.R"))

The file runProperties.tsv could also not be found because of path issues. Since it seems Labkey is setting the working directory to the location of runProperties.tsv, I just added a call to basename() for the runProperties.tsv read.

Finally, my script tries to read in the uploaded file from: /foo/bar/trunk/build/deploy/files/home/HPLC/@files/assaydata/uploaded_file.txt
If I was performing this upload from a pipeline, then labkey.remote.pipeline.root
Would get automatically pre-pended to my script according to: https://www.labkey.org/home/Documentation/wiki-page.view?name=LabKeyRserve. And I could use that to construct an absolute location of the file. But labkey.remote.pipeline.root does not appear to be set during an assay transformation. I can think of some ways to hack around this, but they aren’t as clean or portable. Has anyone else dealt with this problem yet?

thanks,
-Will
 
 
Jon (LabKey DevOps) responded:  2016-03-01 16:18
Hi Will,

Sorry for the late response. I was informed that when LabKey was made to work with Rserve, running transform scripts along with it wasn't planned initially. So this unfortunately isn't supported right now and would require a major feature enhancement to allow this to work since the transform scripts run asynchronously rather than synchronously (which is what Rserve expects).

This however leaves you with only two options:

A. Use transform scripts, but don't use Rserve

or

B. Use Rserve, but don't use transform scripts

Regards,

Jon
 
Will Holtz responded:  2016-03-01 16:55
Hi Jon,

Thanks for the reply. Sounds like I am out of luck. I am a little confused by your comment about asynchronous vs synchronous scripts. Assay transformation scripts can optionally be set to 'Import in Background'. I don't use this option and therefore my assay transformation scripts run in what I would call a synchronous mode.

-Will