Hi Shadi,

Can you provide us with more details?

What version of LabKey are you accessing?

Are you accessing a local version of LabKey on your computer or are you accessing a production version on a server?

What table or query are you working with to generate your R view against? Is it a list? A dataset from a study? An assay table? A table from a custom schema?

If you click on Reports > Create R Report (or Views > Create > R View depending on which version of LabKey you're running), does the default code on the source tab that looks like this work when you go to the view tab afterwards?

# This sample code returns the query data in tab-separated values format, which LabKey then
# renders as HTML. Replace this code with your R script. See the Help tab for more details.
write.table(labkey.data, file = "${tsvout:tsvfile}", sep = "\t", qmethod = "double", col.names=NA)

Can you also send us a screenshot of the error you're seeing?

Regards,

Jon

Hi Jon,

Thanks for your quick reply.

I am running the development version of Labkey 16.1 on a private server and developing an R view for a custom sql query of targetedms schema type.

This is what I get by default in my R Report Builder Interface, which I believe is identical to what you provided.

# This sample code returns the query data in tab-separated values format, which LabKey then
# renders as HTML. Replace this code with your R script. See the Help tab for more details.
write.table(labkey.data, file = "${tsvout:tsvfile}", sep = "\t", qmethod = "double", col.names=NA)

An screenshot of the error is attached. I should also mention that the problem is not always persistent. Sometimes logging out and in again into the server temporarily fixes the issue, but most of the times it doesn't. And I get the same error even when using the default R script copied above, so it does not seem to depend on the script.

Thanks for your help!

Hi Shadi,

Can you try making an R view on a list and tell me if you get the same behavior?

Regards,

Jon

The default R script works fine on the lists and in fact applying the R view to the list seems to reset the flag that disables the default R script on the query, because the default script works again if I run it on the query afterwards.
I think something in my code is triggering the issue. More specifically I suspect the issue is triggered by one of the functions that communicates with the server to retrieve, write or delete data, because everything runs smoothly until I add the last bit of code that uses getRows, labkey.deleteRows and labkey.insertRows. Each of these functions seems to work fine individually, but the issue seems to occur when I run them sequentially in the same script.

Thanks!

Hi Shadi,

Any chance you can provide us with a copy of the R script you're using?

The only issue I know of where R scripts start having issues with is being used as a Transformation Script for an Assay depending on whether its doing multiple API calls within the script, but this shouldn't cause problems with a view though to my understanding.

Regards,

Jon

Hi Jon,

I have attached the piece of the code that is causing the issue and you might be able to use it to reproduce the error. I would appreciate any hints.

Thanks so much for your help and if you have any question please do not hesitate to contact me.

Best,
Shadi

Hi Shadi,

I took a look at your file and noticed that you do have a lot of API calls being made, but there are some things I'm confused about here.

1. The insertRows functions you're using appear to not have the toInsert option referencing anything.

Here's an example from our Rlabkey documentation:

newrow <- data.frame(
DisplayFld="Inserted from R"
, TextFld="how its done"
, IntFld= 98
, DoubleFld = 12.345
, DateTimeFld = "03/01/2010"
, BooleanFld= FALSE
, LongTextFld = "Four score and seven years ago"
# , AttachmentFld = NA #attachment fields not supported
, RequiredText = "Veni, vidi, vici"
, RequiredInt = 0
, Category = "LOOKUP2"
, stringsAsFactors=FALSE)

insertedRow <- labkey.insertRows("http://localhost:8080/labkey",
folderPath="/apisamples", schemaName="lists", queryName="AllTypes",
toInsert=newrow)

newRowId <- insertedRow$rows[[1]]$RowId

Notice how there is a newrow data.frame above the actual insertRows call that has toInsert=newrow?

Your insertRows is just doing the API call, but the toInsert value doesn't exist.

labkey.insertRows(baseUrl="Insert Labkey URL",
                  folderPath="Insert Directory",
                  schemaName = "lists",
                  queryName = "QuaSAR_AuDIT",
                  toInsert = import_AuDIT)

So although this is doing an insertRows, there is no import_AuDIT data.frame to work off of here.


2. Also, what is this script supposed to do in the grand scheme of things? You have it deleting and inserting rows, but for an Rview, doing this type of work is unusual since the report shouldn't be removing data or inserting data, but just rendering it using things such as selectRows.

Regards,

Jon

Jon,

To answer the points you brought up:

1. The toInsert option references the output of a function in my code. I have attached an updated code that generate some dummy data instead. (please see the attached file)

2. The script intends to process the data from a custom query and show the results in multiple tables for the users to view. Is there a better way of doing this? I am very new to Labkey and would appreciate any suggestions.

3. I think I have figured out the source of the problem. It appears that the API calls fail when the number of rows in the data pass some threshold. In the file I have attached, you can change the number of rows of the dummy data table using the rownumber variable. For example, the code works fine for rownumber = 100, but fails for rownumber = 5000. Do you know what causes this and how we can solve the issue?

Thanks,
Shadi

Hi Shadi,

The message you're seeing in the user interface indicates that the browser is getting an error back from the server when it attempts to run the report. It will be very helpful if we can identify the specific error it's hitting. Please try the following:

1. Using Chrome as your browser, go to the page where you're hitting the error, and go to the Source tab.
2. Right click and choose Inspect in the menu.
3. Go to the Network tab in the debugging tool that pops up.
4. Go to the Report tab in the R UI.
5. In the debugging tool, find the viewScriptReport entry.
6. Click on the Response tab in the debugging tool.

Please capture what you see. Check the attached screenshot for an example.

Also, if you can have Jon S or one of the other admins pull the labkey-errors.log file from the server, that will help us as well.

Thanks,
Josh

Hi Josh,

On the error page and for a dataset with rownumber = 5000, the viewScriptReport Response is empty. It basically says "failed to load response data" (screenshot attached). For comparison, I ran the report again with a small dummy dataset (rownumber = 50) and the viewScriptReport Resonse is coppied below.

I have requested the log file and will send it to you as soon as it's available.

Thanks,
Shadi


{
  "requiredJsScripts" : [ ],
  "requiredCssScripts" : [ ],
  "implicitJsIncludes" : [ ],
  "html" : "[superlong console output string ... deleted]"
   "moduleContext" : {
   "pipeline" : { },
   "core" : { },
   "experiment" : { },
   "filecontent" : { },
   "wiki" : { },
   "query" : { },
   "issues" : { },
   "announcements" : { },
   "ms2" : { },
   "targetedms" : {
     "TargetedMS Folder Type" : "Experiment",
     "TargetedMS AutoQCPing Timeout" : "15"
    }
  }
}

Hi Shadi,

Does it take a while for the error message to appear? I wonder if it's hitting a timeout. Can you get an approximate time for how long from the time it starts trying to run to the script to get the message? If it's something like 30 or 60 seconds, that might be the problem, and we can then investigate that side of things.

Thanks,
Josh

Hi Josh,
I takes >3 minutes for the error to show.

Thanks,
Shadi

Hi Shadi,

I don't see signs of any other errors, so it seems likely that it's a timeout issue. Please checking the "Run this report in the background as a pipeline job" checkbox in the configuration for the R view. This will cause it to run in the background on the server, and allow you to collect the results after it completes. This is typically a better approach for these kinds of longer-running jobs, so that the user doesn't have to leave their browser sitting, waiting for the script to finish.

You can find the docs for this here:

https://www.labkey.org/home/Documentation/wiki-page.view?name=rBuilder

Thanks,
Josh

Josh,

Thanks for your note.

I am trying to run the script as a pipeline job in the background, but I get an error in the getSession call. (This is the same exact script that runs smoothly with a dataset of 50 rows.)

Error in fromJSON(mydata) : unexpected character '<'
Calls: getSession -> .setupSchemas -> labkey.getSchemas -> fromJSON
Execution halted

I have attached the output of sessionInfo() in case that helps.

Thanks,
Shadi

PS. To clarify, this script runs smoothly if I run it the normal way and not in the background.

HI Shadi,

Did you manage to get a copy of the logs for us to look at as well?

Just to confirm, you're not running an actual labkey.getSchemas in your R code, right?

http://search.r-project.org/library/Rlabkey/html/labkey.getSchemas.html

Regards,

Jon

Hi Shadi,

I'm guessing that your script is having a problem where it's not being trusted by the web server. It should be connecting as the user who initiated the upload process.

In the latest version of Rlabkey, you should get a better error message. If possible, please ask an admin to update Rlabkey on the server.

Please try adding the following to your Rscript on its own line:

${rLabkeySessionId}

I wouldn't expect it to be needed only when it's running in the background, but I hope it will resolve this problem.

Thanks,
Josh

I should add that you can see a full list of supported substitutions here:

https://www.labkey.org/home/Documentation/wiki-page.view?name=transformationSubstitutionSyntax

Thanks,
Josh

The issue related to running R scripts as pipeline jobs is fixed in the 16.2 update.

Thanks,
Shadi