• Sign In
  1. Messages

Full Microarray Analysis

LabKey Support Forum (Inactive)

View Message

view list print
Full Microarray Analysis bill c white  2014-02-03 10:50
Status: Closed
 
I am attempting to run operations on a full microarray in LabKey. I need to do operations on a 54000x26 array. I normally do this with C++. The workflow is:

-filter the array to a top number of genes, say 5000, based on gene stats
-run a ranking routine on the reduced data, down to say 100
-form a new matrix from the reduced data
-run a routine to discover modular structure
-visualize using d3

I have my analysis routines in R. I've prototyped d3 and it works fine in LabKey. I can call my R routines from views. Now to implement this pipeline. The first thing I've run into is that I need (I think) Rlabkey to query the data (lists so far) and it is SLOW. I waited for about an hour and it never finished retrieving the data query (it works fine on small toy problems I tried). So my question is: Is it feasible to think of doing this type of thing in R? Should I think about a custom Java module? I really like LabKey but performance is a huge issue.
 
 
jeckels responded:  2014-02-03 16:35
Hi Bill,

This should be feasible. We have users who are doing this sort of analysis/visualization in R with much snappier response times than you're seeing here.

Are you setting up your R script as an R report (Views->Create->R View) or through some other mechanism?

Do you know where the bottleneck is? It could be in querying the data from the database, writing it to the R script, running the R script itself, or marshalling the results to send back to the browser. Looking at CPU usage may be enough to determine which stage it's happening at, based on which process is taking up the CPU.

Thanks,
Josh
 
bill c white responded:  2014-02-04 05:34
Thanks Josh.

I am working with the LabKey developers to try and resolve this. Microarray analysis is something currently under development in collaboration with HIPC. All of my methods are in C++, so getting them to work with R and Javascript is a challenge to say the least.

Bill