Something that makes this problem worse is that my user used three different flow panels, and so has three different gating strategies. We have created three "Analysis folders", one for each panel/strategy, and have imported each workspace (several for each panel) into the appropriate "Analysis folder". However, there is only one FCSAnalyses table, so it has columns for all three gating strategies. So now we have three times as many gates/columns (well, not quite 3x, but on that order).

Would it make sense to give each analysis folder its own FCSAnalyses table?

Another question:
In some cases, two workspaces differ in their gate names by only case or white space, eg. "EXCD27HI" vs. "ex CD27hi". When importing a FlowJo workspace, does is the flow module case-insensitive when trying to match gate/column names? Is it white-space-insensitive?

Thanks.

Andy

More questions:

After I imported some of the FlowJo workspaces, I realized that some gate names had minor differences between workspaces. This results in stats that should be in the same column in the FCSAnalyses table to actually be in different columns. So I delete one or more "runs" of the imported workspaces, update the names of the gates in FlowJo so they are consistent, and re-import the workspaces. But the old, "bad" columns are still there in the FCSAnalyses table when I look in the Schema Designer.

Is there any way to:

1.) Get rid of columns in the FCSAnalyses table that are no longer used?
2.) Get rid of an "analysis folder" that is no longer needed? (I can delete the runs in that folder, but don't see a way to get rid of the folder.)

Thanks.

Andy

You can use the JavaScript LABKEY.Query.saveQueryViews() API to create or update a custom view programatically. This is the same API that the customize view panel uses to save views. You may also create a module with custom queries or custom views (.qview.xml) files which can then be version controlled.

The flow module creates a single set of tables per LabKey folder. It would be possible to create a separate set of tables in a LabKey folder for each panel, but this will require some work. Unlike the other assays which support creating more than one assay definition in a LabKey folder, the flow module currently only supports a single 'protocol' in a folder.

The flow module's gates are currently case-sensitive and whitespace-sensitive, unfortunately. It'd like to fix the case-sensitivity issue, and it seems reasonable to remove spaces in gate names as well. The flow module does have the ability to alias keywords, statistics, and graph names, but it isn't exposed as a general feature yet.

Unfortunately, it currently isn't possible to remove the unused keywords, statistics, and graph columns from the flow folder. That wouldn't be too difficult to add.

I've added a 'delete' button to the analysis folder grid to make it easier to remove an empty folder. For now, you can go to the experiment module use the 'run groups' grid to remove the analysis folder. A flow 'analysis folder' is actually an experiment 'run group'.

Kevin:

Thanks for the quick, complete responses. I'll give saveQueryViews() a try, but how do I get a list of all the possible "fields" under "Statistic" from the FCSAnalyses table? The Query APIs require you to specify a view (or default to the "default" view), but I'm trying to get a list of all possible columns - those aren't in any view. Or do I have to use "Customize View" to manually define a view that includes all possible columns first (ugh!), and use that view in a call to Query.getQueryViews()?

Once I know how to do that, I think my approach will be to fetch all possible stats, list them in a grid (or tree widget), and let the user select the ones she wants in her view. Then I save that view for her (and re-direct to that view as well, I suppose).

Has anyone done something like this already?

Andy

The best way to get a list of all statistics is to use the schema browser. Go to the schema browser, select the flow schema and FCSAnalyses table and then expand the 'Statistic' node. You'll see a huge pile of statistic columns.

I should mention that when programatically adding the columns to the view, you'll need to FieldKey encode the column name to escape reserved characters such as '/'. The easiest way to do that is to use the JavaScript LABKEY.FieldKey class. For example, to encode the statistic "A/B:Count" you would need to create a LABKEY.FieldKey from the parts:

var fieldKey = LABKEY.FieldKey.fromParts("Statistic", "A/B:Count");
var s = fieldKey.toString();

The encoded string will look something like 'Statistic/A$SB:Count'. Converting to a string may be unnecessary -- I think that when including a LABKEY.FieldKey instance in an JSON API request, it will be converted into an encoded string for you.

If you are writing a script you might also want to use the getQueryDetails API, which is how the Schema Browser gets the info to display.

Documented here:

https://www.labkey.org/download/clientapi_docs/javascript-api/symbols/LABKEY.Query.html#.getQueryDetails

I need to re-phrase my question:

How do I get a list of all possible "fields" under "Statistic" from the FCSAnalyses table PROGRAMMATICALLY? Sure I can visually see what's there in the Schema Browser, but how do I write a query to get that "huge pile of statistics columns"?

Thanks.

Andy

The LABKEY.Query.getQueryDetails() API should give you that information. It is used by both the schema browser and customize view.

When I try to use getQueryDetails(), I don't get the information I'm looking for. It returns only 24 columns, one of which is "Statistic". That column (FieldMetaData) has a "lookup" property (FieldMetaDataLookup), but the value of that property does not give me enough information to get the list of stats underneath "Statistic" - it just has schema:flow, schemaName:flow, public:false, and nothing else. Looking at the raw database tables, it looks like I want to query the flow.statisticattr table, and for that, I assume I use Query.executeSql().

Am I missing something?

Andy

I think you want to do something like:

<yourUrl>getQueryDetails.view?schemaName=flow&queryName=FCSAnalyses&fk=Statistic

The fk param is not documented -- it returns the columns available on the 'Statistic' lookup table. You can also use LABKEY.Query.selectRows({schemaName:"flow", queryName:"Statistics"}) to get a list of all statistics in the folder, but you'll have to translate the result rows into columns.

Thanks! That last one works well for me - querying the flow.Statistics table. Since it doesn't show up in the left-hand tree of queries/tables in the Schema Browser under the flow schema, I didn't realize it was there.

Andy