• Sign In

How do missing values work.

Study Forum (Inactive)

View Message

view list print
How do missing values work. rick.watts  2009-08-26 10:12
Status: Closed
 
So I've just installed Labkey 9.2 and am attempting to import some data into a study. I am on the 'define dataset from file' page and have selected a tab delimeted file. The problem is that I get lots of 'datatype error for field' messages.I believe the problem is caused by text in otherwise numeric fields. I have text such as UNK to represent missing values.

Can anyone tell me how the missing values feature of Labkey works? I assumed tah if I defined the appropriate codes as missing value indicators they would import ok, but this does not seem to be the case.

Thanks

Rick Watts
Women & Children's Health Research Institute
Edmonton, Alberta.
 
 
adam responded:  2009-08-26 10:29
You need to define valid missing values in Folder Settings (as it sounds like you have done). You also need to enable missing values on each column where you want to allow them; use the "Missing Value Indicators" checkbox in the "Edit Dataset Definitions" UI or "mvEnabled" flag if doing a bulk schema import from schema.tsv.

Let us know if this doesn't solve the problem.

Adam
 
adam responded:  2009-08-26 14:13
After trying your scenario with a sample dataset, it's clear that "define dataset from file" and "missing values" don't work that well together. As I implied above, using missing values requires that you set the missing value flag on the column before inserting data. The code that attempts to infer schemas from .xls / .tsv files does not currently take missing values into consideration. This is something we would like to add in the future -- see https://www.labkey.org/issues/home/Developer/issues/details.view?issueId=8529

Adam