×
We value your feedback! Please click here to access our 2024 Client Survey.
The survey will remain open until November 1st. Your perspective will help guide our product/service planning and goals for the upcoming year and beyond!
Premium Feature — Available with all Premium Editions of LabKey Server. Learn more or contact LabKey.

This topic describes how to load clinical data in the CDISC ODM XML file format into a LabKey Study. For example, data collected using DFdiscover can be exported in this format. The data loading process is based on a File Watcher processing pipeline configured to read CDISC_ODM XML files.

Note that while this topic is using the DFdiscover example, other EDC tools that can output data in a CDISC XML format can also be integrated using this method.

Process Overview

A summary of the data loading process:

  • Data is collected and stored using a tool like DFdiscover.
  • The data is manually exported into XML files that conform to the CDISC ODM format.
  • These XML files are dropped into a location watched by LabKey Server.
  • LabKey Server detects the files, which triggers the data processing pipeline.
  • The pipeline parses the XML files and loads the data into study datasets or lists, as specified.

File Watcher Options

This File Watcher supports two modes of watching for content in a specified location.

  • Trigger on individual file(s) (typically conforming to a naming pattern provided) and then act on each file.
  • Trigger on addition of a new folder, acting on the contents of that folder in a batch. Under this mode, it is important that the files required are already in the folder when it is dropped into the watched location. In a case where a user manually creates a subfolder, then drops files into it there, it is likely that the trigger will fire before all the content is present, leading to unexpected results.
In both cases, the "quiet period" must be long enough for completion of the entire upload to be acted upon.

The Import Study Data from a CDISC ODM XML File integration can use either the single file method (in which case a default configuration is applied) or the use of the folder approach in order to specify the configuration in a separate file uploaded with the data.

Configuration File Options

Default Configuration

In the mode where only the data files are provided to the watched location, the default configuration is used:

  • All forms in the data file are imported as datasets

Customized Configuration

When the data file(s) and a configuration file are provided together in a folder, that configuration file can contain different options, such as importing some files as lists or with specific key field settings etc. You will specify the name of this configuration file in the File Watcher definition, and it must conform to the CDISC XML Schema.

  • If you configure the File Watcher to look for a configuration file, it will need to be placed in the same folder as the target file. If the configuration file is not found, the File Watcher will error out.
  • Best practice is to upload the configuration file before the target file OR set the quiet period long enough to allow enough time to upload the configuration file.
  • If the File Watcher is configured to move files to another container, only the target file will be moved automatically and the job will fail. In this case you will want to upload a folder with both the configuration and target XML files in the folder. To set up, either don't provide a file name expression at all or make sure the expression will accept the folder name(s).
  • This File Watcher trigger has a special file filter that accepts a target folder as long as it contains one or more XML files.
An example custom configuration file, giving different configurations for a set of forms: Click here to download.
<?xml version="1.0" encoding="UTF-8"?>
<cdiscConfig xmlns="http://labkey.org/cdisc/xml">
<timepointType>visit</timepointType>
<studyLabel>CDISC Visit Study</studyLabel>
<forms>
<form>
<formOID>FORM.DEMOG</formOID>
<demographic>true</demographic>
</form>
<form>
<formOID>FORM.VITPHYEX</formOID>
<keyFieldOID>IT.REC_ID</keyFieldOID>
</form>
<form>
<formOID>FORM.PHARMVIT</formOID>
<keyFieldOID>LK_MANAGED_KEY</keyFieldOID>
</form>
<form>
<formOID>FORM.AE</formOID>
<exportAsList>true</exportAsList>
</form>
</forms>
</cdiscConfig>

CDISC XML Schema

The settings available in a configuration file are downloadable in this file.

<?xml version="1.0" encoding="UTF-8"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns="http://labkey.org/cdisc/xml"
targetNamespace="http://labkey.org/cdisc/xml"
elementFormDefault="qualified" attributeFormDefault="unqualified">

<xsd:annotation>
<xsd:documentation xml:lang="en">CDISC ODM Export Configuration</xsd:documentation>
</xsd:annotation>

<xsd:element name="cdiscConfig">
<xsd:complexType>
<xsd:all>
<xsd:element name="timepointType" type="xsd:string" minOccurs="0"/>
<xsd:element name="studyLabel" type="xsd:string" minOccurs="0"/>
<xsd:element name="importAll" type="xsd:boolean" minOccurs="0"/>
<xsd:element name="visitFieldName" type="xsd:string" minOccurs="0"/>
<xsd:element name="keyFieldOID" type="xsd:string" minOccurs="0"/>
<xsd:element name="defaultSequenceNum" type="xsd:double" minOccurs="0"/>
<xsd:element name="disableLookups" type="xsd:boolean" minOccurs="0" default="false"/>
<xsd:element name="forms" minOccurs="0" maxOccurs="1">
<xsd:complexType>
<xsd:sequence minOccurs="0" maxOccurs="unbounded">
<xsd:element name="form">
<xsd:complexType>
<xsd:all>
<xsd:element name="formOID" type="xsd:string"/>
<xsd:element name="visitFieldName" type="xsd:string" minOccurs="0"/>
<xsd:element name="demographic" type="xsd:boolean" minOccurs="0"/>
<xsd:element name="exportAsList" type="xsd:boolean" minOccurs="0"/>
<xsd:element name="keyFieldOID" type="xsd:string" minOccurs="0"/>
<xsd:element name="defaultSequenceNum" type="xsd:double"/>
</xsd:all>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:all>
</xsd:complexType>
</xsd:element>
</xsd:schema>

Set Up the File Watcher

To set up the File Watcher pipeline, follow these steps:

  • Create or identify a study folder where you wish to load the DFdiscover data.
  • Navigate to this study folder.
  • Select (Admin) > Go To Module > FileContent.
  • In the file repository, create a directory where the server will watch for files/folders. In this example, we named the directory "watched".
  • Go to (Admin) > Folder > Management and click the Folder Type tab.
  • Check the box for CDISC_ODM and click Update Folder to enable the module.

Next, determine whether you need to customize the configuration in a file and if necessary, create that file. Both options are shown in these File Watcher configuration steps:

  • Return to (Admin) > Folder > Management and click the Import tab.
  • Configure a File Watcher that uses the task Import Study Data from a CDISC ODM XML File.
  • In this example, we have a file browser directory named "watched" to watch and when we upload our files, want to move the files to "@pipeline/processed_files" for processing.
  • Note that the Quiet Period should be set for enough seconds to allow the data and configuration files to be uploaded.
  • For a data file only configuration, provide a file pattern and do not name a configuration file:
  • For a data and config file configuration, do not provide a file pattern (you will upload both files in a folder together) and do provide the name of your config file.

Use the File Watcher

  • If you are using the data file only method, drop the .xml file into the watched location.
  • If you are importing both a data and config file, on your filesystem, create a folder containing both files.
    • Drop this folder into the "watched" location. You'll want to be sure that the quiet period is sufficient for the entire contents to upload.
  • For either method, the File Watcher will trigger and you'll see the data imported into your study as directed.

Related Topics

Was this content helpful?

Log in or register an account to provide feedback


previousnext
 
expand allcollapse all