This topic describes how to load clinical data in the CDISC ODM XML file format into a LabKey Study. For example, data collected using DFdiscover can be exported in this format. The data loading process is based on a File Watcher processing pipeline configured to read CDISC_ODM XML files.

• Process Overview
• File Watcher Options
• Configuration File Options
• Default Configuration
• Customized Configuration
• CDISC XML Schema
• Set Up the File Watcher
• Use the File Watcher

Process Overview

A summary of the data loading process:

• Data is collected and stored using a tool like DFdiscover.
• The data is manually exported into XML files that conform to the CDISC ODM format.
• These XML files are dropped into a location watched by LabKey Server.
• LabKey Server detects the files, which triggers the data processing pipeline.
• The pipeline parses the XML files and loads the data into study datasets or lists, as specified.

File Watcher Options

This File Watcher supports two modes of watching for content in a specified location.

• Trigger on individual file(s) (typically conforming to a naming pattern provided) and then act on each file.
• Trigger on addition of a new folder, acting on the contents of that folder in a batch. Under this mode, it is important that the files required are already in the folder when it is dropped into the watched location. In a case where a user manually creates a subfolder, then drops files into it there, it is likely that the trigger will fire before all the content is present, leading to unexpected results.

The Import Study Data from a CDISC ODM XML File integration can use either the single file method (in which case a default configuration is applied) or the use of the folder approach in order to specify the configuration in a separate file uploaded with the data.

Configuration File Options

Default Configuration

In the mode where only the data files are provided to the watched location, the default configuration is used:

• All forms in the data file are imported as datasets

Customized Configuration

When the data file(s) and a configuration file are provided together in a folder, that configuration file can contain different options, such as importing some files as lists or with specific key field settings etc. You will specify the name of this configuration file in the File Watcher definition, and it must conform to the CDISC XML Schema.

• If you configure the File Watcher to look for a configuration file, it will need to be placed in the same folder as the target file. If the configuration file is not found, the File Watcher will error out.
• Best practice is to upload the configuration file before the target file OR set the quiet period long enough to allow enough time to upload the configuration file.
• If the File Watcher is configured to move files to another container, only the target file will be moved automatically and the job will fail. In this case you will want to upload a folder with both the configuration and target XML files in the folder. To set up, either don't provide a file name expression at all or make sure the expression will accept the folder name(s).
• This File Watcher trigger has a special file filter that accepts a target folder as long as it contains one or more XML files.

<?xml version="1.0" encoding="UTF-8"?>
<cdiscConfig xmlns="http://labkey.org/cdisc/xml">
    <timepointType>visit</timepointType>
    <studyLabel>CDISC Visit Study</studyLabel>
    <forms>
        <form>
            <formOID>FORM.DEMOG</formOID>
            <demographic>true</demographic>
        </form>
        <form>
            <formOID>FORM.VITPHYEX</formOID>
            <keyFieldOID>IT.REC_ID</keyFieldOID>
        </form>
        <form>
            <formOID>FORM.PHARMVIT</formOID>
            <keyFieldOID>LK_MANAGED_KEY</keyFieldOID>
        </form>
        <form>
            <formOID>FORM.AE</formOID>
            <exportAsList>true</exportAsList>
        </form>
    </forms>
</cdiscConfig>

CDISC XML Schema

The settings available in a configuration file are downloadable in this file.

<?xml version="1.0" encoding="UTF-8"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
            xmlns="http://labkey.org/cdisc/xml"
            targetNamespace="http://labkey.org/cdisc/xml"
            elementFormDefault="qualified" attributeFormDefault="unqualified">

    <xsd:annotation>
        <xsd:documentation xml:lang="en">CDISC ODM Export Configuration</xsd:documentation>
    </xsd:annotation>

    <xsd:element name="cdiscConfig">
        <xsd:complexType>
            <xsd:all>
                <xsd:element name="timepointType" type="xsd:string" minOccurs="0"/>
                <xsd:element name="studyLabel" type="xsd:string" minOccurs="0"/>
                <xsd:element name="importAll" type="xsd:boolean" minOccurs="0"/>
                <xsd:element name="visitFieldName" type="xsd:string" minOccurs="0"/>
                <xsd:element name="keyFieldOID" type="xsd:string" minOccurs="0"/>
                <xsd:element name="defaultSequenceNum" type="xsd:double" minOccurs="0"/>
                <xsd:element name="disableLookups" type="xsd:boolean" minOccurs="0" default="false"/>
                <xsd:element name="forms" minOccurs="0" maxOccurs="1">
                    <xsd:complexType>
                        <xsd:sequence minOccurs="0" maxOccurs="unbounded">
                            <xsd:element name="form">
                                <xsd:complexType>
                                    <xsd:all>
                                        <xsd:element name="formOID" type="xsd:string"/>
                                        <xsd:element name="visitFieldName" type="xsd:string" minOccurs="0"/>
                                        <xsd:element name="demographic" type="xsd:boolean" minOccurs="0"/>
                                        <xsd:element name="exportAsList" type="xsd:boolean" minOccurs="0"/>
                                        <xsd:element name="keyFieldOID" type="xsd:string" minOccurs="0"/>
                                        <xsd:element name="defaultSequenceNum" type="xsd:double"/>
                                    </xsd:all>
                                </xsd:complexType>
                            </xsd:element>
                        </xsd:sequence>
                    </xsd:complexType>
                </xsd:element>
            </xsd:all>
        </xsd:complexType>
    </xsd:element>
</xsd:schema>

Set Up the File Watcher

To set up the File Watcher pipeline, follow these steps:

• Create or identify a study folder where you wish to load the DFdiscover data.
• Navigate to this study folder.
• Select (Admin) > Go To Module > FileContent.
• In the file repository, create a directory where the server will watch for files/folders. In this example, we named the directory "watched".
• Go to (Admin) > Folder > Management and click the Folder Type tab.
• Check the box for CDISC_ODM and click Update Folder to enable the module.

Next, determine whether you need to customize the configuration in a file and if necessary, create that file. Both options are shown in these File Watcher configuration steps:

• Return to (Admin) > Folder > Management and click the Import tab.
• Configure a File Watcher that uses the task Import Study Data from a CDISC ODM XML File.

• In this example, we have a file browser directory named "watched" to watch and when we upload our files, want to move the files to "@pipeline/processed_files" for processing.
• Note that the Quiet Period should be set for enough seconds to allow the data and configuration files to be uploaded.
• For a data file only configuration, provide a file pattern and do not name a configuration file:

• For a data and config file configuration, do not provide a file pattern (you will upload both files in a folder together) and do provide the name of your config file.

Use the File Watcher

Related Topics

• File Watchers
• Download cdiscExport.xsd
• CDISC XML Format for Medidata Integration