This topic covers features within the LabKey Data Finder module.
Using the DataFinder
The LabKey DataFinder provides a portal for users to browse multiple studies and related publications. Categories can be applied to these research assets, which allows users to browse by categories of interest.
The
Studies tab displays the available studies as separate "cards". The links on each card behave as follows:
- View Summary displays a larger pop-up card showing a logo, an abstract, a list of related publications.
- Go To Study navigates to the study folder and all of its data. If there are multiple versions of a study, this link becomes a dropdown to the various versions.
- Manuscript and Abstract links will be shown if any publications are associated with the study. If no publications are associated, then the card has a white fill color.
The
Publications tab displays each publication as a "long card", with a link to view the document and, if provided, a link to the study data.
The
left-side filter panel controls which cards are displayed based on selected categories. Multiple selections within a category set will add more cards. Multiple selections across category sets will narrow the cards available. The
ratio bars show the number of items currently shown vs. the number of items available.
ITN TrialShare provides
a DataFinder implementation, if you want to preview what is possible with this tool. You will need to
register for an account to view their DataFinder.
Set Up and Configuration
- Assemble your studies (= study folders) under one project. This project will become the DataFinder portal.
- Next import a set of utility lists. Many of these lists are user editable, so you can control the population of the various dropdowns that are used to configure study and publication properties. See details below.
- Download the list archive: DataFinder.lists.zip
- Import the list archive into the main project containing the individual study folders:
- Select > Manage Lists.
- Click Import List Archive.
- Browse to and import the list archive you obtained.
- This will create the lists, for example: StudyAgeGroup, StudyAssay, StudyCondition, StudyManuscripts, StudyPhase, StudyProperties, etc.
- Enable the DataFinder module in the main project:
- Go to > Folder > Management.
- Click the Folder Type tab.
- Check the box for the DataFinder module.
- Click Update Folder.
- Go to > Folder > Management and click the Data Finder tab.
- Set the dropdown Study Cube to DataFinder:/StudyCube
- Set the dropdown Publication Cube to DataFinder:/PublicationCube
- Click Submit.
- Add the Data Finder web part to your page. The frame of the data finder appears, along with any public, complete manuscripts and abstracts.
- Now you are ready to enter metadata about your studies and publications which will appear in the cards and pop-ups, as detailed below.
Studies View
The Data Finder
Studies tab shows a layout of "cards" showing the studies available to the user in the project. An administrator can add or edit studies available by clicking
Manage Data.
studyCard.PNG
Manage Studies
- Click Manage Data from the Data Finder Studies tab. Hover to reveal (Details) and (Edit) links for each study.
manageData.PNG
- If you are configuring a new DataFinder portal, you can delete the initial data in "Manage Data". Select all of the rows, and click the Delete button (trash can icon).
- Click (Insert New Row) above the grid to add a new study.
- Enter or edit information about the study, including the name, ID, title, type, and details. Required fields are marked with an asterisk *. The following screenshots illustrate how these elements appear in the user interface:
Study Card
df2.png
Summary Popup Pane
df1.png
- Here you also specify how users access your study. Under Study Access, click Add to define a combination of:
- Visibility: Operational or Public
- Study Container: where the study is located
- Display name: How to show the name on the card in the data finder.
You can have multiple combinations for any given study. Remove one no longer in use by clicking the 'X' for the section.
studyAccess.PNGClick
Save and Close after entering the information for the study you are adding. Adding studies automatically
refreshes the cube.
Study Visibility
The Data Finder respects folder-level user permissions to prevent showing studies, links, and filters that a user is not allowed to see.
For a study card to show up in the finder, all of these things must be true:
- There is at least one StudyContainer associated with this study in the StudyAccess list.
- User has Read permission to one of the containers that is associated with that study in the StudyAccess list.
- The Visibility field associated with that study's StudyContainer is either "Public" or "Operational" (case-sensitive)
- The study satisfies the filter criteria set in the finder.
If a trial has both a public and an operational folder associated with it in the StudyAccess list, the user will see counts in the Visibility dimension that reflect which they have read access to. If a user has access to more than one folder for a particular study, clicking on the “Go To Study” link for that card will produce a menu where the user can choose among the different containers to go to. If a user has no read permission to any operational folders associated with any studies, the Visibility facet, used to choose public or operational studies, will not be shown.
Note that the lists that populate the finders are effectively public (all rows are visible to all logged in users) so users could stumble across the data; we prevent showing any card versions of this data.
Showing Principal Investigators
On the study cards, the names of the principal investigators will be shown below the study name if the names are populated in the "Investigator" column of the StudyProperties list. In the screencap below, only the Casale study has this field populated.
studyCard.PNG
Showing Study Details
When you click on "View Summary" from a study card, the pop-up that appears will display the study description at the top if the description is present in the "Description" column of the StudyProperties list. You will also see any manuscripts and abstracts listed, and links to the study and publications.
Showing Study Icons
On the "View Summary" pop-up, the icon for the study will be displayed above the title of the study if the corresponding iconUrl field in the StudyProperties list is populated.
Data Display and Link Construction
Study Cards
- "Go To Study" - This link appears if the user has read permission to a StudyContainer in the StudyAccess list. If more than one study container is associated with the trial study, clicking this link will show a dropdown menu from which you can choose the study folder to go to.
- X Manuscripts Available - The X value is the number of items the user can access in the ManuscriptsAndAbstracts list that have a PublicationType of "Manuscript".
- Y Abstracts Available - The Y value is the number of items the user can access in the ManuscriptsAndAbstracts list that have a PublicationType of "Abstract".
Study Details
- Links below the publication citation - There is a link created for each item from the ManuscriptsAndAbstracts list for which there is a non-empty corresponding Description text, e.g "DocumentURL", "Link2", and "Link3". If the description is empty, the link is not shown.
- "View Study <Brand>" - This link is shown if there is an associated StudyContainer in the StudyAccess list. If there is more than one study container associated with this <Brand>, we show the last operational study the user has access to, if there is one, and otherwise we show the first public study, where "first" and "last" are determined by the ordering of the StudyAccess list.
- "View Study at <place>" - Displays the link provided in the ExternalURL field in the StudyProperties list, if any. The link text is either the text provided in the ExternalUrlDescription or, if the description is empty, the words "View study at <domain of ExternalURL>" (e.g., "View study at clinicalTrials.gov")
- The description on the details page will be pulled from the Description field in the StudyProperties list.
Card Highlighting
Study cards are highlighted with a blue background if there are any manuscripts or abstracts available.
Manuscripts View
The
Publications tab in the data finder shows a catalog of publications available to the user. Similar "cards" show the publication title, author, type (abstract or manuscript), and other brief details. Click the
icon to expand the card to show additional information and links.
pubCard.PNG
Manuscript Visibility
The manuscript finder must respect user permissions to prevent showing publications, abstracts, links, and filters that a user is not allowed to see. When a user navigates to the
Publications tab, if they are a public user, they will see by default only "Complete" manuscripts. An internal user sees the "In Progress" manuscripts by default.
For a publication to show up in the finder all of these things must be true:
- Show from the ManuscriptsAndAbstracts list is true.
- The publication satisfies the filter criteria selected in the finder.
Manage Publications
An administrator can edit display information or add additional publications by clicking
Manage Data. Hover over a row to reveal links for
(Edit) and
(Details). Click
(Insert New Row) to add a new one. Click
Save and Close after adding or editing information. Adding publications automatically
refreshes the cube. Return to the
Portal tab.
The following screenshots illustrate how properties are displayed in the publication cards.
Collapsed Publication Card
df3.png
Expanded Publication Card
df4.png
Showing Manuscript Identifiers
The details display for a manuscript will include links to PubMed, PubMed Central, and DOI.org if the corresponding fields in the MansucriptsAndAbstracts list are filled in for that publication.
Showing Figure Thumbnails
When you display the details for a publication, a sequence of thumbnails will be displayed if you have associated a ManuscriptContainer with this publication in the ManuscriptsAndAbstracts list and that ManuscriptContainer has a report category whose label contains the text "Manuscript Figures". All thumbnails for all reports in that category and its subcategories will be displayed when the details are shown.
Showing Abstract and Keywords
When you display the details for a publication, the abstract will be displayed if you have provided text for this abstract in the ManuscriptsAndAbstracts list. Similarly, if the keywords field for that publication in the ManuscriptsAndAbstracts list has been populated, these will be shown in the details view as well.
Data Display and Link Construction
Publication Cards
- "View Document" - This link will be the first of the "Document URL", "Link2", "Link3" fields that is not empty.
- The type of publication (Manuscript or Abstract) is shown; for internal users, manuscripts "In Progress" will display that information. Public users will only see completed manuscripts by default.
- "Edit" - This link will appear to administrators who can edit the card information shown.
- "Clinical and Assay Data" - This link is constructed using the ManuscriptContainer field in the ManuscriptsAndAbstracts list, if present. It is a link to the "study.DATA_ANALYSIS" page for that study.
Publication Details
- PubMed - Link text is PMID if provided in the corresponding field in the ManuscriptsAndAbstracts list
- PubMed Central - Link text is PMCID if provided in the corresponding field in the ManuscriptsAndAbstracts list
- DOI - Link text is the DOI if provided in the corresponding field in the ManuscriptAndAbstracts list
- Study links - The studies listed are linked if there is an associated operational study container in the PublicationStudy list and the user has access to the operational study
- Links to the publication - There is a link created for each of "DocumentURL", "Link2", and "Link3" from the ManuscriptsAndAbstracts list for which there is a non-empty corresponding Description text. If the description is empty, the link is not shown.
- Thumbnails - The thumbnails are collected from all reports associated with the ManuscriptContainer field that are in a category whose label contains "Manuscript Figures". The links from the thumbnails are to the corresponding report.
We do not currently have special logic for displaying links to draft manuscripts, but if you populate one of the "DocumentURL", "Link2", "Link3" fields with a link to the draft and set the corresponding description text to "Draft manuscript", you will see a link to the draft displayed. If something more is wanted, please let us know.
Card Highlighting
Publication cards are highlighted with a blue background if the publication is designated as "In progress".
Relocatable Finder and Cube Administration
If you want to locate the Data Finder in a container other than the one in which the cube is defined, or enable
search box functionality, you must set the Site Default value:
- Navigate to the container in which the DataFinder module is enabled.
- Select > Folder > Management.
- Select ModuleProperties.
- Set the Site Default value of the "DataFinderCubeContainer" to the path of the folder in which the cube is defined (e.g., /Studies)
Cube Administration
To facilitate maintenance of the cube data, use the
Data Cube link under the
Management section of the Admin Console. This link brings up a page that allows you to clear the cache for the data cube. This relies on the use of DataFinderCubeContainer module property, so even if you have not relocated the data finder web part, you should set the Site Default value for the module property so that the proper container is associated with the cube definitions.
You can also
Reindex the data cube.
Data Finder Search
The Data Finder provides quick filtering and navigation to studies and manuscripts. The search box adds an additional filtering facet using context-sensitive results of a text search to further narrow the set of studies or manuscripts shown. Cards shown must be both present in the search, and in results of applied filters.
For example, searching on "Asare" and applying a filter for "autoimmune" would list the set of manuscripts authored by Adam Asare which are also tagged "autoimmune" in therapeutic area.
Data finder search leverages the standard Lucene indexing and search capabilities, but the documents are identified with a custom search category (e.g., “DataFinderStudy” and “DataFinderManuscript”, which is used to limit searches to just the studies or manuscripts (as appropriate).
Enable Search
First set the Site Default module property in the folder where the TrialShareDataFinder module is enabled, as described
above. Then index the documents available:
- Go to > Site > Admin Console.
- Click Settings.
- Under Management click Data Cube.
- Click Reindex.
Reindexing always indexes all of the documents. There is no use of lastIndex since the amount of data to index is relatively small and the process of tracking updates across what is essentially a left join across many tables is complex.
Details for Search Document Providers
- The document providers specify, for each document, the container that should be checked for read permissions to determine whether the current user can see the result or not. In the data finder case, most of the metadata resides in lists in a single folder. The document providers must use the container column in the lists as the official container for each document.
- A data finder study document provider. Fields to be indexed and treatment:
- Study short name: medium priority keyword
- Study identifier: medium priority identifier
- Investigator(s): medium priority keyword
- Study title: medium priority keyword
- Study description: body
- Study keywords: body
- All faceted field values: body
- A data finder manuscript document provider. Fields to be indexed and treatment:
- Manuscript title: medium priority keyword
- PMID, PMCID, StudyId: medium priority identifier
- DOI: medium priority keyword
- Authors: medium priority keyword
- Abstract: body
- All faceted field values: body
Identifiers (e.g., study identifiers, PMCID) are indexed without stemming; body and keyword text is indexed with stemming.
List Details
To see the full set of lists used by the Data Finder, select
> Manage Lists. The following lists are directly editable by administrators. Values entered in these lists will populate the dropdowns for study and publication characteristics. Other lists are generally mappings that should not be directly edited; instead these mapping lists are controlled by entering data in the study and publication metadata forms.
User-Controlled Lists
- AgeGroup - Age categories available, Adult, Child, etc.
- Condition - Disease conditions available.
- Phase - Study phase type, Phase 1, Phase 2, etc.
- PublicationType - Publication catagories.
- StudyType - Interventional, Operational, etc.
- Theraputic Area - Alergy, AutoImmune, etc.
- Visibility - Operational or Public
Mapping Lists
- StudyAgeGroup - Maps between studyId and the age groups
- StudyCondition - Maps between the StudyId and the various conditions
- StudyPhase - Maps between the StudyId and the various phases
- StudyAccess - Maps between studyId and the containers associated with a particular trial. The Visibility field can be used to designated either "Public" or "Operational" access. These values are case-sensitive.
- StudyTherapeuticArea - Maps between the studyId and the various therapeutic areas.
Custom Cube Categories
To control the categories for studies and publications:
- Create 2 Lists:
- Base categories list
- Junction list that maps categories to studies (or publications)
- Supply your own category XML file
- Select that file on the DataFinder tab
First create two lists using the supplied lists as models. For example, these two lists work together:
- Phase - contains a single column of values: "Phase 1", "Phase 2", etc.
- StudyPhase - contains mappings between Studies and the phases above.
| Study Id | Phase |
|---|
| Study 1 Short Name | Phase 1 |
| Study 2 Short Name | Phase 0 |
| Study 3 Short Name | Phase 2 |
| Study 4 Short Name | Phase 4 |
Next configure the XML file to capture these tables. Clone the following example XML file as a starting point:
StudyCube.xmlTo supply a category XML file, package it in a file-based module, with the following directory structure:
MyCubeModule (note this directory can be any name)
resources
olap
MyCube.xml (this file can be any name)
Deploy this module to the server, and enable it in your DataFinder project.
Go to Gear > Folder > Folder Management and select the DataFinder tab. Select this file for you studies, publications, or both.
df5.png
Troubleshooting
"ConfigId must be provided to retrieve a cube definition" - To clear this error, configure the study and publication cubes at
> Folder > Management and click the
Data Finder tab.