• Sign In
  1. Messages

Importing updated bulk data

LabKey Support Forum

View Message

view list print
Importing updated bulk data harris  2018-05-09 12:21
Status: Closed
 

Hello,
I am using the import bulk data functionality to load data and the initial data load works out nicely. I am looking into operationalizing this and have a couple of questions.

  1. If I want to update a full set of data (i.e., overwrite the existing dataset), the import bulk data option does not allow me to do this. I've only been able to import new rows. Is there a better way to refresh the entire dataset without having to delete first and then re-import?
  2. Does LabKey have a way to keep track of study data snapshot versions? For Phase 1 dose escalation studies, we need to do this weekly and need to refer to data snapshots from previous weeks (i.e., I want to see the data we used for last week's dose escalation decision.).
 
 
Jon (LabKey DevOps) responded:  2018-05-18 14:28
Hi Laura,

Unfortunately, when it comes to uploading via the UI, there is no ability to truncate the data from within the bulk upload process, so the manual process of deleting all rows is the only viable option within the UI.

Some of our other features like ETLs do allow the ability to truncate the data if the data is being sourced from one table to another:

https://www.labkey.org/Documentation/wiki-page.view?name=etlModule

Also, our FileWatcher, which is available as a Premium feature, allows the ability to drop files in a location, which then can be processed by a study dataset and also does truncation as well:

https://www.labkey.org/Documentation/wiki-page.view?name=fileWatcher

As for snapshots, the only snapshot functionality is the Query Snapshot feature.

https://www.labkey.org/Documentation/wiki-page.view?name=querySnapshot

You could use it to produce multiple query snapshots manually for those specific given timeframes, labeling them as such and creating a fixed snapshot of data for the respective dataset.

Regards,

Jon