Hi Tobin,
We would love to support incremental imports... for specimens (which I'm sure is increasingly a pain point for SCHARP) as well as for lists and datasets. Since most of our clients aren't able to generate incremental archives, we would like to take this a step beyond what you describe. Our hope is to generate an incremental diff by analyzing a full archive against the previously loaded data, then processing that diff to perform the import in a tiny fraction of the time (assuming most data doesn't change between imports). The diff could also be used to provide more meaningful notifications, audit logging, and tracking, by showing exactly what data changed from import to import.
We've prototyped this approach, proving that it's viable and fast. We're now talking to clients about turning this concept into a production feature. Last week, I put this topic on the agenda for our next conversation with SCHARP leadership (May 1). You could join that meeting and/or we can discuss offline if you'd like. I'll reach out via email.
Adam