Help with specimen import error: Cache timeout for SchemaTableInfos for labkey, exceeding 300000ms limit

LabKey Support Forum
Help with specimen import error: Cache timeout for SchemaTableInfos for labkey, exceeding 300000ms limit tstellin  2022-03-01 05:15
Status: Closed
 

On 2022-02-28 and 2022-02-22, our labkey server failed to import specimen data with an error we haven't seen before (stacktrace below). Are there any details on this error someone can provide? Is there a known way to address it?

...
28 Feb 2022 23:22:47,298 INFO : Updating vial rows 8040001 through 8050000.
28 Feb 2022 23:22:49,078 INFO : Updating processing locations on the specimen table...
28 Feb 2022 23:23:28,338 INFO : Complete.
28 Feb 2022 23:23:28,342 INFO : Updating first processed by initials on the specimen table...
28 Feb 2022 23:24:07,391 INFO : Complete.
28 Feb 2022 23:24:07,393 INFO : Updating vial availability...
28 Feb 2022 23:24:07,402 INFO : Marking vials unavailable based on Custom Query: study.UnavailableAccordingToLdms
28 Feb 2022 23:29:30,743 INFO : Deleting /services/data/hvtn/Specimen Management/specimens/220228212006615/primary_types.tsv
28 Feb 2022 23:29:30,760 INFO : Deleting /services/data/hvtn/Specimen Management/specimens/220228212006615/specimens.tsv
28 Feb 2022 23:29:30,764 INFO : Deleting /services/data/hvtn/Specimen Management/specimens/220228212006615/column_translations.tsv
28 Feb 2022 23:29:30,769 INFO : Deleting /services/data/hvtn/Specimen Management/specimens/220228212006615/additives.tsv
28 Feb 2022 23:29:30,772 INFO : Deleting /services/data/hvtn/Specimen Management/specimens/220228212006615/labware_sample_types.tsv
28 Feb 2022 23:29:30,776 INFO : Deleting /services/data/hvtn/Specimen Management/specimens/220228212006615/derivatives.tsv
28 Feb 2022 23:29:30,780 INFO : Deleting /services/data/hvtn/Specimen Management/specimens/220228212006615/labs.tsv
28 Feb 2022 23:29:30,787 INFO : Deleting /services/data/hvtn/Specimen Management/specimens/220228212006615
28 Feb 2022 23:29:30,835 INFO : Failed to complete task 'org.labkey.api.specimen.pipeline.StudyImportSpecimenTask'
28 Feb 2022 23:29:30,836 ERROR: Cache timeout for SchemaTableInfos for labkey, exceeding 300000ms limit
org.labkey.api.pipeline.PipelineJobException: Cache timeout for SchemaTableInfos for labkey, exceeding 300000ms limit
at org.labkey.api.specimen.pipeline.AbstractSpecimenTask.doImport(AbstractSpecimenTask.java:162)
at org.labkey.api.specimen.pipeline.AbstractSpecimenTask.run(AbstractSpecimenTask.java:67)
at org.labkey.api.pipeline.PipelineJob.runActiveTask(PipelineJob.java:818)
at org.labkey.api.pipeline.PipelineJob.run(PipelineJob.java:1055)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:831)
Caused by: java.lang.RuntimeException: Cache timeout for SchemaTableInfos for labkey, exceeding 300000ms limit
at org.labkey.api.cache.BlockingCache.get(BlockingCache.java:130)
at org.labkey.api.cache.BlockingCache.get(BlockingCache.java:91)
at org.labkey.api.data.SchemaTableInfoCache.get(SchemaTableInfoCache.java:49)
at org.labkey.api.data.DbScope.getTable(DbScope.java:1148)
at org.labkey.api.data.DbSchema.getTable(DbSchema.java:426)
at org.labkey.experiment.api.property.StorageProvisionerImpl.getSchemaTableInfo(StorageProvisionerImpl.java:604)
at org.labkey.experiment.api.property.StorageProvisionerImpl.createTableInfoImpl(StorageProvisionerImpl.java:580)
at org.labkey.api.exp.api.StorageProvisioner.createTableInfo(StorageProvisioner.java:71)
at org.labkey.api.specimen.model.SpecimenTablesProvider.createTableInfo(SpecimenTablesProvider.java:201)
at org.labkey.api.specimen.model.SpecimenTablesProvider.createTableInfo(SpecimenTablesProvider.java:125)
at org.labkey.api.specimen.SpecimenSchema.getTableInfoSpecimen(SpecimenSchema.java:145)
at org.labkey.api.specimen.SpecimenSchema.getTableInfoSpecimen(SpecimenSchema.java:138)
at org.labkey.study.query.SpecimenDetailTable.getSpecimenAndVialFromSQL(SpecimenDetailTable.java:404)
at org.labkey.study.query.SpecimenDetailTable.getFromSQL(SpecimenDetailTable.java:374)
at org.labkey.api.data.AbstractTableInfo.getFromSQL(AbstractTableInfo.java:313)
at org.labkey.query.sql.QueryTable._getSql(QueryTable.java:363)
at org.labkey.query.sql.QueryTable.getFromSql(QueryTable.java:267)
at org.labkey.query.sql.QTable.appendSql(QTable.java:85)
at org.labkey.query.sql.QuerySelect._getSql(QuerySelect.java:1524)
at org.labkey.query.sql.QuerySelect.getSql(QuerySelect.java:1468)
at org.labkey.query.sql.QuerySelect$2.getFromSQL(QuerySelect.java:1296)
at org.labkey.query.QueryServiceImpl.getSelectSQL(QueryServiceImpl.java:2624)
at org.labkey.query.QueryServiceImpl.getSelectSQL(QueryServiceImpl.java:2440)
at org.labkey.api.data.Table.getSelectSQL(Table.java:1055)
at org.labkey.api.specimen.importer.RequestabilityManager$CustomQueryRule.getFilterSQL(RequestabilityManager.java:564)
at org.labkey.api.specimen.importer.RequestabilityManager$RequestableRule.updateRequestability(RequestabilityManager.java:408)
at org.labkey.api.specimen.importer.RequestabilityManager.updateRequestability(RequestabilityManager.java:828)
at org.labkey.api.specimen.importer.RequestabilityManager.updateRequestability(RequestabilityManager.java:845)
at org.labkey.api.specimen.importer.SpecimenImporter.updateCalculatedSpecimenData(SpecimenImporter.java:1083)
at org.labkey.api.specimen.importer.SpecimenImporter.process(SpecimenImporter.java:390)
at org.labkey.api.specimen.importer.SpecimenImporter.process(SpecimenImporter.java:306)
at org.labkey.api.specimen.pipeline.AbstractSpecimenTask.doImport(AbstractSpecimenTask.java:132)
... 9 more