[Archivesspace_Users_Group] Exporting very large collections (OAI, EAD, PDF)

[Nick Butler]

Hi all,

We're exploring sharing our archival data with a few partners via OAI, and have been coming up against timeouts when attempting to export some of our larger collections (with, say, over 10,000 published descendant records), even when increasing the timeout threshold to ten minutes. We appreciate that 10,000 archival object records on one resource is fairly large (our largest has more like 90,000), but I can't imagine that there aren't other institutions with individual collections as large or larger (perhaps substantially so). Are others in this same boat able to export these larger collections via OAI? We've implemented a change in our local plugin which stops processing and adding child records to the oai_ead output, which allows all of the resource-level records to be generated without timing out, but does mean we miss out on being able to share multi-level descriptions.

In a similar vein we're unable to generate PDFs on our PUI for some of these larger records, or export their EAD in the staff interface. We've worked around these latter issues by adding functionality to export portions of a record tree rather than the whole thing, but it would be nice to know if this is a problem that others are facing too (or if, say, it's caused by our local setup, infrastructure, etc).

Many thanks,
Nick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20210318/51a8a864/attachment.html>