[Archivesspace_Users_Group] Maximum finding aid size

[Kennedy, Nancy]

We also have finding aids that large (about 10 or so that are larger than 10MB).  I'd definitely recommend 2.5 if you will need to work with large records.  Prior to updating to 2.5, the load times could mean long wait times to view or edit.  In the last few weeks, we've added a record that is now over 100MB, and takes about 15 minutes to export to EAD.

In the past, in previous systems, we were unable to keep records of this size united and had split a few very large records into separate resources due to size. It's always been hard to re-unite / present that scenario in discovery systems.  So far, archivesspace is doing a better job of scaling to handle our size extremes.

Nancy






-----Original Message-----
From: archivesspace_users_group-bounces at lyralists.lyrasis.org <archivesspace_users_group-bounces at lyralists.lyrasis.org> On Behalf Of Majewski, Steven Dennis (sdm7g)
Sent: Friday, October 05, 2018 3:28 PM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org>
Subject: Re: [Archivesspace_Users_Group] Maximum finding aid size


I’ve recently imported a 25MB EAD file as well as some around 10-15MB. 
It took a significant amount of time to ingest — about an hour. 
This was on a test server with minimal load other than that job, so you may want to schedule your ingest at off hours. 
Definitely do it on a test server first, because it’s difficult to back out of if you need to fix something and try again. 

( This was using the current version (v2.5.0). Some very early versions were unacceptably slow. )


I think the primary overhead is in creating objects, more than the EAD parsing, so it probably scales by some other order of complexity rather than just text size. 
i.e. creating a very large bioghist or other note is still only creating a single object and is probably a single MySQL operation, while turning a long list of agent names into agent records and linking them to the resource is many operations. 


— Steve Majewski



> On Oct 5, 2018, at 3:00 PM, Kevin W. Schlottmann <kws2126 at columbia.edu> wrote:
> 
> Hi all,
> 
> Does anyone have a sense of where the size limitations are in 
> ArchivesSpace?  We have some very large finding aids (10+ MB EAD) that 
> we hope to import, manage, and publish using AS. I would image that 
> other institutions have large finding aids, and I'm curious to know 
> what issues we might run into.
> 
> Best,
> 
> Kevin
> --
> Kevin Schlottmann
> Head of Archives Processing
> Rare Book & Manuscript Library
> Butler Library, Room 801
> Columbia University
> 535 W. 114th St., New York, NY  10027
> (212) 854-8483
> 
> _______________________________________________
> Archivesspace_Users_Group mailing list 
> Archivesspace_Users_Group at lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_grou
> p