<div dir="ltr"><div>Thanks all! <br></div><div><br></div><div>And thanks Mark; it's good to know that large clusters of linked resources can become an issue, as we definitely have records that include hundreds of correspondents linked as agents. <br></div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr">On Fri, Oct 5, 2018 at 5:08 PM Custer, Mark <<a href="mailto:mark.custer@yale.edu">mark.custer@yale.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<font face="Calibri" size="2"><span style="font-size:11pt">
<div>Kevin,</div>
<div> </div>
<div>In addition to what Nancy and Steve have already mentioned, I'll add a few other things to consider about the size of the records:</div>
<div> </div>
<ul style="margin:0;padding-left:36pt">
<li>Right now, a note cannot exceed 65k characters in ArchivesSpace. The overall file might not even be that large, but if you have any really, really, really long notes, you won't be able to import the file without splitting up the note.</li><li>If you have a single archival object that's linked to, say, 1,000 or more digital objects, top containers, subjects, agents, or the like, then things might not behave so well in the staff interface, etc. I've found the number of linked records clustered
on a single record to be more problematic than a deeply hierarchical finding aid. As long as you don't have many (or any) linked to over 500 things, I'd say that's good for ASpace. We have a few above that, but not many. </li><li>The tree view for finding aids has also recently been optimized to work better for really flat listing, as well. See these good-natured notes for a great explanation of the work done behind the scenes to accommodate all sorts of different finding aids:
<a href="https://github.com/archivesspace/archivesspace/blob/615fe43e946cbf672e2c35a6c9b79ac6b11a136a/backend/app/model/large_tree.rb#L1-L49" target="_blank"><font color="#0563C1"><u>https://github.com/archivesspace/archivesspace/blob/615fe43e946cbf672e2c35a6c9b79ac6b11a136a/backend/app/model/large_tree.rb#L1-L49</u></font></a>
</li></ul>
<div> </div>
<div>I can also say that we don't have any finding aids as large as 100MB <font face="Segoe UI Emoji">😊</font><font face="Segoe UI Emoji"> </font>We have a few that run the gamut from 10 - 40 MB, though, and those are working well. </div>
<div> </div>
<div>Mark</div>
<div> </div>
<div> </div>
<div> </div>
<div>-----Original Message-----<br>
From: <a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org" target="_blank">archivesspace_users_group-bounces@lyralists.lyrasis.org</a> [<a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org" target="_blank">mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org</a>] On Behalf Of Kennedy, Nancy<br>
Sent: Friday, 05 October, 2018 4:08 PM<br>
To: Archivesspace Users Group <<a href="mailto:archivesspace_users_group@lyralists.lyrasis.org" target="_blank">archivesspace_users_group@lyralists.lyrasis.org</a>><br>
Subject: Re: [Archivesspace_Users_Group] Maximum finding aid size</div>
<div> </div>
<div>We also have finding aids that large (about 10 or so that are larger than 10MB). I'd definitely recommend 2.5 if you will need to work with large records. Prior to updating to 2.5, the load times could mean long wait times to view or edit. In the last
few weeks, we've added a record that is now over 100MB, and takes about 15 minutes to export to EAD.</div>
<div> </div>
<div>In the past, in previous systems, we were unable to keep records of this size united and had split a few very large records into separate resources due to size. It's always been hard to re-unite / present that scenario in discovery systems. So far, archivesspace
is doing a better job of scaling to handle our size extremes.</div>
<div> </div>
<div>Nancy</div>
<div> </div>
<div> </div>
<div> </div>
<div> </div>
<div> </div>
<div> </div>
<div>-----Original Message-----</div>
<div>From: <a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org" target="_blank">archivesspace_users_group-bounces@lyralists.lyrasis.org</a> <<a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org" target="_blank">archivesspace_users_group-bounces@lyralists.lyrasis.org</a>>
On Behalf Of Majewski, Steven Dennis (sdm7g)</div>
<div>Sent: Friday, October 05, 2018 3:28 PM</div>
<div>To: Archivesspace Users Group <<a href="mailto:archivesspace_users_group@lyralists.lyrasis.org" target="_blank">archivesspace_users_group@lyralists.lyrasis.org</a>></div>
<div>Subject: Re: [Archivesspace_Users_Group] Maximum finding aid size</div>
<div> </div>
<div> </div>
<div>I’ve recently imported a 25MB EAD file as well as some around 10-15MB. </div>
<div>It took a significant amount of time to ingest — about an hour. </div>
<div>This was on a test server with minimal load other than that job, so you may want to schedule your ingest at off hours. </div>
<div>Definitely do it on a test server first, because it’s difficult to back out of if you need to fix something and try again. </div>
<div> </div>
<div>( This was using the current version (v2.5.0). Some very early versions were unacceptably slow. )</div>
<div> </div>
<div> </div>
<div>I think the primary overhead is in creating objects, more than the EAD parsing, so it probably scales by some other order of complexity rather than just text size. </div>
<div>i.e. creating a very large bioghist or other note is still only creating a single object and is probably a single MySQL operation, while turning a long list of agent names into agent records and linking them to the resource is many operations. </div>
<div> </div>
<div> </div>
<div>— Steve Majewski</div>
<div> </div>
<div> </div>
<div> </div>
<div>> On Oct 5, 2018, at 3:00 PM, Kevin W. Schlottmann <<a href="mailto:kws2126@columbia.edu" target="_blank">kws2126@columbia.edu</a>> wrote:</div>
<div>> </div>
<div>> Hi all,</div>
<div>> </div>
<div>> Does anyone have a sense of where the size limitations are in </div>
<div>> ArchivesSpace? We have some very large finding aids (10+ MB EAD) that </div>
<div>> we hope to import, manage, and publish using AS. I would image that </div>
<div>> other institutions have large finding aids, and I'm curious to know </div>
<div>> what issues we might run into.</div>
<div>> </div>
<div>> Best,</div>
<div>> </div>
<div>> Kevin</div>
<div>> --</div>
<div>> Kevin Schlottmann</div>
<div>> Head of Archives Processing</div>
<div>> Rare Book & Manuscript Library</div>
<div>> Butler Library, Room 801</div>
<div>> Columbia University</div>
<div>> 535 W. 114th St., New York, NY 10027</div>
<div>> (212) 854-8483</div>
<div>> </div>
<div>> _______________________________________________</div>
<div>> Archivesspace_Users_Group mailing list </div>
<div>> <a href="mailto:Archivesspace_Users_Group@lyralists.lyrasis.org" target="_blank">Archivesspace_Users_Group@lyralists.lyrasis.org</a></div>
<div>> <a href="https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flyrali" target="_blank">https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flyrali</a></div>
<div>> <a href="http://sts.lyrasis.org" target="_blank">sts.lyrasis.org</a>%2Fmailman%2Flistinfo%2Farchivesspace_users_grou&da</div>
<div>> ta=02%7C01%7Cmark.custer%<a href="http://40yale.edu" target="_blank">40yale.edu</a>%7C4ceb88a237be4d5f3f5e08d62afe373d</div>
<div>> %7Cdd8cbebb21394df8b4114e3e87abeb5c%7C0%7C0%7C636743668695267673&s</div>
<div>> data=mbglfUrCPnQFTF1VkDRbbm1hd7nJnr%2F%2B4%2F%2BMFgRKi6A%3D&reserv</div>
<div>> ed=0</div>
<div>> p</div>
<div> </div>
<div>_______________________________________________</div>
<div>Archivesspace_Users_Group mailing list</div>
<div><a href="mailto:Archivesspace_Users_Group@lyralists.lyrasis.org" target="_blank">Archivesspace_Users_Group@lyralists.lyrasis.org</a></div>
<div><a href="https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flyralists.lyrasis.org%2Fmailman%2Flistinfo%2Farchivesspace_users_group&data=02%7C01%7Cmark.custer%40yale.edu%7C4ceb88a237be4d5f3f5e08d62afe373d%7Cdd8cbebb21394df8b4114e3e87abeb5c%7C0%7C0%7C636743668695267673&sdata=fD%2FaVX3oK%2FW%2FrvpBVZxy23ofjIixdmTtHyHCSlIYqzA%3D&reserved=0" target="_blank">https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flyralists.lyrasis.org%2Fmailman%2Flistinfo%2Farchivesspace_users_group&data=02%7C01%7Cmark.custer%40yale.edu%7C4ceb88a237be4d5f3f5e08d62afe373d%7Cdd8cbebb21394df8b4114e3e87abeb5c%7C0%7C0%7C636743668695267673&sdata=fD%2FaVX3oK%2FW%2FrvpBVZxy23ofjIixdmTtHyHCSlIYqzA%3D&reserved=0</a></div>
<div> </div>
</span></font>
</div>
_______________________________________________<br>
Archivesspace_Users_Group mailing list<br>
<a href="mailto:Archivesspace_Users_Group@lyralists.lyrasis.org" target="_blank">Archivesspace_Users_Group@lyralists.lyrasis.org</a><br>
<a href="http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group" rel="noreferrer" target="_blank">http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group</a><br>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature">Kevin Schlottmann<br>Head of Archives Processing<br>Rare Book & Manuscript Library<br>Butler Library, Room 801<br>Columbia University<br>535 W. 114th St., New York, NY 10027<br>(212) 854-8483</div>