[Archivesspace_Users_Group] ArchivesSpace performance problems and solutions

[Chris Fitzpatrick]

Hi Maureen,

Thanks for this. It's very helpful, since HM's work is always thorough and top-notch. 

As luck would have it, I actually committed come code today that implements some of these suggestions.  You can see the Jira ticket here: 
https://archivesspace.atlassian.net/browse/AR-1311
which has a link to the branch with the comments. 

In a nutshell, associated records ( like notes, subjects, agents ) will not be initialized by the form if they have more than 5 subrecords. In these cases,  users can load these records by clicking on subsection. Not having to load the linker, CodeMirror, and other validations really reduces the work the browser has to do.
I've been using a test record that Noah Huffman gave me, which is about 25mb in size. 

I'm doing some additional work to optimize the callbacks and events, especially with giving the user a sense as to what's actually happening. 

I'm still working out some of the kinks and adding tests, but hoping to put this into the next RC.  Please feel free to checkout the code and give it a test against your data.    

And, as always, pull-requests are welcome. 

best, chris. 


Chris Fitzpatrick | Developer, ArchivesSpace
Skype: chrisfitzpat  | Phone: 918.236.6048
http://archivesspace.org/

________________________________________
From: archivesspace_users_group-bounces at lyralists.lyrasis.org <archivesspace_users_group-bounces at lyralists.lyrasis.org> on behalf of Callahan, Maureen <maureen.callahan at yale.edu>
Sent: Friday, August 14, 2015 11:20 PM
To: tadams at cshl.edu; christian.dupont at bc.edu; gpgaidmore at wm.edu; lkachure at richmond.edu; Sniffin-Marinoff, Megan; carol.mandel at nyu.edu; Robert Miller; becs at ucsd.edu; jpwilkin at illinois.edu; Angela Spinazze; Jay Schafer; Brad Westbrook
Cc: yulaspace at mailman.yale.edu; archivesspace at googlegroups.com; Archivesspace Users Group
Subject: [Archivesspace_Users_Group] ArchivesSpace performance problems and     solutions

To the Members of the ArchivesSpace Governance Board:

As ArchivesSpace member institutions shift from the task of migrating data to the utilization of ArchivesSpace for production archival management, endemic issues related to performance and scalability are severely limiting our ability to fully utilize the system.  Please find the attached report, created by Hudson Molonglo and sponsored by Yale, which outlines a set of sensible next steps to provide immediate relief to some of the problems that users of ArchivesSpace have been experiencing. As members, we are eager to participate in the continued improvement of the application. We see the areas targeted in the report as critical and a necessary first step in addressing performance concerns. We would like to see development to address these concerns happen immediately.

Collectively, many of our repositories are experiencing significant performance problems.  In some cases, members are seeing work grind to a halt as accession records take as long as twenty seconds to save, archival object records take as long as ten seconds to save, and records with many links (agents and subjects, digital objects, sibling records) may take as long as ten minutes to load, or never load at all. Without better performance from ArchivesSpace, we are hampered in our mission of providing meaningful descriptions of records for our users.

We think that you will find Hudson Molonglo’s explanation of these problems and proposed solutions helpful and actionable. While the normal procedures of voting on feature prioritization generally work very well for developing new parts of the application, critical issues like these cannot wait for the next round of voting.

As ArchivesSpace continues to mature, production-critical issues will be fewer in number. Until then, it would be helpful to chart an effective path to bringing these problems to the foreground and providing the resources required to address them quickly.

Finally, we recognize that like many other projects, ArchivesSpace’s growth will be an iterative development process.  And as such, there may be areas of the system -- like the thresholds we discovered for records with many links -- where archival practices exceed the system’s current capacity. Part of the competitive advantage of ArchivesSpace must come from transparency around not only its strengths, but also its current limitations.  We request documentation that outlines known thresholds and current capacity, so that we can make responsible decisions about the tool’s implementation at our repositories.

We are eager to continue this conversation.

Respectfully submitted,

Maureen Callahan, on behalf of the Yale University ArchivesSpace Committee
Terry Reese, on behalf of The Ohio State University
Ben Goldman, on behalf of Pennsylvania State University
Chris Prom, on behalf of the University of Illinois at Urbana-Champaign