[Archivesspace_Users_Group] Spreadsheet imports locking up database

[James Bullen]

Hi Peter and Nancy,

Bulk update operations usually run in database transactions. This is great because it guarantees that the database will be in a consistent state when the job completes - if it succeeds then all of the updates are applied, if it fails then none of the updates are applied.

The price for this guarantee is that any affected records are locked until the job completes.

We recently implemented the ability to submit import jobs to be run after hours for a client to work around this.

Ideally, there would be some indication in the UI that a record is locked (maybe hiding the edit button and a helpful message), but we haven’t investigated this.

Hope that helps.


Cheers,
James



> On Jun 5, 2021, at 1:43 AM, Kennedy, Nancy <KennedyN at si.edu> wrote:
> 
> Hi Peter –
> We also see issues with concurrent database access.  If one user initiates a large delete or import, other users can experience long delays and timeouts.  We’ve also noticed that Edit save times are especially long when there is a language subrecord. (An archival_object with a lang_material can take 30 seconds to save, whereas records without a lang_material save much more quickly).
>  
> We are in v 2.71 and we also use MariaDB. We see the same issue in our test 2.8.1 instance. I’d be very interested to hear if MySQL offers a better experience for Cambridge,
>  
> Thanks, 
> Nancy
>  
> kennedyn at si.edu <mailto:kennedyn at si.edu>
> Nancy Kennedy
> Smithsonian Institution
>  
>  
>  
> From: archivesspace_users_group-bounces at lyralists.lyrasis.org <mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> <archivesspace_users_group-bounces at lyralists.lyrasis.org <mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>> On Behalf Of Peter Heiner
> Sent: Friday, June 4, 2021 11:20 AM
> To: archivesspace_users_group at lyralists.lyrasis.org <mailto:archivesspace_users_group at lyralists.lyrasis.org>
> Subject: [Archivesspace_Users_Group] Spreadsheet imports locking up database
>  
> External Email - Exercise Caution
> Dear list users,
>  
> The Cambridge University Library AS instance has recently been hit with severe performance issues while users were running spreadsheet imports to add archival objects, details are in https://github.com/archivesspace/archivesspace/issues/2275 <https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Farchivesspace%2Farchivesspace%2Fissues%2F2275&data=04%7C01%7CKennedyN%40si.edu%7C0d243c2fdc70417e30f508d9276c3359%7C989b5e2a14e44efe93b78cdd5fc5d11c%7C0%7C0%7C637584167938585160%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PAgCNJJNM%2BSFNRvt3r6M5zysCkIJTfO5V6geI6%2BRNLY%3D&reserved=0>; the summary is that a DELETE query with a dependent subquery running for every line of the spreadsheet input seems to make concurrent database access near impossible.
>  
> Has anyone else experienced anything similar? Us using MariaDB instead of MySQL might well be the cause, but I'd like confirmation before work is scheduled to migrate the database that this will indeed help. If you're using spreadsheet imports, we'd be grateful if you could let us know either on or off list whether you use MySQL or MariaDB and if you had any problems with the migration queries hogging the database.
>  
> Thanks,
> p
> _______________________________________________
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group at lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20210606/75ceb7df/attachment.html>