[Archivesspace_Users_Group] Larger records and OAI - 3.2.0

[Mark Cooper]

Hi Tom,

There have been some optimizations made since 3.2.0 that will probably help you with this. You can wait for the next release or try testing one of the more recent master builds:

https://github.com/archivesspace/archivesspace/actions/workflows/build.yml
Most recent:
https://github.com/archivesspace/archivesspace/suites/5867861199/artifacts/197930705

Best,
Mark

Mark Cooper
Technical Lead, Hosting and Support
LYRASIS

________________________________
From: archivesspace_users_group-bounces at lyralists.lyrasis.org <archivesspace_users_group-bounces at lyralists.lyrasis.org> on behalf of Tom Hanstra <hanstra at nd.edu>
Sent: Wednesday, April 13, 2022 11:39 AM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org>
Subject: [Archivesspace_Users_Group] Larger records and OAI - 3.2.0

We are still having difficulties with larger records in OAI searches/responses while using AS 3.2.0. I've just completed running through the soft indexing process to be sure Solr (8.11.1) and AS are in sync. But I'm still running into problems.

When I do these searches, I'm not getting any error messages in any of the AS logs nor in the Solr logs. Small records come back reasonably quickly, but larger records do not. They basically time out after a long period of time.

Watching the server, the load average ticks up during the searches overall, but most of the work is being done by AS. Solr does not seem to be using much CPU during this time. I also tried increasing the database server CPU to see if that would help, but it doesn't seem to be helping.

I'd like to better understand what has to be done when attempting an OAI request. What services get hit the hardest?  Where might I have a bottleneck that I've not yet addressed? The fact that smaller records work but larger records do not indicates to me that the issue is not with Solr, but the lack of any errors in the logs seems to indicate that ArchivesSpace is doing what it is supposed to. So where might things be falling apart?

Thanks,
Tom
--
Tom Hanstra
Sr. Systems Administrator
hanstra at nd.edu<mailto:hanstra at nd.edu>

[https://docs.google.com/uc?export=download&id=1GFX1KaaMTtQ2Kg2u8bMXt1YwBp96bvf0&revid=0B7APN9POn6xAQ244WWFYMFU3aVJwZ0lxbmVHK3FxNXlCd0RRPQ]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20220413/c65e51f6/attachment.html>