<div dir="ltr">Thanks for the suggestion, Blake. A couple additional questions:<br><div><br></div><div>- Staff indexing is done and has its files written. So does the number of threads given to that make a difference? Is it still taking up resources?</div><div><br></div><div>- Does there happen to be any way to stop the staff indexing and just let PUI have full access to the server for indexing? </div><div><br></div><div>- What really bothers me is the slowdown. That indicates to me that some resource is being lost along the way. Anyone have thoughts on what that might be?<br></div><div><div><br></div><div>- Should our repositories be broken up into smaller groupings? I'm beginning to wonder if we have things set up incorrectly, since it sounds like we have a very large data set compared to others.<br></div><div><br></div><div><br></div><div>And a comment....<br><br>It is really frustrating to have to start over on the indexing each time. It seems that there should be some way to document progress along the way so that the indexing can pick up where it left off. Is that something that might also be looked at?</div><div><br></div><div>Thanks all. Appreciate your help.</div><div><br></div><div>Tom</div><div><br></div><div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Mar 16, 2021 at 1:15 PM Blake Carver <<a href="mailto:blake.carver@lyrasis.org" target="_blank">blake.carver@lyrasis.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
> <span style="color:rgb(32,31,30);font-size:15px;background-color:rgb(255,255,255);display:inline">I've now left
my PUI indexing threads and count at the default (which I believe is 1 thread and 25 records/thread).</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Try dropping both indexer_records_per_thread and indexer_thread_count for both PUI and Staff indexers. Maybe in half or so. Sometimes with larger records it just needs to be slowed down. </div>
<div>
<div id="gmail-m_7722645105473406696gmail-m_582999336626351475Signature">
<div>
<div id="gmail-m_7722645105473406696gmail-m_582999336626351475divtagdefaultwrapper" dir="ltr" style="color:rgb(0,0,0);background-color:rgb(255,255,255)">
</div>
</div>
</div>
</div>
<div id="gmail-m_7722645105473406696gmail-m_582999336626351475appendonsend"></div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_7722645105473406696gmail-m_582999336626351475divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> <a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org" target="_blank">archivesspace_users_group-bounces@lyralists.lyrasis.org</a> <<a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org" target="_blank">archivesspace_users_group-bounces@lyralists.lyrasis.org</a>> on behalf of Tom Hanstra
<<a href="mailto:hanstra@nd.edu" target="_blank">hanstra@nd.edu</a>><br>
<b>Sent:</b> Tuesday, March 16, 2021 12:51 PM<br>
<b>To:</b> Archivesspace Users Group <<a href="mailto:archivesspace_users_group@lyralists.lyrasis.org" target="_blank">archivesspace_users_group@lyralists.lyrasis.org</a>><br>
<b>Subject:</b> [Archivesspace_Users_Group] PUI indexing issues</font>
<div> </div>
</div>
<div>
<div dir="ltr">Hello again.
<div><br>
</div>
<div>I'm still trying to understand some indexing issues. I've now left my PUI indexing threads and count at the default (which I believe is 1 thread and 25 records/thread). And I have given 4GB to Java processes. I've tried other values as well, but with similar
results.</div>
<div><br>
</div>
<div>No matter what values I use, I cannot seem to fully index PUI. Each time, it will start well but continuously slow down. I've kept a spreadsheet of the number of records/hr I'm indexing and have several attempts which start in the 50-60K/hr range and then
continuously slow down to the 1800-1500/hr speed until finally dying with a Java Heap error. I think I'm headed to that again this round.</div>
<div><br>
</div>
<div>Why might this be happening? Could my data have been corrupted during the transfer from Lyrasis? (I'm working with a database export of our production data). Is the database too far away (our database is in an AWS RDS being accessed from our AWS EC2).</div>
<div><br>
</div>
<div>I do have one log which gave this error:<br>
<br>
E, [2021-03-12T18:14:53.886243 #2919] ERROR -- : Thread-9472: Failed fetching archival_object id=1484623: too many connection resets (due to Net::ReadTimeout - Net::ReadTimeout) after 0 requests on 3150, last used 1615590893.870297 seconds<br>
ago<br clear="all">
<div><br>
</div>
<div>prior to the Java Heap error. In that log, there were a number of connections for the staff indexer after the PUI indexer stopped reporting, then an 88 minute gap prior to the above connection error and then finally a Java Heap error in the archivesspace.out
log.</div>
<div><br>
</div>
<div>Does the indexer reauthenticate each time it connects to get more information? The earlier question about authentication has me wondering if my database server might be balking at the number of reconnections or something. I'm trying to index 760K records.</div>
<div><br>
</div>
<div>Bottom line is that I'm still not getting my PUI index creation to complete. Each run can take several days before it finally fails and I have to start all over again. I'm looking for any help to track down why this slowdown is occurring and what I can
do to address it.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Tom</div>
-- <br>
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div dir="ltr">
<div><b style="font-family:arial,helvetica,sans-serif;font-size:12.7273px;color:rgb(136,136,136)">Tom Hanstra</b><br>
</div>
<div style="color:rgb(136,136,136);font-size:12.8px">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div style="font-size:12.7273px">
<div>
<div><i style="font-size:12.7273px;font-family:arial,helvetica,sans-serif">Sr. Systems Administrator</i></div>
<div><a href="mailto:hanstra@nd.edu" style="color:rgb(17,85,204);font-size:12.7273px;font-family:arial,helvetica,sans-serif" target="_blank">hanstra@nd.edu</a><br>
</div>
</div>
<div><span style="font-family:arial,helvetica,sans-serif"><br>
</span></div>
</div>
<div style="font-size:12.7273px"><img src="https://docs.google.com/uc?export=download&id=1GFX1KaaMTtQ2Kg2u8bMXt1YwBp96bvf0&revid=0B7APN9POn6xAQ244WWFYMFU3aVJwZ0lxbmVHK3FxNXlCd0RRPQ"><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
_______________________________________________<br>
Archivesspace_Users_Group mailing list<br>
<a href="mailto:Archivesspace_Users_Group@lyralists.lyrasis.org" target="_blank">Archivesspace_Users_Group@lyralists.lyrasis.org</a><br>
<a href="http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group" rel="noreferrer" target="_blank">http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div><b style="font-family:arial,helvetica,sans-serif;font-size:12.7273px;color:rgb(136,136,136)">Tom Hanstra</b><br></div><div style="color:rgb(136,136,136);font-size:12.8px"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div style="font-size:12.7273px"><div><div><i style="font-size:12.7273px;font-family:arial,helvetica,sans-serif">Sr. Systems Administrator</i></div><div><a href="mailto:hanstra@nd.edu" style="color:rgb(17,85,204);font-size:12.7273px;font-family:arial,helvetica,sans-serif" target="_blank">hanstra@nd.edu</a><br></div></div><div><span style="font-family:arial,helvetica,sans-serif"><br></span></div></div><div style="font-size:12.7273px"><img src="https://docs.google.com/uc?export=download&id=1GFX1KaaMTtQ2Kg2u8bMXt1YwBp96bvf0&revid=0B7APN9POn6xAQ244WWFYMFU3aVJwZ0lxbmVHK3FxNXlCd0RRPQ"><br></div></div></div></div></div></div></div></div></div></div></div>