[Archivesspace_Users_Group] Indexer erroring in 1.5.2

Thu Jan 12 18:08:16 EST 2017

I changes indexer values in config.rb

AppConfig[:indexer_records_per_thread] = 1
AppConfig[:indexer_thread_count] = 1

and added some extra debug output to backend logs and was able to identify which record it fails on.

Looking at that record in production 1.4.2, it’s not obvious to me that there’s a significant different that causes this one to fail.

However, when I look at it in the existing test 1.5.1 server, I see that it is missing info in the instance. The highlighted archival_object should say “Box: Artifacts 1” with a barcode just as the record below does. So perhaps something was failing silently in the 1.5.1 migration, which only now triggers an error in 1.5.2 .

I will be investigating this further…

[cid:24FADF87-091C-448B-83E8-99973D11E8B0 at virginia.edu]

[cid:2042A5DB-1D34-4C05-84AD-40E4E9811CE2 at virginia.edu]

On Jan 12, 2017, at 1:52 PM, Tang, Lydia <ltang5 at mail.lib.msu.edu<mailto:ltang5 at mail.lib.msu.edu>> wrote:

We are also currently experiencing this issue as well.  If I find out more of what our problem is, I’ll send that along later!

Lydia
--
Dr. Lydia Tang
Special Collections Archivist-Librarian
Michigan State University Libraries

From: <archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>> on behalf of Mang Sun <mang.sun at rice.edu<mailto:mang.sun at rice.edu>>
Reply-To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>>
Date: Thursday, January 12, 2017 at 1:47 PM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>>
Subject: Re: [Archivesspace_Users_Group] Indexer erroring in 1.5.2

Steve,
You are not alone. We also ran into indexing error when trying to test upgrade v1.5.1 to v.1.5.2 and see no record found again after the test upgrade.
LYRASIS Support Desk ever helped us in solving a similar indexing problem (no  record found ) which ever drove me mad :) when upgrading 1.4.2 to 1.5.1.
I hope there could be more technical document or white paper on the indexing part which could help us do better understanding of the indexing logic and architecture, and consequently could help us do better troubleshooting work.

Mang

On 1/12/2017 9:42 AM, Majewski, Steven Dennis (sdm7g) wrote:

I’m trying to identify the data where it breaks. It looks like it gets the first 275 archival objects in the first repo indexed. But I haven’t yet figured out which one that is or seen anything different in the region where it fails. But I can send you the whole 8MB gzip compressed mysqldump if you want to try to investigate.

— Steve.

On Jan 11, 2017, at 6:54 PM, James Bullen <james at hudmol.com<mailto:james at hudmol.com><mailto:james at hudmol.com>> wrote:

Hi John and Steve,

Can either of you send some sample data that surfaces this bug?

Cheers,
James

On Jan 12, 2017, at 8:17 AM, Majewski, Steven Dennis (sdm7g) <sdm7g at eservices.virginia.edu<mailto:sdm7g at eservices.virginia.edu><mailto:sdm7g at eservices.virginia.edu>> wrote:

After seeing this message, I tried 1.5.2 on my Mac (10.11.6) laptop and I’m also seeing the same
undefined method `related_records' for nil:NilClass
error, whether or not I do an intervening ( and successful, BTW ) update to 1.5.1.
Reindexing never seems to progress and so browse resources never shows any records.

— Steve Majewski

On Jan 11, 2017, at 1:58 PM, Hambleton, John S <jhamblet at nmu.edu<mailto:jhamblet at nmu.edu><mailto:jhamblet at nmu.edu>> wrote:

Folks,

I am testing archivesspace 1.5.2 on CentOS 6, using a copy of our 1.4.2 production database.
Wondering why it seems like the indexer is taking so long to finish, I am seeing this in
the archivesspace.out log, which makes me think the indexer is caught in an endless loop:

Keep seeing the SAME indexer messages over and over again:
~~~ Indexed 1475 of 1941 accession records in repository 3 ( added 25 records in 8229.0ms ) ~~~
~~~ Indexed 1500 of 1941 accession records in repository 3 ( added 25 records in 5251.0ms ) ~~~

And then,

E, [2017-01-11T13:41:10.740000 #32733] ERROR -- : Thread-6445852: Unhandled exception!
E, [2017-01-11T13:41:10.741000 #32733] ERROR -- :
undefined method `related_records' for nil:NilClass
/usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:64:in `top_container'
/usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:24:in `type_1'
/usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:14:in `to_hash'
org/jruby/RubyHash.java:1341:in `each'

And then seems to start indexing the same data over again:
~~~ Indexed 50 of 1941 accession records in repository 3 ( added 25 records in 11153.0ms ) ~~~
~~~ Indexed 75 of 1941 accession records in repository 3 ( added 25 records in 10051.0ms ) ~~~
~~~ Indexed 100 of 1941 accession records in repository 3 ( added 25 records in 9558.0ms ) ~~~

Kind of looks like the indexer errors on “something” then, tries to reindex the same accession
records over and over again. I had followed the advice here:https://github.com/archivesspace/archivesspace/blob/master/UPGRADING_1.5.0.md#conversion<http://github.com/archivesspace/archivesspace/blob/master/UPGRADING_1.5.0.md#conversion>
before starting archivesspace, that is, “delete your Solr index files to start with a fresh index”.

Any help is appreciated.
Thanks,
John H
NMU

_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org><mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

!DSPAM:5876a103173858981945283!
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org><mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

!DSPAM:5876a103173858981945283!

_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org><mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

_______________________________________________

Archivesspace_Users_Group mailing list

Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org><mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>

http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20170112/53c79ddc/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: record.v1.5.1.png
Type: image/png
Size: 513940 bytes
Desc: record.v1.5.1.png
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20170112/53c79ddc/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bottomrecord.v1.5.1.png
Type: image/png
Size: 348066 bytes
Desc: bottomrecord.v1.5.1.png
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20170112/53c79ddc/attachment-0003.png>