[Archivesspace_Users_Group] Indexer erroring in 1.5.2
Majewski, Steven Dennis (sdm7g)
sdm7g at eservices.virginia.edu
Fri Jan 13 16:37:32 EST 2017
It looks like the indexer gives and error because the backend is getting an error turning the SQL model into a JSON model.
For some of the archival objects that are failing to index, the significant difference is that they do not have a sub_container in their instance. Trying to access the json from the backend in 1.5.2 just yields an error message, but going back to test 1.5.1 migration:
[ This is json from the same screen capture in my previous message. ]
archival_object 255 ( which causes error on index )
{
"lock_version": 0,
"position": 17,
"publish": true,
"ref_id": "aspace_22a35d6a6ee59ac454ca8f35232b0b40",
"title": "Buttons and uniform parts, souvenirs, and medals",
"display_string": "Buttons and uniform parts, souvenirs, and medals, undated",
"restrictions_apply": false,
"created_by": "admin",
"last_modified_by": "admin",
"create_time": "2015-08-11T21:43:55Z",
"system_mtime": "2017-01-06T22:25:38Z",
"user_mtime": "2015-08-11T21:43:55Z",
"suppressed": false,
"level": "file",
"jsonmodel_type": "archival_object",
"external_ids": [],
"subjects": [],
"linked_events": [],
"extents": [],
"dates": [
{
"lock_version": 0,
"expression": "undated",
"created_by": "admin",
"last_modified_by": "admin",
"create_time": "2015-08-11T21:43:55Z",
"system_mtime": "2015-08-11T21:43:55Z",
"user_mtime": "2015-08-11T21:43:55Z",
"date_type": "inclusive",
"label": "creation",
"jsonmodel_type": "date"
}
],
"external_documents": [],
"rights_statements": [],
"linked_agents": [],
"instances": [
{
"lock_version": 0,
"created_by": "admin",
"last_modified_by": "admin",
"create_time": "2015-08-11T21:43:55Z",
"system_mtime": "2015-08-11T21:43:55Z",
"user_mtime": "2015-08-11T21:43:55Z",
"instance_type": "Realia",
"jsonmodel_type": "instance",
"is_representative": false,
"container": {
"lock_version": 0,
"indicator_1": "Artifacts 1",
"barcode_1": "X030898965",
"created_by": "admin",
"last_modified_by": "admin",
"create_time": "2015-08-11T21:43:55Z",
"system_mtime": "2016-11-22T16:58:54Z",
"user_mtime": "2015-08-11T21:43:55Z",
"type_1": "box",
"jsonmodel_type": "container",
"container_locations": []
}
}
],
"notes": [],
"uri": "/repositories/3/archival_objects/255",
"repository": {
"ref": "/repositories/3"
},
"resource": {
"ref": "/repositories/3/resources/11"
},
"has_unpublished_ancestor": false
}
archival_object 256: ( indexes successfully, once 255 is deleted. )
{
"lock_version": 0,
"position": 18,
"publish": false,
"ref_id": "aspace_15da7f2a9b0184a65b1280ae05a7598a",
"title": "Buttons and uniform parts, souvenirs, and medals",
"display_string": "Buttons and uniform parts, souvenirs, and medals, undated",
"restrictions_apply": false,
"created_by": "admin",
"last_modified_by": "admin",
"create_time": "2015-08-11T21:43:55Z",
"system_mtime": "2017-01-06T22:25:38Z",
"user_mtime": "2015-08-11T21:43:55Z",
"suppressed": false,
"level": "file",
"jsonmodel_type": "archival_object",
"external_ids": [],
"subjects": [],
"linked_events": [],
"extents": [],
"dates": [
{
"lock_version": 0,
"expression": "undated",
"created_by": "admin",
"last_modified_by": "admin",
"create_time": "2015-08-11T21:43:55Z",
"system_mtime": "2015-08-11T21:43:55Z",
"user_mtime": "2015-08-11T21:43:55Z",
"date_type": "inclusive",
"label": "creation",
"jsonmodel_type": "date"
}
],
"external_documents": [],
"rights_statements": [],
"linked_agents": [],
"instances": [
{
"lock_version": 0,
"created_by": "admin",
"last_modified_by": "admin",
"create_time": "2015-08-11T21:43:55Z",
"system_mtime": "2015-08-11T21:43:55Z",
"user_mtime": "2015-08-11T21:43:55Z",
"instance_type": "Realia",
"jsonmodel_type": "instance",
"is_representative": false,
"sub_container": {
"lock_version": 0,
"created_by": "admin",
"last_modified_by": "admin",
"create_time": "2017-01-06T22:28:29Z",
"system_mtime": "2017-01-06T22:28:29Z",
"user_mtime": "2017-01-06T22:28:29Z",
"jsonmodel_type": "sub_container",
"top_container": {
"ref": "/repositories/3/top_containers/1573"
}
},
"container": {
"type_1": "box",
"indicator_1": "Artifacts 2",
"barcode_1": "X030866782",
"container_locations": [],
"lock_version": 1
}
}
],
"notes": [],
"uri": "/repositories/3/archival_objects/256",
"repository": {
"ref": "/repositories/3"
},
"resource": {
"ref": "/repositories/3/resources/11"
},
"has_unpublished_ancestor": false
}
But as I noted previously, I don’t see the significant difference in the data in 1.4.2 that was migrated that would account for one failing and the other succeeding.
( It appears that there are other records causing errors that don’t necessarily fit this same pattern. )
But the fact that this info is missing in the 1.5.1 migration does suggest it’s not a 1.5.2 only bug, but a problem with the container migration in earlier versions that is just made more visible by some 1.5.2 changes.
— Steve.
On Jan 12, 2017, at 11:39 PM, James Bullen <james at hudmol.com<mailto:james at hudmol.com>> wrote:
Hi,
It looks like the problem John is seeing with the indexer failing after upgrading to 1.5.2 is caused by an issue with a record following the container conversion. I’m following up with John off-list on the specifics of his problem. Hopefully we’ll be able to confirm this when we get to a resolution.
The container management upgrade has been the source of quite a few bugs and it is on our radar for review.
Cheers,
James
On Jan 13, 2017, at 1:30 AM, Hambleton, John S <jhamblet at nmu.edu<mailto:jhamblet at nmu.edu>> wrote:
Yes, thank you,
I sent my data to you off-list.
John H
NMU
From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] On Behalf Of James Bullen
Sent: Wednesday, January 11, 2017 6:56 PM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>>
Subject: Re: [Archivesspace_Users_Group] Indexer erroring in 1.5.2
Hi John and Steve,
Can either of you send some sample data that surfaces this bug?
Cheers,
James
On Jan 12, 2017, at 8:17 AM, Majewski, Steven Dennis (sdm7g) <sdm7g at eservices.virginia.edu<mailto:sdm7g at eservices.virginia.edu>> wrote:
After seeing this message, I tried 1.5.2 on my Mac (10.11.6) laptop and I’m also seeing the same
undefined method `related_records' for nil:NilClass
error, whether or not I do an intervening ( and successful, BTW ) update to 1.5.1.
Reindexing never seems to progress and so browse resources never shows any records.
— Steve Majewski
On Jan 11, 2017, at 1:58 PM, Hambleton, John S <jhamblet at nmu.edu<mailto:jhamblet at nmu.edu>> wrote:
Folks,
I am testing archivesspace 1.5.2 on CentOS 6, using a copy of our 1.4.2 production database.
Wondering why it seems like the indexer is taking so long to finish, I am seeing this in
the archivesspace.out log, which makes me think the indexer is caught in an endless loop:
Keep seeing the SAME indexer messages over and over again:
~~~ Indexed 1475 of 1941 accession records in repository 3 ( added 25 records in 8229.0ms ) ~~~
~~~ Indexed 1500 of 1941 accession records in repository 3 ( added 25 records in 5251.0ms ) ~~~
And then,
E, [2017-01-11T13:41:10.740000 #32733] ERROR -- : Thread-6445852: Unhandled exception!
E, [2017-01-11T13:41:10.741000 #32733] ERROR -- :
undefined method `related_records' for nil:NilClass
/usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:64:in `top_container'
/usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:24:in `type_1'
/usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:14:in `to_hash'
org/jruby/RubyHash.java:1341:in `each'
And then seems to start indexing the same data over again:
~~~ Indexed 50 of 1941 accession records in repository 3 ( added 25 records in 11153.0ms ) ~~~
~~~ Indexed 75 of 1941 accession records in repository 3 ( added 25 records in 10051.0ms ) ~~~
~~~ Indexed 100 of 1941 accession records in repository 3 ( added 25 records in 9558.0ms ) ~~~
Kind of looks like the indexer errors on “something” then, tries to reindex the same accession
records over and over again. I had followed the advice here:https://github.com/archivesspace/archivesspace/blob/master/UPGRADING_1.5.0.md#conversion
before starting archivesspace, that is, “delete your Solr index files to start with a fresh index”.
Any help is appreciated.
Thanks,
John H
NMU
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
!DSPAM:5876a103173858981945283!
!DSPAM:587793334611909212977! _______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
!DSPAM:587793334611909212977!
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20170113/c3e333a7/attachment.html>
More information about the Archivesspace_Users_Group
mailing list