[Archivesspace_Users_Group] Indexer erroring in 1.5.2
Majewski, Steven Dennis (sdm7g)
sdm7g at eservices.virginia.edu
Mon Jan 23 14:54:53 EST 2017
The first two issues appear to be due to trailing spaces in some of the container barcodes.
Top containers have already been created for barcodes without trailing space, and so it appears that the barcodes don’t match.
On Jan 20, 2017, at 3:49 PM, Majewski, Steven Dennis (sdm7g) <sdm7g at eservices.virginia.edu<mailto:sdm7g at eservices.virginia.edu>> wrote:
On Jan 13, 2017, at 6:15 PM, James Bullen <james at hudmol.com<mailto:james at hudmol.com>> wrote:
Hi Steve,
That fits the pattern I’m seeing in John’s data. Have you checked the container conversion log for any errors relating to that record?
Cheers,
James
Just re-did migration from 1.4.2 to 1.5.1 again, and I’m inspecting the logs.
Yes, archival_objects/255 is on that list.
error,message,url
JSONModel::ValidationException,barcode_1 -- Mismatch when mapping between barcode and barcode_1,http://UL-sdm7g-mbp.local/resolve/readonly?uri=/repositories/3/archival_objects/331<http://ul-sdm7g-mbp.local/resolve/readonly?uri=/repositories/3/archival_objects/331>
JSONModel::ValidationException,barcode_1 -- Mismatch when mapping between barcode and barcode_1,http://UL-sdm7g-mbp.local/resolve/readonly?uri=/repositories/3/archival_objects/2059<http://ul-sdm7g-mbp.local/resolve/readonly?uri=/repositories/3/archival_objects/2059>
JSONModel::ValidationException,indicator_1 -- Mismatch when mapping between indicator and indicator_1,http://UL-sdm7g-mbp.local/resolve/readonly?uri=/repositories/3/archival_objects/255<http://ul-sdm7g-mbp.local/resolve/readonly?uri=/repositories/3/archival_objects/255>
I need to inspect the other repo logs and see if all of the ones that break are getting caught, but it looks likely.
I’m trying to interpret what exactly those messages are complaining about.
Do those error messages and object dumps mean anything to you off the bat?
— Steve.
On Jan 14, 2017, at 8:37 AM, Majewski, Steven Dennis (sdm7g) <sdm7g at eservices.virginia.edu<mailto:sdm7g at eservices.virginia.edu>> wrote:
It looks like the indexer gives and error because the backend is getting an error turning the SQL model into a JSON model.
For some of the archival objects that are failing to index, the significant difference is that they do not have a sub_container in their instance. Trying to access the json from the backend in 1.5.2 just yields an error message, but going back to test 1.5.1 migration:
[ This is json from the same screen capture in my previous message. ]
archival_object 255 ( which causes error on index )
{
"lock_version": 0,
"position": 17,
"publish": true,
"ref_id": "aspace_22a35d6a6ee59ac454ca8f35232b0b40",
"title": "Buttons and uniform parts, souvenirs, and medals",
"display_string": "Buttons and uniform parts, souvenirs, and medals, undated",
"restrictions_apply": false,
"created_by": "admin",
"last_modified_by": "admin",
"create_time": "2015-08-11T21:43:55Z",
"system_mtime": "2017-01-06T22:25:38Z",
"user_mtime": "2015-08-11T21:43:55Z",
"suppressed": false,
"level": "file",
"jsonmodel_type": "archival_object",
"external_ids": [],
"subjects": [],
"linked_events": [],
"extents": [],
"dates": [
{
"lock_version": 0,
"expression": "undated",
"created_by": "admin",
"last_modified_by": "admin",
"create_time": "2015-08-11T21:43:55Z",
"system_mtime": "2015-08-11T21:43:55Z",
"user_mtime": "2015-08-11T21:43:55Z",
"date_type": "inclusive",
"label": "creation",
"jsonmodel_type": "date"
}
],
"external_documents": [],
"rights_statements": [],
"linked_agents": [],
"instances": [
{
"lock_version": 0,
"created_by": "admin",
"last_modified_by": "admin",
"create_time": "2015-08-11T21:43:55Z",
"system_mtime": "2015-08-11T21:43:55Z",
"user_mtime": "2015-08-11T21:43:55Z",
"instance_type": "Realia",
"jsonmodel_type": "instance",
"is_representative": false,
"container": {
"lock_version": 0,
"indicator_1": "Artifacts 1",
"barcode_1": "X030898965",
"created_by": "admin",
"last_modified_by": "admin",
"create_time": "2015-08-11T21:43:55Z",
"system_mtime": "2016-11-22T16:58:54Z",
"user_mtime": "2015-08-11T21:43:55Z",
"type_1": "box",
"jsonmodel_type": "container",
"container_locations": []
}
}
],
"notes": [],
"uri": "/repositories/3/archival_objects/255",
"repository": {
"ref": "/repositories/3"
},
"resource": {
"ref": "/repositories/3/resources/11"
},
"has_unpublished_ancestor": false
}
archival_object 256: ( indexes successfully, once 255 is deleted. )
{
"lock_version": 0,
"position": 18,
"publish": false,
"ref_id": "aspace_15da7f2a9b0184a65b1280ae05a7598a",
"title": "Buttons and uniform parts, souvenirs, and medals",
"display_string": "Buttons and uniform parts, souvenirs, and medals, undated",
"restrictions_apply": false,
"created_by": "admin",
"last_modified_by": "admin",
"create_time": "2015-08-11T21:43:55Z",
"system_mtime": "2017-01-06T22:25:38Z",
"user_mtime": "2015-08-11T21:43:55Z",
"suppressed": false,
"level": "file",
"jsonmodel_type": "archival_object",
"external_ids": [],
"subjects": [],
"linked_events": [],
"extents": [],
"dates": [
{
"lock_version": 0,
"expression": "undated",
"created_by": "admin",
"last_modified_by": "admin",
"create_time": "2015-08-11T21:43:55Z",
"system_mtime": "2015-08-11T21:43:55Z",
"user_mtime": "2015-08-11T21:43:55Z",
"date_type": "inclusive",
"label": "creation",
"jsonmodel_type": "date"
}
],
"external_documents": [],
"rights_statements": [],
"linked_agents": [],
"instances": [
{
"lock_version": 0,
"created_by": "admin",
"last_modified_by": "admin",
"create_time": "2015-08-11T21:43:55Z",
"system_mtime": "2015-08-11T21:43:55Z",
"user_mtime": "2015-08-11T21:43:55Z",
"instance_type": "Realia",
"jsonmodel_type": "instance",
"is_representative": false,
"sub_container": {
"lock_version": 0,
"created_by": "admin",
"last_modified_by": "admin",
"create_time": "2017-01-06T22:28:29Z",
"system_mtime": "2017-01-06T22:28:29Z",
"user_mtime": "2017-01-06T22:28:29Z",
"jsonmodel_type": "sub_container",
"top_container": {
"ref": "/repositories/3/top_containers/1573"
}
},
"container": {
"type_1": "box",
"indicator_1": "Artifacts 2",
"barcode_1": "X030866782",
"container_locations": [],
"lock_version": 1
}
}
],
"notes": [],
"uri": "/repositories/3/archival_objects/256",
"repository": {
"ref": "/repositories/3"
},
"resource": {
"ref": "/repositories/3/resources/11"
},
"has_unpublished_ancestor": false
}
But as I noted previously, I don’t see the significant difference in the data in 1.4.2 that was migrated that would account for one failing and the other succeeding.
( It appears that there are other records causing errors that don’t necessarily fit this same pattern. )
But the fact that this info is missing in the 1.5.1 migration does suggest it’s not a 1.5.2 only bug, but a problem with the container migration in earlier versions that is just made more visible by some 1.5.2 changes.
— Steve.
On Jan 12, 2017, at 11:39 PM, James Bullen <james at hudmol.com<mailto:james at hudmol.com>> wrote:
Hi,
It looks like the problem John is seeing with the indexer failing after upgrading to 1.5.2 is caused by an issue with a record following the container conversion. I’m following up with John off-list on the specifics of his problem. Hopefully we’ll be able to confirm this when we get to a resolution.
The container management upgrade has been the source of quite a few bugs and it is on our radar for review.
Cheers,
James
On Jan 13, 2017, at 1:30 AM, Hambleton, John S <jhamblet at nmu.edu<mailto:jhamblet at nmu.edu>> wrote:
Yes, thank you,
I sent my data to you off-list.
John H
NMU
From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] On Behalf Of James Bullen
Sent: Wednesday, January 11, 2017 6:56 PM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>>
Subject: Re: [Archivesspace_Users_Group] Indexer erroring in 1.5.2
Hi John and Steve,
Can either of you send some sample data that surfaces this bug?
Cheers,
James
On Jan 12, 2017, at 8:17 AM, Majewski, Steven Dennis (sdm7g) <sdm7g at eservices.virginia.edu<mailto:sdm7g at eservices.virginia.edu>> wrote:
After seeing this message, I tried 1.5.2 on my Mac (10.11.6) laptop and I’m also seeing the same
undefined method `related_records' for nil:NilClass
error, whether or not I do an intervening ( and successful, BTW ) update to 1.5.1.
Reindexing never seems to progress and so browse resources never shows any records.
— Steve Majewski
On Jan 11, 2017, at 1:58 PM, Hambleton, John S <jhamblet at nmu.edu<mailto:jhamblet at nmu.edu>> wrote:
Folks,
I am testing archivesspace 1.5.2 on CentOS 6, using a copy of our 1.4.2 production database.
Wondering why it seems like the indexer is taking so long to finish, I am seeing this in
the archivesspace.out log, which makes me think the indexer is caught in an endless loop:
Keep seeing the SAME indexer messages over and over again:
~~~ Indexed 1475 of 1941 accession records in repository 3 ( added 25 records in 8229.0ms ) ~~~
~~~ Indexed 1500 of 1941 accession records in repository 3 ( added 25 records in 5251.0ms ) ~~~
And then,
E, [2017-01-11T13:41:10.740000 #32733] ERROR -- : Thread-6445852: Unhandled exception!
E, [2017-01-11T13:41:10.741000 #32733] ERROR -- :
undefined method `related_records' for nil:NilClass
/usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:64:in `top_container'
/usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:24:in `type_1'
/usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:14:in `to_hash'
org/jruby/RubyHash.java:1341:in `each'
And then seems to start indexing the same data over again:
~~~ Indexed 50 of 1941 accession records in repository 3 ( added 25 records in 11153.0ms ) ~~~
~~~ Indexed 75 of 1941 accession records in repository 3 ( added 25 records in 10051.0ms ) ~~~
~~~ Indexed 100 of 1941 accession records in repository 3 ( added 25 records in 9558.0ms ) ~~~
Kind of looks like the indexer errors on “something” then, tries to reindex the same accession
records over and over again. I had followed the advice here:https://github.com/archivesspace/archivesspace/blob/master/UPGRADING_1.5.0.md#conversion
before starting archivesspace, that is, “delete your Solr index files to start with a fresh index”.
Any help is appreciated.
Thanks,
John H
NMU
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
!DSPAM:587793334611909212977! _______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
!DSPAM:587793334611909212977!
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
!DSPAM:587948af109451970717657!
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
!DSPAM:587948af109451970717657!
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
<7ebc58861f8c38997d24d1316e9276ec>_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20170123/9776af4f/attachment.html>
More information about the Archivesspace_Users_Group
mailing list