[Archivesspace_Users_Group] Indexer erroring in 1.5.2
James Bullen
james at hudmol.com
Fri Jan 13 18:15:38 EST 2017
Hi Steve,
That fits the pattern I’m seeing in John’s data. Have you checked the container conversion log for any errors relating to that record?
Cheers,
James
> On Jan 14, 2017, at 8:37 AM, Majewski, Steven Dennis (sdm7g) <sdm7g at eservices.virginia.edu> wrote:
>
>
> It looks like the indexer gives and error because the backend is getting an error turning the SQL model into a JSON model.
>
> For some of the archival objects that are failing to index, the significant difference is that they do not have a sub_container in their instance. Trying to access the json from the backend in 1.5.2 just yields an error message, but going back to test 1.5.1 migration:
>
> [ This is json from the same screen capture in my previous message. ]
>
> archival_object 255 ( which causes error on index )
>
> {
> "lock_version": 0,
> "position": 17,
> "publish": true,
> "ref_id": "aspace_22a35d6a6ee59ac454ca8f35232b0b40",
> "title": "Buttons and uniform parts, souvenirs, and medals",
> "display_string": "Buttons and uniform parts, souvenirs, and medals, undated",
> "restrictions_apply": false,
> "created_by": "admin",
> "last_modified_by": "admin",
> "create_time": "2015-08-11T21:43:55Z",
> "system_mtime": "2017-01-06T22:25:38Z",
> "user_mtime": "2015-08-11T21:43:55Z",
> "suppressed": false,
> "level": "file",
> "jsonmodel_type": "archival_object",
> "external_ids": [],
> "subjects": [],
> "linked_events": [],
> "extents": [],
> "dates": [
> {
> "lock_version": 0,
> "expression": "undated",
> "created_by": "admin",
> "last_modified_by": "admin",
> "create_time": "2015-08-11T21:43:55Z",
> "system_mtime": "2015-08-11T21:43:55Z",
> "user_mtime": "2015-08-11T21:43:55Z",
> "date_type": "inclusive",
> "label": "creation",
> "jsonmodel_type": "date"
> }
> ],
> "external_documents": [],
> "rights_statements": [],
> "linked_agents": [],
> "instances": [
> {
> "lock_version": 0,
> "created_by": "admin",
> "last_modified_by": "admin",
> "create_time": "2015-08-11T21:43:55Z",
> "system_mtime": "2015-08-11T21:43:55Z",
> "user_mtime": "2015-08-11T21:43:55Z",
> "instance_type": "Realia",
> "jsonmodel_type": "instance",
> "is_representative": false,
> "container": {
> "lock_version": 0,
> "indicator_1": "Artifacts 1",
> "barcode_1": "X030898965",
> "created_by": "admin",
> "last_modified_by": "admin",
> "create_time": "2015-08-11T21:43:55Z",
> "system_mtime": "2016-11-22T16:58:54Z",
> "user_mtime": "2015-08-11T21:43:55Z",
> "type_1": "box",
> "jsonmodel_type": "container",
> "container_locations": []
> }
> }
> ],
> "notes": [],
> "uri": "/repositories/3/archival_objects/255",
> "repository": {
> "ref": "/repositories/3"
> },
> "resource": {
> "ref": "/repositories/3/resources/11"
> },
> "has_unpublished_ancestor": false
> }
>
> archival_object 256: ( indexes successfully, once 255 is deleted. )
>
> {
> "lock_version": 0,
> "position": 18,
> "publish": false,
> "ref_id": "aspace_15da7f2a9b0184a65b1280ae05a7598a",
> "title": "Buttons and uniform parts, souvenirs, and medals",
> "display_string": "Buttons and uniform parts, souvenirs, and medals, undated",
> "restrictions_apply": false,
> "created_by": "admin",
> "last_modified_by": "admin",
> "create_time": "2015-08-11T21:43:55Z",
> "system_mtime": "2017-01-06T22:25:38Z",
> "user_mtime": "2015-08-11T21:43:55Z",
> "suppressed": false,
> "level": "file",
> "jsonmodel_type": "archival_object",
> "external_ids": [],
> "subjects": [],
> "linked_events": [],
> "extents": [],
> "dates": [
> {
> "lock_version": 0,
> "expression": "undated",
> "created_by": "admin",
> "last_modified_by": "admin",
> "create_time": "2015-08-11T21:43:55Z",
> "system_mtime": "2015-08-11T21:43:55Z",
> "user_mtime": "2015-08-11T21:43:55Z",
> "date_type": "inclusive",
> "label": "creation",
> "jsonmodel_type": "date"
> }
> ],
> "external_documents": [],
> "rights_statements": [],
> "linked_agents": [],
> "instances": [
> {
> "lock_version": 0,
> "created_by": "admin",
> "last_modified_by": "admin",
> "create_time": "2015-08-11T21:43:55Z",
> "system_mtime": "2015-08-11T21:43:55Z",
> "user_mtime": "2015-08-11T21:43:55Z",
> "instance_type": "Realia",
> "jsonmodel_type": "instance",
> "is_representative": false,
> "sub_container": {
> "lock_version": 0,
> "created_by": "admin",
> "last_modified_by": "admin",
> "create_time": "2017-01-06T22:28:29Z",
> "system_mtime": "2017-01-06T22:28:29Z",
> "user_mtime": "2017-01-06T22:28:29Z",
> "jsonmodel_type": "sub_container",
> "top_container": {
> "ref": "/repositories/3/top_containers/1573"
> }
> },
> "container": {
> "type_1": "box",
> "indicator_1": "Artifacts 2",
> "barcode_1": "X030866782",
> "container_locations": [],
> "lock_version": 1
> }
> }
> ],
> "notes": [],
> "uri": "/repositories/3/archival_objects/256",
> "repository": {
> "ref": "/repositories/3"
> },
> "resource": {
> "ref": "/repositories/3/resources/11"
> },
> "has_unpublished_ancestor": false
> }
>
>
>
> But as I noted previously, I don’t see the significant difference in the data in 1.4.2 that was migrated that would account for one failing and the other succeeding.
>
>
> ( It appears that there are other records causing errors that don’t necessarily fit this same pattern. )
>
> But the fact that this info is missing in the 1.5.1 migration does suggest it’s not a 1.5.2 only bug, but a problem with the container migration in earlier versions that is just made more visible by some 1.5.2 changes.
>
>
> — Steve.
>
>
>
>> On Jan 12, 2017, at 11:39 PM, James Bullen <james at hudmol.com <mailto:james at hudmol.com>> wrote:
>>
>>
>> Hi,
>>
>> It looks like the problem John is seeing with the indexer failing after upgrading to 1.5.2 is caused by an issue with a record following the container conversion. I’m following up with John off-list on the specifics of his problem. Hopefully we’ll be able to confirm this when we get to a resolution.
>>
>> The container management upgrade has been the source of quite a few bugs and it is on our radar for review.
>>
>>
>> Cheers,
>> James
>>
>>
>>
>>> On Jan 13, 2017, at 1:30 AM, Hambleton, John S <jhamblet at nmu.edu <mailto:jhamblet at nmu.edu>> wrote:
>>>
>>> Yes, thank you,
>>> I sent my data to you off-list.
>>>
>>> John H
>>> NMU
>>>
>>> From: archivesspace_users_group-bounces at lyralists.lyrasis.org <mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org <mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>] On Behalf Of James Bullen
>>> Sent: Wednesday, January 11, 2017 6:56 PM
>>> To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org <mailto:archivesspace_users_group at lyralists.lyrasis.org>>
>>> Subject: Re: [Archivesspace_Users_Group] Indexer erroring in 1.5.2
>>>
>>>
>>> Hi John and Steve,
>>>
>>> Can either of you send some sample data that surfaces this bug?
>>>
>>>
>>> Cheers,
>>> James
>>>
>>>
>>> On Jan 12, 2017, at 8:17 AM, Majewski, Steven Dennis (sdm7g) <sdm7g at eservices.virginia.edu <mailto:sdm7g at eservices.virginia.edu>> wrote:
>>>
>>>
>>> After seeing this message, I tried 1.5.2 on my Mac (10.11.6) laptop and I’m also seeing the same
>>> undefined method `related_records' for nil:NilClass
>>> error, whether or not I do an intervening ( and successful, BTW ) update to 1.5.1.
>>> Reindexing never seems to progress and so browse resources never shows any records.
>>>
>>>
>>> — Steve Majewski
>>>
>>>
>>>
>>> On Jan 11, 2017, at 1:58 PM, Hambleton, John S <jhamblet at nmu.edu <mailto:jhamblet at nmu.edu>> wrote:
>>>
>>> Folks,
>>>
>>> I am testing archivesspace 1.5.2 on CentOS 6, using a copy of our 1.4.2 production database.
>>> Wondering why it seems like the indexer is taking so long to finish, I am seeing this in
>>> the archivesspace.out log, which makes me think the indexer is caught in an endless loop:
>>>
>>> Keep seeing the SAME indexer messages over and over again:
>>> ~~~ Indexed 1475 of 1941 accession records in repository 3 ( added 25 records in 8229.0ms ) ~~~
>>> ~~~ Indexed 1500 of 1941 accession records in repository 3 ( added 25 records in 5251.0ms ) ~~~
>>>
>>> And then,
>>>
>>> E, [2017-01-11T13:41:10.740000 #32733] ERROR -- : Thread-6445852: Unhandled exception!
>>> E, [2017-01-11T13:41:10.741000 #32733] ERROR -- :
>>> undefined method `related_records' for nil:NilClass
>>> /usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:64:in `top_container'
>>> /usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:24:in `type_1'
>>> /usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:14:in `to_hash'
>>> org/jruby/RubyHash.java:1341:in `each'
>>>
>>> And then seems to start indexing the same data over again:
>>> ~~~ Indexed 50 of 1941 accession records in repository 3 ( added 25 records in 11153.0ms ) ~~~
>>> ~~~ Indexed 75 of 1941 accession records in repository 3 ( added 25 records in 10051.0ms ) ~~~
>>> ~~~ Indexed 100 of 1941 accession records in repository 3 ( added 25 records in 9558.0ms ) ~~~
>>>
>>> Kind of looks like the indexer errors on “something” then, tries to reindex the same accession
>>> records over and over again. I had followed the advice here:https://github.com/archivesspace/archivesspace/blob/master/UPGRADING_1.5.0.md#conversion <https://github.com/archivesspace/archivesspace/blob/master/UPGRADING_1.5.0.md#conversion>
>>> before starting archivesspace, that is, “delete your Solr index files to start with a fresh index”.
>>>
>>> Any help is appreciated.
>>> Thanks,
>>> John H
>>> NMU
>>>
>>> _______________________________________________
>>> Archivesspace_Users_Group mailing list
>>> Archivesspace_Users_Group at lyralists.lyrasis.org <mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
>>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group <http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group>
>>>
>>> _______________________________________________
>>> Archivesspace_Users_Group mailing list
>>> Archivesspace_Users_Group at lyralists.lyrasis.org <mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
>>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group <http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group>
>>>
>>>
>>>
>>> !DSPAM:587793334611909212977! _______________________________________________
>>> Archivesspace_Users_Group mailing list
>>> Archivesspace_Users_Group at lyralists.lyrasis.org <mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
>>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group <http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group>
>>>
>>>
>>> !DSPAM:587793334611909212977!
>>
>> _______________________________________________
>> Archivesspace_Users_Group mailing list
>> Archivesspace_Users_Group at lyralists.lyrasis.org <mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>
> !DSPAM:587948af109451970717657!
> _______________________________________________
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group at lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>
>
> !DSPAM:587948af109451970717657!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20170114/354cba41/attachment.html>
More information about the Archivesspace_Users_Group
mailing list