[Archivesspace_Users_Group] Indexer erroring in 1.5.2
James Bullen
james at hudmol.com
Mon Jan 23 18:07:46 EST 2017
Hi Steve,
Yes that’s how it looks to me too. It should really be able to handle that case so I would consider this to be a bug.
Cheers,
James
> On Jan 24, 2017, at 6:54 AM, Majewski, Steven Dennis (sdm7g) <sdm7g at eservices.virginia.edu> wrote:
>
>
> The first two issues appear to be due to trailing spaces in some of the container barcodes.
> Top containers have already been created for barcodes without trailing space, and so it appears that the barcodes don’t match.
>
>
>
>> On Jan 20, 2017, at 3:49 PM, Majewski, Steven Dennis (sdm7g) <sdm7g at eservices.virginia.edu <mailto:sdm7g at eservices.virginia.edu>> wrote:
>>
>>>
>>> On Jan 13, 2017, at 6:15 PM, James Bullen <james at hudmol.com <mailto:james at hudmol.com>> wrote:
>>>
>>>
>>> Hi Steve,
>>>
>>> That fits the pattern I’m seeing in John’s data. Have you checked the container conversion log for any errors relating to that record?
>>>
>>>
>>> Cheers,
>>> James
>>>
>>
>>
>> Just re-did migration from 1.4.2 to 1.5.1 again, and I’m inspecting the logs.
>>
>> Yes, archival_objects/255 is on that list.
>>
>> error,message,url
>> JSONModel::ValidationException,barcode_1 -- Mismatch when mapping between barcode and barcode_1,http://UL-sdm7g-mbp.local/resolve/readonly?uri=/repositories/3/archival_objects/331 <http://ul-sdm7g-mbp.local/resolve/readonly?uri=/repositories/3/archival_objects/331>
>> JSONModel::ValidationException,barcode_1 -- Mismatch when mapping between barcode and barcode_1,http://UL-sdm7g-mbp.local/resolve/readonly?uri=/repositories/3/archival_objects/2059 <http://ul-sdm7g-mbp.local/resolve/readonly?uri=/repositories/3/archival_objects/2059>
>> JSONModel::ValidationException,indicator_1 -- Mismatch when mapping between indicator and indicator_1,http://UL-sdm7g-mbp.local/resolve/readonly?uri=/repositories/3/archival_objects/255 <http://ul-sdm7g-mbp.local/resolve/readonly?uri=/repositories/3/archival_objects/255>
>>
>>
>>
>> I need to inspect the other repo logs and see if all of the ones that break are getting caught, but it looks likely.
>>
>> I’m trying to interpret what exactly those messages are complaining about.
>> Do those error messages and object dumps mean anything to you off the bat?
>>
>> — Steve.
>>
>>
>>>> On Jan 14, 2017, at 8:37 AM, Majewski, Steven Dennis (sdm7g) <sdm7g at eservices.virginia.edu <mailto:sdm7g at eservices.virginia.edu>> wrote:
>>>>
>>>>
>>>> It looks like the indexer gives and error because the backend is getting an error turning the SQL model into a JSON model.
>>>>
>>>> For some of the archival objects that are failing to index, the significant difference is that they do not have a sub_container in their instance. Trying to access the json from the backend in 1.5.2 just yields an error message, but going back to test 1.5.1 migration:
>>>>
>>>> [ This is json from the same screen capture in my previous message. ]
>>>>
>>>> archival_object 255 ( which causes error on index )
>>>>
>>>> {
>>>> "lock_version": 0,
>>>> "position": 17,
>>>> "publish": true,
>>>> "ref_id": "aspace_22a35d6a6ee59ac454ca8f35232b0b40",
>>>> "title": "Buttons and uniform parts, souvenirs, and medals",
>>>> "display_string": "Buttons and uniform parts, souvenirs, and medals, undated",
>>>> "restrictions_apply": false,
>>>> "created_by": "admin",
>>>> "last_modified_by": "admin",
>>>> "create_time": "2015-08-11T21:43:55Z",
>>>> "system_mtime": "2017-01-06T22:25:38Z",
>>>> "user_mtime": "2015-08-11T21:43:55Z",
>>>> "suppressed": false,
>>>> "level": "file",
>>>> "jsonmodel_type": "archival_object",
>>>> "external_ids": [],
>>>> "subjects": [],
>>>> "linked_events": [],
>>>> "extents": [],
>>>> "dates": [
>>>> {
>>>> "lock_version": 0,
>>>> "expression": "undated",
>>>> "created_by": "admin",
>>>> "last_modified_by": "admin",
>>>> "create_time": "2015-08-11T21:43:55Z",
>>>> "system_mtime": "2015-08-11T21:43:55Z",
>>>> "user_mtime": "2015-08-11T21:43:55Z",
>>>> "date_type": "inclusive",
>>>> "label": "creation",
>>>> "jsonmodel_type": "date"
>>>> }
>>>> ],
>>>> "external_documents": [],
>>>> "rights_statements": [],
>>>> "linked_agents": [],
>>>> "instances": [
>>>> {
>>>> "lock_version": 0,
>>>> "created_by": "admin",
>>>> "last_modified_by": "admin",
>>>> "create_time": "2015-08-11T21:43:55Z",
>>>> "system_mtime": "2015-08-11T21:43:55Z",
>>>> "user_mtime": "2015-08-11T21:43:55Z",
>>>> "instance_type": "Realia",
>>>> "jsonmodel_type": "instance",
>>>> "is_representative": false,
>>>> "container": {
>>>> "lock_version": 0,
>>>> "indicator_1": "Artifacts 1",
>>>> "barcode_1": "X030898965",
>>>> "created_by": "admin",
>>>> "last_modified_by": "admin",
>>>> "create_time": "2015-08-11T21:43:55Z",
>>>> "system_mtime": "2016-11-22T16:58:54Z",
>>>> "user_mtime": "2015-08-11T21:43:55Z",
>>>> "type_1": "box",
>>>> "jsonmodel_type": "container",
>>>> "container_locations": []
>>>> }
>>>> }
>>>> ],
>>>> "notes": [],
>>>> "uri": "/repositories/3/archival_objects/255",
>>>> "repository": {
>>>> "ref": "/repositories/3"
>>>> },
>>>> "resource": {
>>>> "ref": "/repositories/3/resources/11"
>>>> },
>>>> "has_unpublished_ancestor": false
>>>> }
>>>>
>>>> archival_object 256: ( indexes successfully, once 255 is deleted. )
>>>>
>>>> {
>>>> "lock_version": 0,
>>>> "position": 18,
>>>> "publish": false,
>>>> "ref_id": "aspace_15da7f2a9b0184a65b1280ae05a7598a",
>>>> "title": "Buttons and uniform parts, souvenirs, and medals",
>>>> "display_string": "Buttons and uniform parts, souvenirs, and medals, undated",
>>>> "restrictions_apply": false,
>>>> "created_by": "admin",
>>>> "last_modified_by": "admin",
>>>> "create_time": "2015-08-11T21:43:55Z",
>>>> "system_mtime": "2017-01-06T22:25:38Z",
>>>> "user_mtime": "2015-08-11T21:43:55Z",
>>>> "suppressed": false,
>>>> "level": "file",
>>>> "jsonmodel_type": "archival_object",
>>>> "external_ids": [],
>>>> "subjects": [],
>>>> "linked_events": [],
>>>> "extents": [],
>>>> "dates": [
>>>> {
>>>> "lock_version": 0,
>>>> "expression": "undated",
>>>> "created_by": "admin",
>>>> "last_modified_by": "admin",
>>>> "create_time": "2015-08-11T21:43:55Z",
>>>> "system_mtime": "2015-08-11T21:43:55Z",
>>>> "user_mtime": "2015-08-11T21:43:55Z",
>>>> "date_type": "inclusive",
>>>> "label": "creation",
>>>> "jsonmodel_type": "date"
>>>> }
>>>> ],
>>>> "external_documents": [],
>>>> "rights_statements": [],
>>>> "linked_agents": [],
>>>> "instances": [
>>>> {
>>>> "lock_version": 0,
>>>> "created_by": "admin",
>>>> "last_modified_by": "admin",
>>>> "create_time": "2015-08-11T21:43:55Z",
>>>> "system_mtime": "2015-08-11T21:43:55Z",
>>>> "user_mtime": "2015-08-11T21:43:55Z",
>>>> "instance_type": "Realia",
>>>> "jsonmodel_type": "instance",
>>>> "is_representative": false,
>>>> "sub_container": {
>>>> "lock_version": 0,
>>>> "created_by": "admin",
>>>> "last_modified_by": "admin",
>>>> "create_time": "2017-01-06T22:28:29Z",
>>>> "system_mtime": "2017-01-06T22:28:29Z",
>>>> "user_mtime": "2017-01-06T22:28:29Z",
>>>> "jsonmodel_type": "sub_container",
>>>> "top_container": {
>>>> "ref": "/repositories/3/top_containers/1573"
>>>> }
>>>> },
>>>> "container": {
>>>> "type_1": "box",
>>>> "indicator_1": "Artifacts 2",
>>>> "barcode_1": "X030866782",
>>>> "container_locations": [],
>>>> "lock_version": 1
>>>> }
>>>> }
>>>> ],
>>>> "notes": [],
>>>> "uri": "/repositories/3/archival_objects/256",
>>>> "repository": {
>>>> "ref": "/repositories/3"
>>>> },
>>>> "resource": {
>>>> "ref": "/repositories/3/resources/11"
>>>> },
>>>> "has_unpublished_ancestor": false
>>>> }
>>>>
>>>>
>>>>
>>>> But as I noted previously, I don’t see the significant difference in the data in 1.4.2 that was migrated that would account for one failing and the other succeeding.
>>>>
>>>>
>>>> ( It appears that there are other records causing errors that don’t necessarily fit this same pattern. )
>>>>
>>>> But the fact that this info is missing in the 1.5.1 migration does suggest it’s not a 1.5.2 only bug, but a problem with the container migration in earlier versions that is just made more visible by some 1.5.2 changes.
>>>>
>>>>
>>>> — Steve.
>>>>
>>>>
>>>>
>>>>> On Jan 12, 2017, at 11:39 PM, James Bullen <james at hudmol.com <mailto:james at hudmol.com>> wrote:
>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> It looks like the problem John is seeing with the indexer failing after upgrading to 1.5.2 is caused by an issue with a record following the container conversion. I’m following up with John off-list on the specifics of his problem. Hopefully we’ll be able to confirm this when we get to a resolution.
>>>>>
>>>>> The container management upgrade has been the source of quite a few bugs and it is on our radar for review.
>>>>>
>>>>>
>>>>> Cheers,
>>>>> James
>>>>>
>>>>>
>>>>>
>>>>>> On Jan 13, 2017, at 1:30 AM, Hambleton, John S <jhamblet at nmu.edu <mailto:jhamblet at nmu.edu>> wrote:
>>>>>>
>>>>>> Yes, thank you,
>>>>>> I sent my data to you off-list.
>>>>>>
>>>>>> John H
>>>>>> NMU
>>>>>>
>>>>>> From: archivesspace_users_group-bounces at lyralists.lyrasis.org <mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org <mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>] On Behalf Of James Bullen
>>>>>> Sent: Wednesday, January 11, 2017 6:56 PM
>>>>>> To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org <mailto:archivesspace_users_group at lyralists.lyrasis.org>>
>>>>>> Subject: Re: [Archivesspace_Users_Group] Indexer erroring in 1.5.2
>>>>>>
>>>>>>
>>>>>> Hi John and Steve,
>>>>>>
>>>>>> Can either of you send some sample data that surfaces this bug?
>>>>>>
>>>>>>
>>>>>> Cheers,
>>>>>> James
>>>>>>
>>>>>>
>>>>>> On Jan 12, 2017, at 8:17 AM, Majewski, Steven Dennis (sdm7g) <sdm7g at eservices.virginia.edu <mailto:sdm7g at eservices.virginia.edu>> wrote:
>>>>>>
>>>>>>
>>>>>> After seeing this message, I tried 1.5.2 on my Mac (10.11.6) laptop and I’m also seeing the same
>>>>>> undefined method `related_records' for nil:NilClass
>>>>>> error, whether or not I do an intervening ( and successful, BTW ) update to 1.5.1.
>>>>>> Reindexing never seems to progress and so browse resources never shows any records.
>>>>>>
>>>>>>
>>>>>> — Steve Majewski
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Jan 11, 2017, at 1:58 PM, Hambleton, John S <jhamblet at nmu.edu <mailto:jhamblet at nmu.edu>> wrote:
>>>>>>
>>>>>> Folks,
>>>>>>
>>>>>> I am testing archivesspace 1.5.2 on CentOS 6, using a copy of our 1.4.2 production database.
>>>>>> Wondering why it seems like the indexer is taking so long to finish, I am seeing this in
>>>>>> the archivesspace.out log, which makes me think the indexer is caught in an endless loop:
>>>>>>
>>>>>> Keep seeing the SAME indexer messages over and over again:
>>>>>> ~~~ Indexed 1475 of 1941 accession records in repository 3 ( added 25 records in 8229.0ms ) ~~~
>>>>>> ~~~ Indexed 1500 of 1941 accession records in repository 3 ( added 25 records in 5251.0ms ) ~~~
>>>>>>
>>>>>> And then,
>>>>>>
>>>>>> E, [2017-01-11T13:41:10.740000 #32733] ERROR -- : Thread-6445852: Unhandled exception!
>>>>>> E, [2017-01-11T13:41:10.741000 #32733] ERROR -- :
>>>>>> undefined method `related_records' for nil:NilClass
>>>>>> /usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:64:in `top_container'
>>>>>> /usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:24:in `type_1'
>>>>>> /usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:14:in `to_hash'
>>>>>> org/jruby/RubyHash.java:1341:in `each'
>>>>>>
>>>>>> And then seems to start indexing the same data over again:
>>>>>> ~~~ Indexed 50 of 1941 accession records in repository 3 ( added 25 records in 11153.0ms ) ~~~
>>>>>> ~~~ Indexed 75 of 1941 accession records in repository 3 ( added 25 records in 10051.0ms ) ~~~
>>>>>> ~~~ Indexed 100 of 1941 accession records in repository 3 ( added 25 records in 9558.0ms ) ~~~
>>>>>>
>>>>>> Kind of looks like the indexer errors on “something” then, tries to reindex the same accession
>>>>>> records over and over again. I had followed the advice here:https://github.com/archivesspace/archivesspace/blob/master/UPGRADING_1.5.0.md#conversion <https://github.com/archivesspace/archivesspace/blob/master/UPGRADING_1.5.0.md#conversion>
>>>>>> before starting archivesspace, that is, “delete your Solr index files to start with a fresh index”.
>>>>>>
>>>>>> Any help is appreciated.
>>>>>> Thanks,
>>>>>> John H
>>>>>> NMU
>>>>>>
>>>>>> _______________________________________________
>>>>>> Archivesspace_Users_Group mailing list
>>>>>> Archivesspace_Users_Group at lyralists.lyrasis.org <mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
>>>>>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group <http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Archivesspace_Users_Group mailing list
>>>>>> Archivesspace_Users_Group at lyralists.lyrasis.org <mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
>>>>>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group <http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Archivesspace_Users_Group mailing list
>>>>>> Archivesspace_Users_Group at lyralists.lyrasis.org <mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
>>>>>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group <http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group>
>>>>>>
>>>>>>
>>>>>> !DSPAM:587793334611909212977!
>>>>>
>>>>> _______________________________________________
>>>>> Archivesspace_Users_Group mailing list
>>>>> Archivesspace_Users_Group at lyralists.lyrasis.org <mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
>>>>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group <http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group>
>>>>
>>>> !DSPAM:587948af109451970717657!
>>>> _______________________________________________
>>>> Archivesspace_Users_Group mailing list
>>>> Archivesspace_Users_Group at lyralists.lyrasis.org <mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
>>>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group <http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group>
>>>>
>>>>
>>>> !DSPAM:587948af109451970717657!
>>>
>>> _______________________________________________
>>> Archivesspace_Users_Group mailing list
>>> Archivesspace_Users_Group at lyralists.lyrasis.org <mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
>>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group <http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group>
>>
>> <7ebc58861f8c38997d24d1316e9276ec>_______________________________________________
>> Archivesspace_Users_Group mailing list
>> Archivesspace_Users_Group at lyralists.lyrasis.org <mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group <http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group>
> !DSPAM:58865fa6242613324871184!
> _______________________________________________
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group at lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>
>
> !DSPAM:58865fa6242613324871184!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20170124/bcf5057d/attachment-0001.html>
More information about the Archivesspace_Users_Group
mailing list