[Archivesspace_Users_Group] Failure in periodic indexer worker thread
Hawk, Patrick
hawkpf at miamioh.edu
Thu Apr 6 09:54:17 EDT 2017
I wanted to respond to this thread on what I had to do to complete the
migration. I decided to remove all of the barcode_1 that were duplicates
and make them null. I did not want to lose the information so I placed the
barcodes in the notes section of the housed_at_rlshp table. I hid the
database name for security reasons and replaced it with databasename.
update databasename.housed_at_rlshp a, databasename.container b set a.note
= CONCAT(a.note, ', ', b.barcode_1)
where a.container_id = b.id
and b.barcode_1 is not null;
update databasename.housed_at_rlshp a, databasename.container b set a.note
= b.barcode_1
where a.container_id = b.id
and b.barcode_1 is not null
and a.note is null;
update databasename.housed_at_rlshp a, databasename.container b set
b.barcode_1 = null
where a.container_id = b.id
and b.barcode_1 = a.note
and b.barcode_1 is not null
and a.note is not null;
Thanks for the feedback everyone!
Sorry if there's a duplicate, but I have not seen this posted to the
message boards so I posted a second time.
On Wed, Mar 29, 2017 at 8:31 PM, James Bullen <james at hudmol.com> wrote:
>
> Hi Patrick,
>
> This is a bit confusing because there are currently two container tables:
> ‘container’ and ‘top_container’.
>
> ‘container’ is the old container table - ie from before the container
> management plugin was merged in.
> ‘top_container’ is the new container table - ie it was introduced as part
> of the container management merge.
>
> The ‘container’ table should really have been dropped when the container
> management plugin was merged in, but that can’t be done yet because a bunch
> of code still depends on it, and there’s some cunning mapping that goes on
> between the two tables behind the scenes. It is on our list of priorities
> for the next major release (currently scheduled for June) to resolve this
> and a few other issues surrounding the container management merge.
>
> So, to your question about barcodes:
> container.barcode does not have a unique constraint.
> top_container.barcode must be unique within a repository.
>
> In the old container model, container records are nested sub-records, so
> many container records might refer to the same physical box. This means
> that is was not possible to have barcodes be unique. In the new container
> management model a top_container record represents a physical box, so it is
> possible to enforce unique barcodes in the database.
>
> Hope that helps.
>
>
> Cheers,
> James
>
>
> On Mar 30, 2017, at 5:27 AM, Majewski, Steven Dennis (sdm7g) <
> sdm7g at eservices.virginia.edu> wrote:
>
> PS: There was also Mark Cooper’s container migration notes:
> https://gist.github.com/mark-cooper/96892ab8734cf96a5a6ab0268107ab45
>
> And some Yale screencasts:
> http://guides.library.yale.edu/archivesspace/ASpaceContainerManagement
>
> — Steve.
>
>
> On Mar 29, 2017, at 2:21 PM, Majewski, Steven Dennis (sdm7g) <
> sdm7g at eservices.virginia.edu> wrote:
>
>
> On Mar 29, 2017, at 1:26 PM, Hawk, Patrick <hawkpf at miamioh.edu> wrote:
>
> Steve,
>
> I'm attaching a screen capture. The barcode_1 field needs to be unique in
> the table container?
>
>
>
> That’s my understanding, although I’m not certain if they have to be
> globally unique or just unique within that resource. ( I expect the
> intention is for them to be globally unique, but I’m not sure if they
> actually check beyond the current resource. ) I’m not sure, but I don’t
> think a barcode is required, so another option may be to just delete those
> barcodes. Or, as I initially suggested, append “box 1”, “box 2” , etc. to
> your existing barcodes to make them unique.
>
>
> I believe there was some discussion relating to these issues in the 1.5.0
> release webinar, probably in Noah’s talks and notes:
>
> http://archivesspace.org/recording-and-slides-for-v1-5-0-release-webinar/
>
> If you haven’t already, you should check it out, as it’s the easiest intro
> the what’s involved in the migration. ( I don’t think the upgrading docs in
> the distribution explain the terminology and background sufficiently. )
>
>
> — Steve.
>
>
> On Thu, Mar 16, 2017 at 1:10 PM, Majewski, Steven Dennis (sdm7g) <
> sdm7g at eservices.virginia.edu> wrote:
>
>>
>> I interpret that error message as saying that you are trying to assign
>> the same barcode ( "Miami University Archives (Western College for Women
>> Collection)” ) to “box 1” and “box 2” (type + indicator). Barcodes must be
>> unique. A collection name is not likely to be unique if there are more than
>> one containers. Typically, these are machine scannable barcode numbers. If
>> you really want to use the collection name here, you could maybe append the
>> box numbers to the ends of both, making them unique.
>>
>>
>> — Steve.
>>
>>
>> On Mar 16, 2017, at 12:10 PM, Hawk, Patrick <hawkpf at miamioh.edu> wrote:
>>
>> John,
>> Do you know which two accessions you had to modify?
>>
>> Also, this is what I see in archivesspace/logs/archivesspace.out
>>
>> <:ValidationException: {:errors=>{"indicator_1"=>["Mismatch when mapping
>> between indicator and indicator_1"]}, :object_context=>{:top_container=>#<TopContainer
>> @values={:id=>2, :repo_id=>2, :lock_version=>0, :json_schema_version=>1,
>> :barcode=>"Miami University Archives (Western College for Women
>> Collection)", :ils_holding_id=>nil, :ils_item_id=>nil,
>> :exported_to_ils=>nil, :indicator=>"1", :created_by=>"XXXX",
>> :last_modified_by=>"XXXX", :create_time=>2017-03-15 20:01:48 UTC,
>> :system_mtime=>2017-03-15 20:01:48 UTC, :user_mtime=>2017-03-15 20:01:48
>> UTC, :type_id=>318}>, :aspace_container=>{"lock_version"=>0,
>> "indicator_1"=>"2", "barcode_1"=>"Miami University Archives (Western
>> College for Women Collection)", "container_extent"=>"1.00",
>> "created_by"=>"XXXX", "last_modified_by"=>"XXXX",
>> "create_time"=>"2016-02-16T15:42:01Z", "system_mtime"=>"2016-11-15T18:16:09Z",
>> "user_mtime"=>"2016-02-16T15:42:01Z", "type_1"=>"box",
>> "container_extent_type"=>"Boxes", "jsonmodel_type"=>"container",
>> "container_locations"=>[{"status"=>"current",
>> "start_date"=>"1999-12-31", "system_mtime"=>2016-02-16 15:42:01 UTC,
>> "user_mtime"=>2016-02-16 15:42:01 UTC, "ref"=>"/locations/8584"}],
>> "type_id"=>318}}}>
>>
>>
>>
>> On Tue, Mar 14, 2017 at 3:40 PM, Hambleton, John S <jhamblet at nmu.edu>
>> wrote:
>>
>>> Hello!
>>>
>>>
>>>
>>> That thread below was us, Northern Michigan University.
>>>
>>> Looking in the container conversion log, was this error:
>>>
>>>
>>>
>>> Error:
>>>
>>> JSONModel::ValidationException
>>>
>>>
>>>
>>> Message:
>>>
>>> indicator_1 -- Mismatch when mapping between indicator and indicator_1
>>>
>>>
>>>
>>> James Bullen took a look at our log and determined that there was a
>>> barcode
>>>
>>> uniqueness problem between two accessions. So we cleaned that up and
>>> sure enough,
>>>
>>> that was it. We re-ran the conversion and the indexer did not have any
>>> problems.
>>>
>>>
>>>
>>> Hope this helps you.
>>>
>>>
>>>
>>> John H
>>>
>>> NMU
>>>
>>>
>>>
>>> *From:* archivesspace_users_group-bounces at lyralists.lyrasis.org [mailto:
>>> archivesspace_users_group-bounces at lyralists.lyrasis.org] *On Behalf Of *Hawk,
>>> Patrick
>>> *Sent:* Tuesday, March 14, 2017 3:27 PM
>>> *To:* Archivesspace Users Group <archivesspace_users_group at lyr
>>> alists.lyrasis.org>
>>> *Subject:* Re: [Archivesspace_Users_Group] Failure in periodic indexer
>>> worker thread
>>>
>>>
>>>
>>> Steve,
>>> The url you linked would be the same issue, so yes the indexer tries to
>>> index the same items over and over... I'll have to inspect the records to
>>> see what is causing the migration to fail.
>>>
>>>
>>>
>>> On Tue, Mar 14, 2017 at 3:13 PM, Majewski, Steven Dennis (sdm7g) <
>>> sdm7g at eservices.virginia.edu> wrote:
>>>
>>>
>>>
>>> Check for errors in the container conversion batch job log files.
>>>
>>>
>>>
>>> Does the indexer appear to keep trying to reindex the same records over
>>> and over ?
>>>
>>>
>>>
>>> See previous thread:
>>>
>>> http://lyralists.lyrasis.org/pipermail/archivesspace_users_g
>>> roup/2017-January/004370.html
>>>
>>>
>>>
>>> — Steve Majewski
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Mar 14, 2017, at 2:45 PM, Hawk, Patrick <hawkpf at miamioh.edu> wrote:
>>>
>>>
>>>
>>> Hello,
>>>
>>> I'm trying to upgrade Archivesspace from 1.4.2 to 1.5.3 but I'm having
>>> issues getting my content to fully load. I followed the instructions here:
>>> http://archivesspace.github.io/archivesspace/user/
>>> upgrading-to-a-new-release-of-archivesspace/. I removed the directory
>>> /archivesspace/data/indexter_state and /archivesspace/data/solr_index/index
>>> as the directions stated I should do when upgrading to 1.5.0. I also ran
>>> the setup-database.sh without issue.
>>>
>>>
>>>
>>> However, I receive the following error: Failure in periodic indexer
>>> worker thread: {"error":"undefined method `related_records' for
>>> nil:NilClass"} in the archivesspace.out file. At this point the logs
>>> suggest that all items have been re-indexed but my content does not fully
>>> load. Any suggestions?
>>>
>>> _______________________________________________
>>> Archivesspace_Users_Group mailing list
>>> Archivesspace_Users_Group at lyralists.lyrasis.org
>>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Archivesspace_Users_Group mailing list
>>> Archivesspace_Users_Group at lyralists.lyrasis.org
>>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>>>
>>>
>>>
>>>
>>>
>>> --
>>>
>>> Patrick Hawk
>>>
>>> King Library Systems
>>>
>>> King Library Room 314
>>>
>>> 513-529-6122 <(513)%20529-6122>
>>>
>>> _______________________________________________
>>> Archivesspace_Users_Group mailing list
>>> Archivesspace_Users_Group at lyralists.lyrasis.org
>>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>>>
>>>
>>
>>
>> --
>>
>> Patrick Hawk
>>
>>
>> King Library Systems
>>
>> King Library Room 314
>>
>> 513-529-6122 <(513)%20529-6122>
>> _______________________________________________
>> Archivesspace_Users_Group mailing list
>> Archivesspace_Users_Group at lyralists.lyrasis.org
>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>>
>>
>>
>> _______________________________________________
>> Archivesspace_Users_Group mailing list
>> Archivesspace_Users_Group at lyralists.lyrasis.org
>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>>
>>
>
>
> --
>
> Patrick Hawk
>
>
> King Library Systems
>
> King Library Room 314
>
> 513-529-6122 <(513)%20529-6122>
> <AS.PNG>_______________________________________________
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group at lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>
>
> _______________________________________________
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group at lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>
>
> !DSPAM:58dbfcac43573521512206!
> _______________________________________________
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group at lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>
>
> !DSPAM:58dbfcac43573521512206!
>
>
>
> _______________________________________________
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group at lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>
>
--
Patrick Hawk
King Library Systems
King Library Room 314
513-529-6122
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20170406/b56c97ee/attachment.html>
More information about the Archivesspace_Users_Group
mailing list