[Archivesspace_Users_Group] Plugin: Custom import mapping for MarcXML Agents, help

Mon Nov 9 17:44:12 EST 2015

Hello Brian,

Sorry to bring this up again, but something came up and I have a quick
question.

What does the importer check to see whether it is creating a new
ArchivesSPace record or modifying one that it has found?

What we've accomplished works great for new records, but it seems that, for
example, if there is a record with an empty AuthorityId and I want to
modify that record to have an AuthorityId by importing it's authority file
the AuthorityId is not updated in the ArchivesSpace record.

My guess, from running several tests, is that the authority id is not
updated because even though the record we have imported has the AuthorityId
and even the same primaryname and source as the record in ArchivesSpace;
because, the ArchviesSpace record does not have the authorityId it does not
think it is modifying the file. This is just a guess of course, I can't
really tell what may be going on.

Any insight on this?

Thank you,
Carlos Lemus

On Wed, Oct 28, 2015 at 4:19 PM, brian <brianjhoffman at gmail.com> wrote:

> Glad we got that working! Definitely not a use case envisioned by the
> original  importer design committee!
>
>
> -------- Original message --------
> From: Carlos Lemus <carlos.lemus at unlv.edu>
> Date:10/28/2015 3:06 PM (GMT-05:00)
> To: Brian Hoffman <brianjhoffman at gmail.com>
> Cc: Archivesspace Users Group <
> archivesspace_users_group at lyralists.lyrasis.org>, Karla Irwin <
> karla.irwin at unlv.edu>, Cynthia Shein <cyndi.shein at unlv.edu>, Carol Ou <
> carol.ou at unlv.edu>
> Subject: Re: Plugin: Custom import mapping for MarcXML Agents, help
>
> Hello Brian,
>
> Thank you for all your help! it's working now.
>
> Carlos Lemus
>
> On Wed, Oct 28, 2015 at 7:40 AM, Brian Hoffman <brianjhoffman at gmail.com>
> wrote:
>
>> Hi Carlos,
>>
>> I think you can solve this by inserting above line 24:
>>
>> return false unless
>> AgentManager.known_agent_type?(record.class.record_type) || record.class.record_type == ‘subject'
>>
>> Brian
>>
>> On Oct 27, 2015, at 9:31 PM, Carlos Lemus <carlos.lemus at unlv.edu> wrote:
>>
>> Hello Brian,
>>
>> I just noticed something actually, that lines 47-50 are preventing a
>> proper return value
>>
>>
>> https://github.com/l3mus/ArchivesSpace-authority-project/blob/master/unlv_importer/backend/model/unlv_marcxml_converter.rb#L47-L50
>>
>> After removing those log values I get something like this. It's so close,
>> I think the right agent is there, but now it's trying to export a resource,
>> instead of an agent.
>>
>> extents : At least 1 item(s) is required dates : At least 1 item(s) is
>> required title : Property is required but was missing For
>> JSONModel(:resource): #<JSONModel(:resource) {"jsonmodel_type"=>"resource",
>> "external_ids"=>[], "subjects"=>[], "linked_events"=>[], "extents"=>[],
>> "dates"=>[], "external_documents"=>[], "rights_statements"=>[],
>> "linked_agents"=>[{"role"=>"subject", "terms"=>[],
>> "ref"=>"/agents/people/import_fda55473-b8f5-43c1-85a3-955b37a57c28"}],
>> "restrictions"=>false, "revision_statements"=>[], "instances"=>[],
>> "deaccessions"=>[], "related_accessions"=>[], "classifications"=>[],
>> "notes"=>[],
>> "uri"=>"/repositories/import/resources/import_4b9b1731-28ba-4e9b-afd6-ed2d933ce882",
>> "finding_aid_description_rules"=>"rda",
>> "id_0"=>"imported-e5c9504d-33ea-4538-91ef-54fdc2cd67d5",
>> "level"=>"collection"}> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>>
>> Thank you,
>>
>> Carlos Lemus
>>
>> On Tue, Oct 27, 2015 at 3:17 PM, Carlos Lemus <carlos.lemus at unlv.edu>
>> wrote:
>>
>>> Hello Brian,
>>>
>>> Sorry to keep bothering you with this, I was wondering if you got a
>>> chance to take a look at this.
>>>
>>> https://github.com/l3mus/ArchivesSpace-authority-project/tree/master/unlv_importer
>>>
>>> Thank you,
>>>
>>> Carlos Lemus
>>>
>>> On Tue, Oct 6, 2015 at 3:01 PM, Carlos Lemus <carlos.lemus at unlv.edu>
>>> wrote:
>>>
>>>> Hey Brian,
>>>>
>>>> Thank you for the reply, I have been able to piggy back of your code
>>>> quite a bit which I'll update on github, but there is still some small
>>>> issues. I've had to change the code a little bit to get it to work. For
>>>> example it won't add the authority_id unless I first declare it because
>>>> it's trying to add the authority_id to a nil object and then the source is
>>>> duplicating multiple times. Also, the to_hash function wasn't seeming to
>>>> return anything but to_hash(:raw) did. I have attached an example marcxml
>>>> just in case.
>>>>
>>>> Looking at the record, it doesn't seem like I'm even working with the
>>>> right one. They all have different uri, especially the last one that is
>>>> giving me the error. I think that last one that is giving me the error
>>>> might be the one that actually needs to be changed, but I can't find where
>>>> it's coming from. I'll keep looking into it.
>>>>
>>>> I've got the other object to come out right
>>>> #<JSONModel(:agent_person) {"jsonmodel_type"=>"agent_person",
>>>> "agent_contacts"=>[], "linked_agent_roles"=>[], "external_documents"=>[],
>>>> "rights_statements"=>[], "notes"=>[], "dates_of_existence"=>[],
>>>> "places"=>[], "names"=>[#<JSONModel(:name_person)
>>>> {"jsonmodel_type"=>"name_person", "use_dates"=>[], "authorized"=>false,
>>>> "is_display_name"=>false, "sort_name_auto_generate"=>true, "uri"=>nil,
>>>> "primary_name"=>"Goodman", "rest_of_name"=>"Oscar Baylin,",
>>>> "dates"=>"1939-", "name_order"=>"inverted", "source"=>"ingest ingest
>>>> ingest", "authority_id"=>"  n 2003051366"}>], "related_agents"=>[],
>>>> "uri"=>"/agents/people/import_8a20f2f4-3560-41de-92b1-2aa888253143",
>>>> "dates"=>"1939-"}>
>>>>
>>>> and the record is still the same, maybe I should be updating this one
>>>> instead I'm not sure.
>>>> #<JSONModel(:agent_person) {"jsonmodel_type"=>"agent_person",
>>>> "agent_contacts"=>[], "linked_agent_roles"=>[], "external_documents"=>[],
>>>> "rights_statements"=>[], "notes"=>[], "dates_of_existence"=>[],
>>>> "places"=>[], "names"=>[#<JSONModel(:name_person)
>>>> {"jsonmodel_type"=>"name_person", "use_dates"=>[], "authorized"=>false,
>>>> "is_display_name"=>false, "sort_name_auto_generate"=>true, "uri"=>nil,
>>>> "authority_id"=>"n 2003051366", "name_order"=>"direct",
>>>> "source"=>"ingest"}>], "related_agents"=>[],
>>>> "uri"=>"/agents/people/import_308b3936-d294-4e4e-96cb-e9d35447f8cb"}>
>>>>
>>>> but I'm getting an error for a completely different one
>>>> names/0/primary_name : Property is required but was missing For
>>>> JSONModel(:agent_person): #<JSONModel(:agent_person)
>>>> {"jsonmodel_type"=>"agent_person", "agent_contacts"=>[],
>>>> "linked_agent_roles"=>[], "external_documents"=>[],
>>>> "rights_statements"=>[], "notes"=>[], "dates_of_existence"=>[],
>>>> "places"=>[], "names"=>[{"jsonmodel_type"=>"name_person", "use_dates"=>[],
>>>> "authorized"=>false, "is_display_name"=>false,
>>>> "sort_name_auto_generate"=>true, "uri"=>nil, "authority_id"=>"n
>>>> 2003051366", "name_order"=>"direct", "source"=>"ingest"}],
>>>> "related_agents"=>[],
>>>> "uri"=>"/agents/people/import_40843311-b96a-4e59-9ebb-2d685e3e63ea"}>
>>>>
>>>> Thank you
>>>> Carlos Lemus
>>>>
>>>> On Tue, Oct 6, 2015 at 10:05 AM, Brian Hoffman <brianjhoffman at gmail.com
>>>> > wrote:
>>>>
>>>>> Hi Carlos,
>>>>>
>>>>> I sent a pull request with an example of how you could merge names
>>>>> together. I hope this helps.
>>>>>
>>>>>
>>>>> https://github.com/UNLV-Libraries/ArchivesSpace-authority-project/pull/2
>>>>>
>>>>> Brian
>>>>>
>>>>>
>>>>> On Sep 28, 2015, at 3:33 PM, Carlos Lemus <carlos.lemus at unlv.edu>
>>>>> wrote:
>>>>>
>>>>> Hello Brian,
>>>>>
>>>>> I was just checking if you got a chance to look at this issue. I am
>>>>> still having some trouble merging the two names together and was wondering
>>>>> if there was something else I needed to do.
>>>>>
>>>>> Thank you,
>>>>>
>>>>> Carlos Lemus
>>>>>
>>>>> On Mon, Sep 21, 2015 at 3:34 PM, Carlos Lemus <carlos.lemus at unlv.edu>
>>>>> wrote:
>>>>>
>>>>>> Sorry Brian, the message got cut off. It's concatenating the names
>>>>>> to the agent accordingly, however the names are still separated and I
>>>>>> believe it is still looking for a primary name in that name
>>>>>>
>>>>>> Here is an example of the JSON
>>>>>>
>>>>>> For JSONModel(:agent_person): #<JSONModel(:agent_person)
>>>>>> {"jsonmodel_type"=>"agent_person", "agent_contacts"=>[],
>>>>>> "linked_agent_roles"=>[], "external_documents"=>[],
>>>>>> "rights_statements"=>[], "notes"=>[], "dates_of_existence"=>[],
>>>>>> "names"=>[{"jsonmodel_type"=>"name_person", "use_dates"=>[],
>>>>>> "authorized"=>false, "is_display_name"=>false,
>>>>>> "sort_name_auto_generate"=>true, "uri"=>nil, "primary_name"=>"Goodman",
>>>>>> "rest_of_name"=>"Oscar Baylin,", "dates"=>"1939-",
>>>>>> "name_order"=>"inverted", "source"=>"ingest"},
>>>>>> {"jsonmodel_type"=>"name_person", "use_dates"=>[], "authorized"=>false,
>>>>>> "is_display_name"=>false, "sort_name_auto_generate"=>true, "uri"=>nil,
>>>>>> "authority_id"=>"n 2003051366", "name_order"=>"direct",
>>>>>> "source"=>"ingest"}], "related_agents"=>[],
>>>>>> "uri"=>"/agents/people/import_4cd10c10-330e-4d71-916d-1f354f4e1f89"}>
>>>>>>
>>>>>> and here is the code for the plugin
>>>>>> https://github.com/UNLV-Libraries/ArchivesSpace-authority-project/tree/master/unlv_importer
>>>>>>
>>>>>> I am still having trouble making it only one name_person instead of
>>>>>> two under the agent person. I appreciate your help, sorry again for the
>>>>>> delay and the double message.
>>>>>>
>>>>>> Thank you,
>>>>>> Carlos Lemus
>>>>>>
>>>>>> On Mon, Sep 21, 2015 at 3:30 PM, Carlos Lemus <carlos.lemus at unlv.edu>
>>>>>> wrote:
>>>>>>
>>>>>>> Hello Brian,
>>>>>>>
>>>>>>> Sorry for getting back to you so late, I have been attempting to
>>>>>>> implement your code. It's concatenating the names to the agent accordingly,
>>>>>>> however the names are still separated and I believe it is still looking for
>>>>>>> a primary name in that name
>>>>>>>
>>>>>>> Here is an example of the JSON
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Sep 11, 2015 at 6:55 AM, Brian Hoffman <
>>>>>>> brianjhoffman at gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Carlos,
>>>>>>>>
>>>>>>>> Yes, that’s actually something I overlooked. The way the importers
>>>>>>>> are constructed, you can’t actually access the queue of records to import
>>>>>>>> from the configuration code. So, I posted an example of a workaround (and
>>>>>>>> some workarounds for a few other issues with the way importers are
>>>>>>>> sub-classed):
>>>>>>>>
>>>>>>>>
>>>>>>>> https://github.com/quoideneuf/aspace_single_agent_marcxml/blob/master/backend/model/marcxml_accession_converter.rb
>>>>>>>>
>>>>>>>> Brian
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sep 10, 2015, at 12:44 AM, Carlos Lemus <carlos.lemus at unlv.edu>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hello Brian,
>>>>>>>>
>>>>>>>> Sorry if I'm not replying to our current thread correctly. I have
>>>>>>>> yet to set up git for the plugin, but as soon as I do I will let you know.
>>>>>>>> It is mostly some of the yale marcxml plugin setup and a lot of commented
>>>>>>>> trial and error code at this point anyways.
>>>>>>>>
>>>>>>>> I tried the code you gave me and I think you understand perfectly
>>>>>>>> what my issue is and I understand what you're trying to do by matching the
>>>>>>>> agent to the current resource, exactly what I need.
>>>>>>>> However, it's not working because the batch object seems to be
>>>>>>>> Null. Not sure if maybe I'm required to import something.
>>>>>>>>
>>>>>>>> Error: #<NoMethodError: undefined method `working_area' for
>>>>>>>> nil:NilClass>
>>>>>>>>
>>>>>>>> Here is the current state of my code
>>>>>>>>
>>>>>>>> UNLVMarcXMLAgentsConverter.configure do |config|
>>>>>>>> config["/record"][:map]["controlfield[@tag='001']"] = -> resource,
>>>>>>>> node {
>>>>>>>> existing_agent_uri = resource.linked_agents.find {|link| link[:ref]
>>>>>>>> =~ /people/ }
>>>>>>>> existing_agent = @batch.working_area.find {|obj| obj.uri ==
>>>>>>>> existing_agent_uri }
>>>>>>>> make(:name_person) do |name|
>>>>>>>> name.primary = node.xpath("subfield[@code='a']").inner_text
>>>>>>>> # add more name fields as necessary
>>>>>>>> val = node.inner_text
>>>>>>>> name['authority_id'] = val
>>>>>>>> existing_agent.names << name
>>>>>>>> end
>>>>>>>> }
>>>>>>>> end
>>>>>>>> Thanks for the quick reply by the way. I would have gotten to you
>>>>>>>> sooner, but I didn't get an email that you had replied. I'll keep
>>>>>>>> an eye on the listserv from now on.
>>>>>>>>
>>>>>>>> Carlos Lemus
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20151109/7945b545/attachment.html>