<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><br><div><div>On Feb 18, 2014, at 2:40 PM, Noah Huffman <<a href="mailto:noah.huffman@duke.edu">noah.huffman@duke.edu</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div lang="EN-US" link="blue" vlink="purple" style="font-family: Helvetica; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><div class="WordSection1" style="page: WordSection1;"><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">Hello,<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">As others have mentioned, some of the EAD import error log messages are rather cryptic. Could anyone help decipher the two messages below? After running a few test imports, I’m getting the first “wrong number of arguments” error quite a bit on schema valid EAD files.<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div><pre style="margin: 0in 0in 7.5pt; font-size: 10pt; font-family: 'Courier New'; line-height: 15pt; background-color: rgb(51, 51, 51); word-break: break-all; background-position: initial initial; background-repeat: initial initial;"><span style="font-size: 9pt; color: rgb(51, 238, 51);">Error: wrong number of arguments (6 for 4)<o:p></o:p></span></pre><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div><p class="MsoNormal" style="margin: 0in 0in 7.5pt; font-size: 11pt; font-family: Calibri, sans-serif; line-height: 15pt; background-color: rgb(51, 51, 51); word-break: break-all; background-position: initial initial; background-repeat: initial initial;"><span style="font-size: 9pt; font-family: 'Courier New'; color: rgb(51, 238, 51);">Error: Unexpected Object Type in Queue: Expected archival_object got file_version<o:p></o:p></span></p><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">Also, is there any method for batch importing EAD that will allow the entire batch to process even if one particular file fails?<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">Thanks,<o:p></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">-Noah<o:p></o:p></div></div></div></blockquote><br></div><div><br></div><div>Here’s what we’re hacked together: </div><div><br></div><div>[1] created a top-level env.sh file with PATH settings and env variables cribbed from scripts/jirb:</div><div><br></div><div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;"># cd to archivespace top-level directory and source this file</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); min-height: 20px; position: static; z-index: auto;"><br></div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;">export BUNDLE_GEMFILE="$PWD/backend/Gemfile"</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;">JAVA_OPTS="$JAVA_OPTS -Daspace.config.search_user_secret=devserver -Daspace.config.public_user_secret=devserver -Daspace.config.staff_user_secret=devserver"</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;">export JAVA_OPTS</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;">export RUBYLIB=$PWD/common/</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;">PATH=$PWD/build/gems/bin:$PATH:$PWD/scripts:$PWD/backend/scripts</div><div><br></div></div><br><div><br></div><div>[2] created a Ruby script in backend/scripts/ead_parse.rb</div><div>( figured out the gist of this from looking at spec test: backend/spec/lib_ead_converter_spec.rb )</div><div><br></div><div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;">#!/usr/bin/env jruby</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;">#</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;"># script attempts to parse files with EADConverter</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;">#</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;"># You need to source env.sh to get settings for </div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;"># JAVA_OPTS, RUBYLIB, etc. before running the script. </div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;">#</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); min-height: 20px; position: static; z-index: auto;"><br></div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;">require_relative '../spec/spec_helper'</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;">require_relative '../app/converters/ead_converter'</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); min-height: 20px; position: static; z-index: auto;"><br></div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); min-height: 20px; position: static; z-index: auto;"><br></div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;">ARGV.each do |eadxml|</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;"><span class="Apple-tab-span" style="white-space:pre"> </span>begin</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;"><span class="Apple-tab-span" style="white-space:pre"> </span>converter = EADConverter.new( eadxml )</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;"><span class="Apple-tab-span" style="white-space:pre"> </span>converter.run</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;"><span class="Apple-tab-span" style="white-space:pre"> </span>puts eadxml + " : ok."</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;"><span class="Apple-tab-span" style="white-space:pre"> </span># if parse is successful, then write out json for later import</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;"><span class="Apple-tab-span" style="white-space:pre"> </span>outname = eadxml.slice(0..eadxml.rindex('.')) + 'json' </div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;"><span class="Apple-tab-span" style="white-space:pre"> </span>out = File::open( outname, 'w' )</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;"><span class="Apple-tab-span" style="white-space:pre"> </span>puts "writing json to: " + outname</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;"><span class="Apple-tab-span" style="white-space:pre"> </span>out.write(IO.read(converter.get_output_path))</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;"><span class="Apple-tab-span" style="white-space:pre"> </span>out.close</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;"><span class="Apple-tab-span" style="white-space:pre"> </span>rescue Exception => e</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;"><span class="Apple-tab-span" style="white-space:pre"> </span>puts eadxml + " : failed: " + e.message</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;"><span class="Apple-tab-span" style="white-space:pre"> </span>$stderr.puts e.backtrace</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;"><span class="Apple-tab-span" style="white-space:pre"> </span>end</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;">end</div></div><div><br></div><div><br></div><div>There will be a lot of stack traces on stderr, but stdout will just be a listing of success and failures along with the JSON output filenames. </div><div><br></div><div><br></div><div><br></div><div><br></div><div>Running that script on a directory full of EAD xml files will leave behind .json files for the ones that successfully parse.</div><div><br></div><div>You can then import the json files with something like ( where $N = repository number ) : </div><div><br></div><div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;">for JSON in *.json</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;">> do</div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;">> curl_as_osx admin admin -d @$JSON <a href="http://localhost:8089/repositories/$N/batch_imports">http://localhost:8089/repositories/$N/batch_imports</a> </div><div style="margin: 0px; font-size: 17px; font-family: Menlo; background-color: rgb(178, 182, 177); position: static; z-index: auto;">> done</div></div><div><br></div><div><br></div><div>An advantage of the separate parse & import is that you can do all of the parsing on your development laptop, </div><div>but post to another server’s backend port. </div><div><br></div><div><br></div><div>— Steve Majewski</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></body></html>