<div dir="ltr">The other thing I should have mentioned is that we chose to go the route of the API, but there is always the option of importing directly to the database. I'm curious if other institutions have done this. It would take an understanding of the table structures, which seemed quite complex to me, but perhaps not to a MySQL guru. The JSON was easier for me to absorb because it's actually simpler than XML and I don't have to wrap my head around a complex table structure. You definitely need to spend time understanding the JSON, but after a bit it wasn't that hard, particularly if you create a dummy record with the fields you want to migrate to.<div><br></div><div>I'm also wondering if we are in a unique situation. My observations (and this might not be accurate) is that a lot of institutions were either 1) already in Archon or some other archival management system or 2) starting from scratch. We never had an archival management system to migrate from. A database for accessions and some EAD, though we've still got plenty of paper records that fall outside of both of those data sources. OpenRefine was great for normalizing dates, disambiguating agent names, etc. In theory you could develop an EAD template to export and then upload to ASpace but that would be an insanely huge project.</div><div><br></div><div>I'd recommend a plugin for OpenRefine called BITS VIB. If you've got a bunch of OpenRefine projects, it makes it super easy to combine projects provided the records you want to join share a common identifier.</div><div><br></div><div>The other nice thing about OpenRefine is the Custom Tabular Export feature. If you've got a project with a bazillion columns, as I tend to have, you can select the columns you want to export, rearrange them, and set a custom delimiter. Other options too. You can just export a subset of your OpenRefine project depending on the facets and filters you've got selected. I would imagine it would be useful if you are importing directly to the database, though I've no experience here.</div><div><br></div><div>Thanks,</div><div>Olivia</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Feb 13, 2018 at 5:09 PM, Olivia S Solis <span dir="ltr"><<a href="mailto:livsolis@utexas.edu" target="_blank">livsolis@utexas.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">:) I'm a bit corgi obsessed.</div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Feb 13, 2018 at 8:54 AM, Margaret Kidd <span dir="ltr"><<a href="mailto:kiddm@vcu.edu" target="_blank">kiddm@vcu.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Totally off topic but, Olivia, your title slide might be the best ever!<div><br></div><div>Margaret</div></div><div class="gmail_extra"><br clear="all"><div><div class="m_1204943437834923138m_-783283539278659583gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><br></div><div><span><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6666666666667px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent"></span></p><span><span style="font-size:14.6666666666667px;font-family:Arial;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent"><hr></span></span><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6666666666667px;font-family:Arial;color:rgb(100,100,100);font-weight:bold;vertical-align:baseline;white-space:pre-wrap">Margaret Turman Kidd</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6666666666667px;font-family:Arial;color:rgb(100,100,100);vertical-align:baseline;white-space:pre-wrap">Access and Electronic Records Archivist, Special Collections & Archives</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6666666666667px;font-family:Arial;color:rgb(100,100,100);vertical-align:baseline;white-space:pre-wrap">VCU Libraries | Tompkins-McCaw Library for the Health Sciences</span></p><p style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13.3333333333333px;font-family:Arial;color:rgb(141,141,141);vertical-align:baseline;white-space:pre-wrap">509 N. 12th Street / Box 980582, Richmond, VA 23298-0582</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13.3333333333333px;font-family:Arial;color:rgb(141,141,141);vertical-align:baseline;white-space:pre-wrap"><a href="tel:(804)%20828-3152" value="+18048283152" target="_blank">(804) 828-3152</a></span></p><a href="https://twitter.com/VCUTMLibrary" style="text-decoration:none" target="_blank"><span style="font-size:13.3333333333333px;font-family:Arial;color:rgb(17,85,204);text-decoration:underline;vertical-align:baseline;white-space:pre-wrap"><img src="https://lh3.googleusercontent.com/CvwxmA9bdbubHH0SSvacsiCM35wJ3MMtdY8qFmOw7soBA46uNNHXG89hFi99w8XCP3MCPTkbSx7uxdAcW5YyV2eqviun-t6OoCTFe7aUbHhv3JJNonLo6sEH49IczUqAHOw8B8w" width="25px;" height="25px;" style="border:none" alt="em_twitter.png"></span></a><span style="font-size:13.3333333333333px;font-family:Arial;color:rgb(141,141,141);vertical-align:baseline;white-space:pre-wrap"> </span><a href="https://www.facebook.com/VCUTMLib" style="text-decoration:none" target="_blank"><span style="font-size:13.3333333333333px;font-family:Arial;color:rgb(17,85,204);text-decoration:underline;vertical-align:baseline;white-space:pre-wrap"><img src="https://lh6.googleusercontent.com/bBkgQaDHlTe4cYH8-0RSwlNxAs91Xv-D8Jewu8tOyHXmx4Efy0m8hspT1EvUAnFQTuqH8NcCOb8OndayaOVeZeKkpPxKcC8yI0Guylmkdithc0ZWyxfzPPFHt_6NDWmVNoQvr3M" width="25px;" height="25px;" style="border:none" alt="em_fb.png"></span></a></span><br></div><div><b><br></b></div><div><br></div><div><img><a href="http://www.vcu.edu/" target="_blank"><img src="http://identity.vcu.edu/images/photos/vcu_brand_mark_email_sig.jpg"></a> <a href="http://vaheritage.org" target="_blank"><img src="http://pages.shanti.virginia.edu/Virginia_Heritage/files/2013/01/vhpban.jpg" width="200" height="96"></a></div><div><span></span><span></span><br></div></div></div></div></div></div></div></div></div></div></div></div></div></div><div><div class="m_1204943437834923138h5">
<br><div class="gmail_quote">On Tue, Feb 13, 2018 at 9:25 AM, Olivia S Solis <span dir="ltr"><<a href="mailto:livsolis@utexas.edu" target="_blank">livsolis@utexas.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi there,<div><br></div><div>Yes it is definitely possible to use OpenRefine to migrate data into the system. It's my primary tool for our data migration. One of the nice things about OpenRefine is its Templating export option. I was inspired by an extremely helpful University of Maryland Chaos to Order post:</div><div><a href="https://icantiemyownshoes.wordpress.com/2015/11/20/how-i-learned-to-stop-worrying-and-love-the-api/" target="_blank">https://icantiemyownshoes.word<wbr>press.com/2015/11/20/how-i-lea<wbr>rned-to-stop-worrying-and-love<wbr>-the-api/</a><br></div><div><br></div><div>That is how we've been creating records and we have developed a number of templates to export the JSON. There are a few differences in our process. For instance, the bash script didn't like spaces in content, so we added a line </div><div>IFS=$| </div><div><br></div><div>and all of my templates end in a pipe.</div><div><br></div><div>To update records, we were inspired by a Duke Python script:</div><div><a href="https://github.com/duke-libraries/archivesspace-duke-scripts/blob/master/python/duke_archival_object_metadata_adder.py" target="_blank">https://github.com/duke-librar<wbr>ies/archivesspace-duke-scripts<wbr>/blob/master/python/duke_archi<wbr>val_object_metadata_adder.py</a><br></div><div><a href="https://blogs.library.duke.edu/bitstreams/2016/09/21/archivesspace-api-fun/" target="_blank">https://blogs.library.duke.edu<wbr>/bitstreams/2016/09/21/archive<wbr>sspace-api-fun/</a><br></div><div>as described in the blog post above.</div><div><br></div><div>So for instance, we migrated our EAD by using BaseX to extract elements. I created minimal resource records and incrementally added notes.</div><div><br></div><div>The process is somewhat laid out in the slides I made for a presentation:</div><div><a href="https://docs.google.com/presentation/d/1cBrd8qzHK4i8S6SQ_vDlScERixX95lAbE__hd48yIDg/edit?usp=sharing" target="_blank">https://docs.google.com/presen<wbr>tation/d/1cBrd8qzHK4i8S6SQ_vDl<wbr>ScERixX95lAbE__hd48yIDg/edit?u<wbr>sp=sharing</a><br></div><div><br></div><div>Hopefully, this helps.</div><div><br></div><div>-Olivia</div><div><br></div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="m_1204943437834923138m_-783283539278659583h5">On Mon, Feb 12, 2018 at 9:45 AM, Joan Curbow <span dir="ltr"><<a href="mailto:CurbowJ@bvu.edu" target="_blank">CurbowJ@bvu.edu</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="m_1204943437834923138m_-783283539278659583h5">
<div lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="m_1204943437834923138m_-783283539278659583m_-8389237142535732969m_-7486821003269065284WordSection1">
<p class="MsoNormal">We are a new archives, so we have not had to import any existing data. Lucky us, right? But I’m all-too-human, and I’d now like to do some data cleanup. Is it possible to do data cleanup using OpenRefine? Theoretically, it seems possible
to export “stuff” and pull it into OpenRefine, but I’ve only ever used OpenRefine in a classroom situation, where the data was already populated for us. We did not import the data back into an existing database, either, so my experience is limited to just
the mechanics of OpenRefine. Has anyone used OpenRefine in a real-world situation with data that’s already in Aspace? Or is there a better method for data cleanup?<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">A further question/complication is that I’m a lone arranger, and my instance is hosted by Libraryhost, so any data cleanup may have to be done by them? My tech skills are rudimentary, so I’m not clear just how I could get this to work.
I asked them once, but didn’t get a real answer.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Sincerely,<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><b><span style="font-size:12.0pt;font-family:"Times New Roman",serif;color:#002244">Joan Curbow</span></b><span style="font-size:12.0pt;font-family:"Times New Roman",serif"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman",serif;color:#90a4b7">Reference Librarian and Archivist</span><span style="font-size:12.0pt;font-family:"Times New Roman",serif"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman",serif;color:#90a4b7">Buena Vista University Library<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman",serif;color:#90a4b7">Buena Vista University<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman",serif;color:#90a4b7">610 West Fourth Street<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman",serif;color:#90a4b7">Storm Lake, Iowa 50588</span><span style="font-size:12.0pt;font-family:"Times New Roman",serif"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman",serif;color:#90a4b7"><a href="tel:(712)%20749-2094" value="+17127492094" target="_blank">712.749.2094</a><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman",serif;color:#90a4b7"><a href="http://www.library.bvu.edu" target="_blank">www.library.bvu.edu</a></span><span style="font-size:12.0pt;font-family:"Times New Roman",serif"><u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<br></div></div>______________________________<wbr>_________________<br>
Archivesspace_Users_Group mailing list<br>
<a href="mailto:Archivesspace_Users_Group@lyralists.lyrasis.org" target="_blank">Archivesspace_Users_Group@lyra<wbr>lists.lyrasis.org</a><br>
<a href="http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group" rel="noreferrer" target="_blank">http://lyralists.lyrasis.org/m<wbr>ailman/listinfo/archivesspace_<wbr>users_group</a><br>
<br></blockquote></div><span class="m_1204943437834923138m_-783283539278659583HOEnZb"><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div class="m_1204943437834923138m_-783283539278659583m_-8389237142535732969gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div style="font-size:12.8px"><font color="#000000">Olivia Solis, MSIS</font></div><div style="font-size:12.8px"><font color="#000000">Metadata Coordinator</font></div><div style="font-size:12.8px"><font color="#000000">Dolph Briscoe Center for American History</font></div><div style="font-size:12.8px"><font color="#000000">The University of Texas at Austin</font></div><div style="font-size:12.8px"><font color="#000000">2300 Red River St. Stop D1100</font></div><div style="font-size:12.8px"><font color="#000000">Austin TX, 78712-1426</font></div><div style="color:rgb(136,136,136);font-size:12.8px"><span style="color:rgb(0,0,0);font-size:12.8px"><a href="tel:(512)%20232-8013" value="+15122328013" target="_blank">(512) 232-8013</a></span></div></div></div></div></div></div></div>
</font></span></div>
<br>______________________________<wbr>_________________<br>
Archivesspace_Users_Group mailing list<br>
<a href="mailto:Archivesspace_Users_Group@lyralists.lyrasis.org" target="_blank">Archivesspace_Users_Group@lyra<wbr>lists.lyrasis.org</a><br>
<a href="http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group" rel="noreferrer" target="_blank">http://lyralists.lyrasis.org/m<wbr>ailman/listinfo/archivesspace_<wbr>users_group</a><br>
<br></blockquote></div><br></div></div></div>
<br>______________________________<wbr>_________________<br>
Archivesspace_Users_Group mailing list<br>
<a href="mailto:Archivesspace_Users_Group@lyralists.lyrasis.org" target="_blank">Archivesspace_Users_Group@lyra<wbr>lists.lyrasis.org</a><br>
<a href="http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group" rel="noreferrer" target="_blank">http://lyralists.lyrasis.org/m<wbr>ailman/listinfo/archivesspace_<wbr>users_group</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="m_1204943437834923138gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div style="font-size:12.8px"><font color="#000000">Olivia Solis, MSIS</font></div><div style="font-size:12.8px"><font color="#000000">Metadata Coordinator</font></div><div style="font-size:12.8px"><font color="#000000">Dolph Briscoe Center for American History</font></div><div style="font-size:12.8px"><font color="#000000">The University of Texas at Austin</font></div><div style="font-size:12.8px"><font color="#000000">2300 Red River St. Stop D1100</font></div><div style="font-size:12.8px"><font color="#000000">Austin TX, 78712-1426</font></div><div style="color:rgb(136,136,136);font-size:12.8px"><span style="color:rgb(0,0,0);font-size:12.8px"><a href="tel:(512)%20232-8013" value="+15122328013" target="_blank">(512) 232-8013</a></span></div></div></div></div></div></div></div>
</div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div style="font-size:12.8px"><font color="#000000">Olivia Solis, MSIS</font></div><div style="font-size:12.8px"><font color="#000000">Metadata Coordinator</font></div><div style="font-size:12.8px"><font color="#000000">Dolph Briscoe Center for American History</font></div><div style="font-size:12.8px"><font color="#000000">The University of Texas at Austin</font></div><div style="font-size:12.8px"><font color="#000000">2300 Red River St. Stop D1100</font></div><div style="font-size:12.8px"><font color="#000000">Austin TX, 78712-1426</font></div><div style="color:rgb(136,136,136);font-size:12.8px"><span style="color:rgb(0,0,0);font-size:12.8px">(512) 232-8013</span></div></div></div></div></div></div></div>
</div>