<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div class=""><br class="">
</div>
<div class="">Do you have a sample import file that fails this way ? </div>
<div class="">Do you know if it still fail on current release ? </div>
<div class="">( and is bug reported on Jira ? ) </div>
<div class=""><br class="">
</div>
<div class="">— Steve. </div>
<div class=""><br class="">
</div>
<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Feb 15, 2017, at 3:25 PM, Lisa Calahan <<a href="mailto:lcalahan@umn.edu" class="">lcalahan@umn.edu</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">I've also received the same UTF8 error when importing legacy accession records that have
<b class="">valid</b> diacritical marks in the title and/or agent name.
<div class=""><br class="">
</div>
<div class="">Lisa</div>
</div>
<div class="gmail_extra"><br class="">
<div class="gmail_quote">On Wed, Feb 15, 2017 at 2:17 PM, Reese, Terry P. <span dir="ltr" class="">
<<a href="mailto:reese.2179@osu.edu" target="_blank" class="">reese.2179@osu.edu</a>></span> wrote:<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple" class="">
<div class="m_8494465543217940090WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif" class="">I guess my question would be – is your legacy data UTF8? For whatever reason, I’ve found that historically, Archives have often used other charactersets when encoding
their EAD files (though to be fair, I see this in MARC records as well; confusion between MARC8, ISO8859-1, and codepage 1252). The simply solution (and this would maintain your characters) would be to convert the character set to UTF8. Otherwise, even if
you held on to these values – they wouldn’t display in any form that you could read; and in fact, that is what the error message is trying to tell you. That as a UTF8 value, your data is going to be gibberish, regardless of if you keep it or not.<u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif" class=""><u class=""></u> <u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif" class="">--tr<u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif" class=""><u class=""></u> <u class=""></u></span></p>
<div class="">
<div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0in 0in 0in" class="">
<p class="MsoNormal"><b class=""><span style="font-size:11.0pt;font-family:"Calibri",sans-serif" class="">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif" class="">
<a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org" target="_blank" class="">
archivesspace_users_group-<wbr class="">bounces@lyralists.lyrasis.org</a> [mailto:<a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org" target="_blank" class="">archivesspace_users_<wbr class="">group-bounces@lyralists.<wbr class="">lyrasis.org</a>]
<b class="">On Behalf Of </b>Stasiulatis, Suzanne<br class="">
<b class="">Sent:</b> Wednesday, February 15, 2017 3:12 PM</span></p>
<div class="">
<div class="h5"><br class="">
<b class="">To:</b> Archivesspace Users Group <<a href="mailto:archivesspace_users_group@lyralists.lyrasis.org" target="_blank" class="">archivesspace_users_group@<wbr class="">lyralists.lyrasis.org</a>><br class="">
<b class="">Subject:</b> Re: [Archivesspace_Users_Group] Enumerations Findings<u class=""></u><u class=""></u></div>
</div>
<div class=""><br class="webkit-block-placeholder">
</div>
</div>
</div>
<div class="">
<div class="h5">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d" class="">I totally agree that we shouldn’t have special characters if at all possible, but a large amount of our legacy data uses them. Especially in titles,
staff want to use those characters as they are reflected on original materials.<u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d" class=""><u class=""></u> <u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d" class="">Suzanne<u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d" class=""><u class=""></u> <u class=""></u></span></p>
<div class="">
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in" class="">
<p class="MsoNormal"><b class=""><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif" class="">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif" class="">
<a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org" target="_blank" class="">
archivesspace_users_group-<wbr class="">bounces@lyralists.lyrasis.org</a> [<a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org" target="_blank" class="">mailto:archivesspace_users_<wbr class="">group-bounces@lyralists.<wbr class="">lyrasis.org</a>]
<b class="">On Behalf Of </b>Reese, Terry P.<br class="">
<b class="">Sent:</b> Wednesday, February 15, 2017 2:58 PM<br class="">
<b class="">To:</b> Archivesspace Users Group<br class="">
<b class="">Subject:</b> Re: [Archivesspace_Users_Group] Enumerations Findings<u class=""></u><u class=""></u></span></p>
</div>
</div>
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif" class="">Why would you want to retain invalid special characters? My guess is that one of the reasons for this error is that invalid characters would cause problems with indexing
for search, as well as impact display and export. I would think you’d want to use the error as a flag to identify data that needs to be corrected. Or am I missing something?<u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif" class=""><u class=""></u> <u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif" class="">--tr<u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif" class=""><u class=""></u> <u class=""></u></span></p>
<div class="">
<div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0in 0in 0in" class="">
<p class="MsoNormal"><b class=""><span style="font-size:11.0pt;font-family:"Calibri",sans-serif" class="">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif" class="">
<a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org" target="_blank" class="">
archivesspace_users_group-<wbr class="">bounces@lyralists.lyrasis.org</a> [<a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org" target="_blank" class="">mailto:archivesspace_users_<wbr class="">group-bounces@lyralists.<wbr class="">lyrasis.org</a>]
<b class="">On Behalf Of </b>Stasiulatis, Suzanne<br class="">
<b class="">Sent:</b> Wednesday, February 15, 2017 2:52 PM<br class="">
<b class="">To:</b> Archivesspace Users Group <<a href="mailto:archivesspace_users_group@lyralists.lyrasis.org" target="_blank" class="">archivesspace_users_group@<wbr class="">lyralists.lyrasis.org</a>><br class="">
<b class="">Subject:</b> Re: [Archivesspace_Users_Group] Enumerations Findings<u class=""></u><u class=""></u></span></p>
</div>
</div>
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d" class="">This also came up for me recently. If invalid special characters are present in the content titles, I get this error. I’m not sure quite how to adjust
to accept those special characters.<u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d" class=""><u class=""></u> <u class=""></u></span></p>
<p class="MsoNormal"><span id="cid:image002.png@01D2879E.AE4E3830"><image002.png></span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d" class=""><u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d" class=""><u class=""></u> <u class=""></u></span></p>
<p class="MsoNormal"><b class=""><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:navy" class="">Suzanne Stasiulatis
</span></b><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:navy" class="">| Archivist II<br class="">
Pennsylvania Historical and Museum Commission<b class=""> </b>| Pennsylvania State Archives<br class="">
350 North Street | Harrisburg, PA 17120-0090<u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:navy" class="">Phone:
<a href="tel:(717)%20787-5953" value="+17177875953" target="_blank" class="">717-787-5953</a><u class=""></u><u class=""></u></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:#003e7e" class=""><a href="http://www.phmc.pa.gov/" target="_blank" class="">http://www.phmc.pa.gov</a><u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-family:"Calibri",sans-serif;color:#17365d" class=""><a href="mailto:sustasiula@pa.gov" target="_blank" class="">sustasiula@pa.gov</a></span><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#17365d" class=""><u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d" class=""><u class=""></u> <u class=""></u></span></p>
<div class="">
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in" class="">
<p class="MsoNormal"><b class=""><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif" class="">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif" class="">
<a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org" target="_blank" class="">
archivesspace_users_group-<wbr class="">bounces@lyralists.lyrasis.org</a> [<a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org" target="_blank" class="">mailto:archivesspace_users_<wbr class="">group-bounces@lyralists.<wbr class="">lyrasis.org</a>]
<b class="">On Behalf Of </b>Majewski, Steven Dennis (sdm7g)<br class="">
<b class="">Sent:</b> Wednesday, February 15, 2017 2:36 PM<br class="">
<b class="">To:</b> Archivesspace Users Group<br class="">
<b class="">Subject:</b> Re: [Archivesspace_Users_Group] Enumerations Findings<u class=""></u><u class=""></u></span></p>
</div>
</div>
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">We have run into the case that some EAD attribute values are required to be NMTOKENs, thus no embedded spaces or other disallowed characters. We replaced enumerations with embedded spaces with underscores. <u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">This has only come to my attention in the last week or so, so I haven’t made a thorough investigation of which attributes or which enumerations this applies to — just fixed them as I’ve encountered that error. <u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">So it may be intentional that it is using the non translated value. <u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">( And I wouldn’t be surprised, if for simplicity, it may be over applying that rule in places where it’s not actually required. ) <u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">— Steve. <u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
<div class="">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt" class="">
<div class="">
<p class="MsoNormal">On Feb 15, 2017, at 2:09 PM, Carlos Lemus <<a href="mailto:carlos.lemus@unlv.edu" target="_blank" class="">carlos.lemus@unlv.edu</a>> wrote:<u class=""></u><u class=""></u></p>
</div>
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
<div class="">
<div class="">
<p class="MsoNormal">Hello, <u class=""></u><u class=""></u></p>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">At UNLV Special Collections, we've been working on cleaning up our enumeration values because in many cases there were duplicates caused by imports (i.e value: linear_feet vs value: Linear feet vs Linear Feet). We wanted to stick as close
as possible to ArchivesSpace standards and decided to make our enumeration values all lowercase seperated by an underscore and then merge any records with incorrect enumerations into that correct value (i.e value: linear Feet into linear_feet). We also have
some custom enumerations such as: value: oversized_box, translation: Oversized Box; digital_file; Digital File<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">After we had that set up correctly, we had some findings and was wondering if anyone has experienced the same things or had a standard we could use. <u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">1. When generating PDFs and EADs the enumeration values that were custom (such as the oversized_box) would come out as machine readable oversized_box instead of using our local en.yml value (located in the local plugin).<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal"> This was something I found in the EAD serializer (<a href="https://github.com/archivesspace/archivesspace/blob/master/backend/app/exporters/serializers/ead.rb#L490" target="_blank" class="">https://github.com/<wbr class="">archivesspace/archivesspace/<wbr class="">blob/master/backend/app/<wbr class="">exporters/serializers/ead.rb#<wbr class="">L490</a>) and
was able to create a temporary solution of generating it , but required altering the enumeration instead of referencing our file. I thought i'd point it out because anyone creating custom enumerations even with a translation in an en.yml file would not see
their change reflected in the EAD export. (I've attached an image reflecting this) Anyone experience this?<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">2. Another example of this case was in the container "type" attribute. Before something like Oversized Box would be export to EAD as is because that was it's value in the enumeration. After we changed the value correctly to oversized_box,
it would export to the EAD container "type" as is and translate to the PDF as well. With some XSLT manipulation I was able to get it to show up as oversized box (shown in attachments). I've looked through <a href="https://www.loc.gov/ead/tglib/elements/container.html" target="_blank" class="">https://www.loc.gov/<wbr class="">ead/tglib/elements/container.<wbr class="">html</a>
and cannot find an example of a two+ attribute value. <u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">Should attributes be machine readable (i.e oversized_box), human readable (Oversized Box), or does it even matter? Of course, exporting it as Oversized Box would be easiest to translate a user friendly version to the user. <u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal">Excuse me for the lengthy post, I'm trying to be thorough with my explenation, but please let me know if you've come accross something similar or have a finite solution.<u class=""></u><u class=""></u></p>
</div>
<div class="">
<p class="MsoNormal"><br clear="all" class="">
<u class=""></u><u class=""></u></p>
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<p class="MsoNormal"><span style="font-size:9.5pt" class="">Carlos Lemus<u class=""></u><u class=""></u></span></p>
</div>
<div class="">
<p class="MsoNormal"><span style="font-size:7.5pt" class="">Application Programmer, Special Collections Technical Services</span><span style="font-size:9.5pt" class=""><u class=""></u><u class=""></u></span></p>
</div>
<div class="">
<p class="MsoNormal"><span style="font-size:7.5pt" class="">University Libraries, University of Nevada, Las Vegas</span><span style="font-size:9.5pt" class=""><u class=""></u><u class=""></u></span></p>
</div>
<div class="">
<p class="MsoNormal"><span style="font-size:9.5pt" class=""><u class=""></u> <u class=""></u></span></p>
</div>
<div class="">
<p class="MsoNormal"><i class=""><span style="font-size:9.5pt;font-family:"Arial",sans-serif;color:#252525;background:white" class="">How often have I said to you that when you have eliminated the impossible, whatever remains, however improbable, must be the
truth? - Sherlock Holmes</span></i><span style="font-size:9.5pt" class=""><u class=""></u><u class=""></u></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><enumeration_ead.PNG><<wbr class="">containers_enum.PNG>__________<wbr class="">______________________________<wbr class="">_______<br class="">
Archivesspace_Users_Group mailing list<br class="">
<a href="mailto:Archivesspace_Users_Group@lyralists.lyrasis.org" target="_blank" class="">Archivesspace_Users_Group@<wbr class="">lyralists.lyrasis.org</a><br class="">
<a href="http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group" target="_blank" class="">http://lyralists.lyrasis.org/<wbr class="">mailman/listinfo/<wbr class="">archivesspace_users_group</a><u class=""></u><u class=""></u></p>
</div>
</blockquote>
</div>
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
</div>
</div>
</div>
<br class="">
______________________________<wbr class="">_________________<br class="">
Archivesspace_Users_Group mailing list<br class="">
<a href="mailto:Archivesspace_Users_Group@lyralists.lyrasis.org" class="">Archivesspace_Users_Group@<wbr class="">lyralists.lyrasis.org</a><br class="">
<a href="http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group" rel="noreferrer" target="_blank" class="">http://lyralists.lyrasis.org/<wbr class="">mailman/listinfo/<wbr class="">archivesspace_users_group</a><br class="">
<br class="">
</blockquote>
</div>
<br class="">
<br clear="all" class="">
<div class=""><br class="">
</div>
-- <br class="">
<div class="gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr" class="">
<div class="">
<div dir="ltr" class=""><br class="">
Head of Archival Processing<br class="">
<br class="">
University of Minnesota Libraries<br class="">
Archives and Special Collections<br class="">
Elmer L. Andersen Library, Suite 315<br class="">
222-21st Ave. S.<br class="">
Minneapolis MN 55455<br class="">
<br class="">
Phone: <a value="+16126248812" style="color:rgb(17,85,204)" class="">612.626.2531</a><br class="">
</div>
</div>
</div>
</div>
</div>
_______________________________________________<br class="">
Archivesspace_Users_Group mailing list<br class="">
<a href="mailto:Archivesspace_Users_Group@lyralists.lyrasis.org" class="">Archivesspace_Users_Group@lyralists.lyrasis.org</a><br class="">
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group<br class="">
</div>
</blockquote>
</div>
<br class="">
</body>
</html>