[Archivesspace_Users_Group] Encoding errors while importing from Archivist Toolkit

[Laney McGlohon]

Hey Ryan,

I am not sure why you are receiving this error if the data is UTF-8.  I know there is a check in the migration tool to make sure that failures due to non-UTF-8 characters do not cause the migration tool to fail and stop, but instead the tool logs the error and moves on to the next record.

Would it be possible for you to provide me with a copy of your AT database or could you export one (or several) of the resources that is failing from your AT instance and send it to me? Having an example that is failing incorrectly will definitely help me determine the appropriate fix for this.

Thanks,
Laney

Laney McGlohon
ArchivesSpace Tech Lead
laney.mcglohon at lyrasis.org<mailto:laney.mcglohon at lyrasis.org>
laneymcglohon Skype


From: <archivesspace_users_group-bounces at lyralists.lyrasis.org> on behalf of Ryan Rotter <rrotter at umich.edu>
Reply-To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org>
Date: Tuesday, February 6, 2018 at 5:13 PM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org>
Subject: [Archivesspace_Users_Group] Encoding errors while importing from Archivist Toolkit

While running the ArchivesSpace AT Migration plugin I have users repeatedly seeing errors like this:

(Encoding::UndefinedConversionError) "\xB4" from ASCII-8BIT to UTF-8
You're seeing this error because you use JRuby::Rack::ErrorApp::ShowStatus.

This error is what ruby should throw while attempting to reencode invalid ASCII to UTF-8 but, I've checked the data and it's not ASCII at all, it's valid UTF-8. The strings it hangs up on are all multibyte characters. Why would ASpace think the data is ASCII? Is there a way to force the plugin to properly represent the encoding to ASpace to prevent this error?

Ryan Rotter
Library IT
University of Michigan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20180208/05856d0a/attachment.html>