<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
h3
{mso-style-priority:9;
mso-style-link:"Heading 3 Char";
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:13.5pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
code
{mso-style-priority:99;
font-family:"Courier New";}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
span.Heading3Char
{mso-style-name:"Heading 3 Char";
mso-style-priority:9;
mso-style-link:"Heading 3";
font-family:"Calibri Light";
color:#1F3763;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Consolas",serif;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">I created an issue for this in JIRA with references to the information shared by Megan and James.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><a href="https://archivesspace.atlassian.net/browse/ANW-1229">https://archivesspace.atlassian.net/browse/ANW-1229</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Brian<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black"><archivesspace_users_group-bounces@lyralists.lyrasis.org> on behalf of James Bullen <james@hudmol.com><br>
<b>Reply-To: </b>Archivesspace Users Group <archivesspace_users_group@lyralists.lyrasis.org><br>
<b>Date: </b>Monday, March 15, 2021 at 7:49 PM<br>
<b>To: </b>Archivesspace Users Group <archivesspace_users_group@lyralists.lyrasis.org><br>
<b>Subject: </b>Re: [Archivesspace_Users_Group] Java error - Java::JavaSql::SQLException: HOUR_OF_DAY: 2 -> 3<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Many thanks Megan for the lead. I just poked around and found a few things out.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">It turns out to be interesting in at least three bad ways.<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">1. User records are updated every time they authenticate. This is unnecessary because it is just updating their authentication source, which usually never changes. So this line could be made conditional on the source being different:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><a href="https://github.com/archivesspace/archivesspace/blob/master/backend/app/model/authentication_manager.rb#L40">https://github.com/archivesspace/archivesspace/blob/master/backend/app/model/authentication_manager.rb#L40</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">2. The search_indexer user is being re-authenticated a lot. This shouldn’t happen because it should be a non-expiring session. I haven’t looked into this one, but I did check the lock_version on the search_inxdexer user on a large-ish
AS db I have lying around, that has been in production for less than a year and it is over 100,000. That’s nuts. Actually I just took a quick look and saw this:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><a href="https://github.com/archivesspace/archivesspace/blob/master/indexer/app/lib/periodic_indexer.rb#L285">https://github.com/archivesspace/archivesspace/blob/master/indexer/app/lib/periodic_indexer.rb#L285</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">The session is reset if there is an error during an indexing round. So that could be the culprit.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">3. There is clearly a bug somewhere in the stack, probably in the Sequel layer, because it shouldn’t crash the system just because you have a whacky datetime somewhere. This bug may well have been fixed in a more recent version of that
component.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Dealing with 1 will likely fix the problem, but 2 and 3 should also be investigated.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Cheers,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">James<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On Mar 16, 2021, at 1:24 AM, Tom Hanstra <<a href="mailto:hanstra@nd.edu">hanstra@nd.edu</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">Thanks, Megan. That update did fix things.<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">So, is this not something that can be addressed at the software fix level? Do other sites simply stop indexing or turn something else off when we hit DST in March?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Tom<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Mon, Mar 15, 2021 at 10:12 AM Schanz, Megan <<a href="mailto:schanzme@msu.edu">schanzme@msu.edu</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">I run across this every March for daylight savings. This is what I have in my notes to do each time if the indexer is running. Luckily it seems like I didn't have this scenario this year.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
<div>
<h3>java.lang.IllegalArgumentException: HOUR_OF_DAY: 2 -> 3:Java::JavaLang::IllegalArgumentException:<o:p></o:p></h3>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt">This error will appear in the logs when trying to start ArchivesSpace after daylight savings time. Reference:
<a href="http://lyralists.lyrasis.org/mailman/htdig/archivesspace_users_group/2019-March/006652.html" target="_blank">
http://lyralists.lyrasis.org/mailman/htdig/archivesspace_users_group/2019-March/006652.html</a><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt">Basically, the indexer user is being updated in the database with a time that does not exist due to daylight savings time.<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt">To verify:<o:p></o:p></span></p>
<pre><code>SELECT * FROM user WHERE (user_mtime >= '2021-03-14 02:00:00' and user_mtime <= '2021-03-14 03:00:00') OR (system_mtime >= '2021-03-04 02:00:00' and system_mtime <= '2021-03-14 03:00:00');</code><o:p></o:p></pre>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt">The record that should come back is the
</span><code><span style="font-size:10.0pt">search_indexer</span></code><span style="font-size:12.0pt"> user.<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt">To resolve:<o:p></o:p></span></p>
<pre><code>UPDATE user set user_mtime = NOW(), system_mtime=NOW() where username='search_indexer';</code><o:p></o:p></pre>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="0" width="100%" align="center">
</div>
<div id="gmail-m_-3548360350817852693divRplyFwdMsg">
<p class="MsoNormal"><b>From:</b> <a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org" target="_blank">
archivesspace_users_group-bounces@lyralists.lyrasis.org</a> <<a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org" target="_blank">archivesspace_users_group-bounces@lyralists.lyrasis.org</a>> on behalf of Tom Hanstra <<a href="mailto:hanstra@nd.edu" target="_blank">hanstra@nd.edu</a>><br>
<b>Sent:</b> Monday, March 15, 2021 9:10 AM<br>
<b>To:</b> Archivesspace Users Group <<a href="mailto:archivesspace_users_group@lyralists.lyrasis.org" target="_blank">archivesspace_users_group@lyralists.lyrasis.org</a>><br>
<b>Subject:</b> Re: [Archivesspace_Users_Group] Java error - Java::JavaSql::SQLException: HOUR_OF_DAY: 2 -> 3
<o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Any suggestion on what ArchivesSpace might have changed? I had the server running but indexing was complete. What might it have been changing and in what database table would I look for that change?<br>
<br>
Alternately, since this is still test data, should I just overlay a backup copy of the database from earlier than the EDT cutover time? I don't know what affect that might have on other portions of ArchivesSpace.
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Finally, what is the overall fix for this issue. If others have seen it, what can be done to make sure to avoid it in the future?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Tom<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Mon, Mar 15, 2021 at 9:03 AM Blake Carver <<a href="mailto:blake.carver@lyrasis.org" target="_blank">blake.carver@lyrasis.org</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><a href="https://urldefense.com/v3/__https:/gist.github.com/Blake-/d493da28be5554a49a3a3835bbd98f05__;!!HXCxUKc!lsrL41tXx-WGwpdlFWm60sfuIbz0DRfuyn4RvaCAka7R1eaKVNkWL7PbhWHEm8g$" target="_blank">https://gist.github.com/Blake-/d493da28be5554a49a3a3835bbd98f05</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">You'll want to find the date more like '2021-03-14 02:00%' or would it be <span style="color:black;background:white">03-13? Whatever the date was this year.</span><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black;background:white">Find any date with a time between 2-3am and just change it to any real hour.</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black;background:white">ArchivesSpace did "something" (probably restarted?) at a bad time on Sunday morning and wrote a time that never happened.</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="0" width="100%" align="center">
</div>
<div id="gmail-m_-3548360350817852693x_gmail-m_7814653410892901514divRplyFwdMsg">
<p class="MsoNormal"><b>From:</b> <a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org" target="_blank">
archivesspace_users_group-bounces@lyralists.lyrasis.org</a> <<a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org" target="_blank">archivesspace_users_group-bounces@lyralists.lyrasis.org</a>> on behalf of James Bullen <<a href="mailto:james@hudmol.com" target="_blank">james@hudmol.com</a>><br>
<b>Sent:</b> Sunday, March 14, 2021 11:04 PM<br>
<b>To:</b> Archivesspace Users Group <<a href="mailto:archivesspace_users_group@lyralists.lyrasis.org" target="_blank">archivesspace_users_group@lyralists.lyrasis.org</a>><br>
<b>Subject:</b> Re: [Archivesspace_Users_Group] Java error - Java::JavaSql::SQLException: HOUR_OF_DAY: 2 -> 3
<o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">Oh gee, I’ve seen that horror. And, yes, it is caused by records having times in that non-existent hour. From memory the only fix I could find was to delete the offending rows, or update their times.
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Cheers,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">James<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On Mar 15, 2021, at 1:59 PM, Tom Hanstra <<a href="mailto:hanstra@nd.edu" target="_blank">hanstra@nd.edu</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">I don't seem to be able to win with ArchivesSpace. <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">After having indexing again finally give up because of another Java heap error, I restarted archivesspace again with some different java settings. I believe it actually got through the indexing, at least of the first repository.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">But tonight, looking at the logs, there is another error that seems to have something to do with, perhaps, daylight savings time? I'm seeing this Java error in the logs, even after a restart:<br>
<br>
--------<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">INFO: An exception happened during JRuby-Rack startup<br>
Java::JavaSql::SQLException: HOUR_OF_DAY: 2 -> 3<br>
--- System<br>
jruby 9.2.12.0 (2.5.7) 2020-07-01 db01a49ba6 OpenJDK 64-Bit Server VM 25.272-b10 on 1.8.0_272-b10 +jit [linux-x86_64]<br>
Time: 2021-03-14 22:46:48 -0400<br>
Server: jetty/8.1.5.v20120716<br>
jruby.home: uri:classloader://META-INF/jruby.home<br>
<br>
--- Context Init Parameters:<br>
jruby.max.runtimes = 1<br>
jruby.min.runtimes = 1<br>
public.root = /<br>
rack.env = production<br>
<br>
--- Backtrace<br>
Sequel::DatabaseError: Java::JavaSql::SQLException: HOUR_OF_DAY: 2 -> 3<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">--------<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Did something get corrupted in the database? What might be happening to result in this error?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">How do I fix it?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Tom<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><b><span style="font-size:9.5pt;font-family:"Arial",sans-serif;color:#888888">Tom Hanstra</span></b><o:p></o:p></p>
</div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><i><span style="font-size:9.5pt;font-family:"Arial",sans-serif;color:#888888">Sr. Systems Administrator</span></i><span style="font-size:9.5pt;color:#888888"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.5pt;color:#888888"><a href="mailto:hanstra@nd.edu" target="_blank"><span style="font-family:"Arial",sans-serif;color:#1155CC">hanstra@nd.edu</span></a><o:p></o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.5pt;color:#888888"><o:p> </o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.5pt;color:#888888;border:solid windowtext 1.0pt;padding:0in"><img border="0" width="32" height="32" style="width:.3333in;height:.3333in" id="_x0000_i1027" src="cid:~WRD0000.jpg" alt="Image removed by sender."></span><span style="font-size:9.5pt;color:#888888"><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal">_______________________________________________<br>
Archivesspace_Users_Group mailing list<br>
<a href="mailto:Archivesspace_Users_Group@lyralists.lyrasis.org" target="_blank">Archivesspace_Users_Group@lyralists.lyrasis.org</a><br>
<a href="https://urldefense.com/v3/__http:/lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group__;!!HXCxUKc!lsrL41tXx-WGwpdlFWm60sfuIbz0DRfuyn4RvaCAka7R1eaKVNkWL7PbLTKdGT0$" target="_blank">http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group</a><o:p></o:p></p>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
<p class="MsoNormal">_______________________________________________<br>
Archivesspace_Users_Group mailing list<br>
<a href="mailto:Archivesspace_Users_Group@lyralists.lyrasis.org" target="_blank">Archivesspace_Users_Group@lyralists.lyrasis.org</a><br>
<a href="https://urldefense.com/v3/__http:/lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group__;!!HXCxUKc!lsrL41tXx-WGwpdlFWm60sfuIbz0DRfuyn4RvaCAka7R1eaKVNkWL7PbLTKdGT0$" target="_blank">http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group</a><o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><b><span style="font-size:9.5pt;font-family:"Arial",sans-serif;color:#888888">Tom Hanstra</span></b><o:p></o:p></p>
</div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><i><span style="font-size:9.5pt;font-family:"Arial",sans-serif;color:#888888">Sr. Systems Administrator</span></i><span style="font-size:9.5pt;color:#888888"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.5pt;color:#888888"><a href="mailto:hanstra@nd.edu" target="_blank"><span style="font-family:"Arial",sans-serif;color:#1155CC">hanstra@nd.edu</span></a><o:p></o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.5pt;color:#888888"><o:p> </o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.5pt;color:#888888;border:solid windowtext 1.0pt;padding:0in"><img border="0" width="32" height="32" style="width:.3333in;height:.3333in" id="_x0000_i1026" src="cid:~WRD0000.jpg" alt="Image removed by sender."></span><span style="font-size:9.5pt;color:#888888"><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal">_______________________________________________<br>
Archivesspace_Users_Group mailing list<br>
<a href="mailto:Archivesspace_Users_Group@lyralists.lyrasis.org" target="_blank">Archivesspace_Users_Group@lyralists.lyrasis.org</a><br>
<a href="http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group" target="_blank">http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group</a><o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><b><span style="font-size:9.5pt;font-family:"Arial",sans-serif;color:#888888">Tom Hanstra</span></b><o:p></o:p></p>
</div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><i><span style="font-size:9.5pt;font-family:"Arial",sans-serif;color:#888888">Sr. Systems Administrator</span></i><span style="font-size:9.5pt;color:#888888"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.5pt;color:#888888"><a href="mailto:hanstra@nd.edu" target="_blank"><span style="font-family:"Arial",sans-serif;color:#1155CC">hanstra@nd.edu</span></a><o:p></o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.5pt;color:#888888"><o:p> </o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.5pt;color:#888888;border:solid windowtext 1.0pt;padding:0in"><img border="0" width="32" height="32" style="width:.3333in;height:.3333in" id="_x0000_i1025" src="cid:~WRD0000.jpg" alt="Image removed by sender."></span><span style="font-size:9.5pt;color:#888888"><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal">_______________________________________________<br>
Archivesspace_Users_Group mailing list<br>
<a href="mailto:Archivesspace_Users_Group@lyralists.lyrasis.org">Archivesspace_Users_Group@lyralists.lyrasis.org</a><br>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group<o:p></o:p></p>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
</div>
</body>
</html>