<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p><br>
</p>
<p><br>
</p>
<p>Hi, </p>
<p><br>
</p>
<p>@Jason: thanks! Any thoughts would be great about MySQL performance...we could add it to the document ( you can add it in git or just send it to the list and I'll add it..).<br>
</p>
<p><br>
</p>
<p>@Chris : Yeah, you have to be careful there... If you cache the EAD each time the resource gets updated, that means you'll have to export and EAD every time someone does a save to any record that is involved. Tweak a note? EAD gets exported. Add a few components?
EAD gets exported a few times. I think the cache feature would have to be user driven, that is someone have to click an "Update EAD" button. </p>
<p><br>
</p>
<p>The other big problem is the pages people usually complain about loading are edit pages, which cannot be cached. We probably could just have the View pages pull from Solr ( I think that was a suggestion in the HM report? ) . If you want caching, you can
already implement that right now with Squid or Varnish. </p>
<p><br>
</p>
<p>What I think would be great if AT and Archon people could time how many queries exporting an EAD of similar size takes. My gut is telling me that the number of queries has a lot to do with the nature of EAD, but I could be wrong. </p>
<p><br>
</p>
<p>Also, with the page load times, I'd really suggest using something like Google analytics or New Relic, which will give you a much better sense of what's happening.</p>
<p><br>
</p>
<p>b,chris. <br>
</p>
<p><br>
</p>
<div id="Signature">
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0">
<div class="BodyFragment"><font size="2">
<div class="PlainText">Chris Fitzpatrick | <font size="2">Developer, ArchivesSpace</font><br>
Skype: chrisfitzpat | Phone: 918.236.6048<br>
http://archivesspace.org/<br>
</div>
</font></div>
</div>
</div>
<br>
<br>
<div style="color: rgb(0, 0, 0);">
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> archivesspace_users_group-bounces@lyralists.lyrasis.org <archivesspace_users_group-bounces@lyralists.lyrasis.org> on behalf of Prom, Christopher
John <prom@illinois.edu><br>
<b>Sent:</b> Thursday, August 27, 2015 6:08 PM<br>
<b>To:</b> Archivesspace Users Group<br>
<b>Subject:</b> Re: [Archivesspace_Users_Group] More ArchivesSpace performance analysis -- this time, the database</font>
<div> </div>
</div>
<div>While this should not absolve the project from addressing the fundamental issues that report identifies (which are quite concerning, especially the extraordinary number of SQL queries being generated), I second the motion for caching the EAD and html output,
with the cache being updated whenever an underlying component of the cached output is updated.
<div class=""><br class="">
</div>
<div class="">It can be a bit tricky to implement, but many enterprise style web apps have this kind of feature, and there may even be an external library that can be incorporated to do it. (As a side note, we incorporated caching to archon at a certain point
in time, which dramatically cut page load time for long finding aids and EADs where the content had not been modified. We implemented it so the cache was updated whenever the public page for a modified resource was requested--which was not ideal, but at least
contained the problem the initial page load of a modified page.)</div>
<div class=""><br class="">
</div>
<div class="">Another suggestion I have is to add some performance data to the bottom of each page in the UI (which can be enabled with a configuration setting) along the lines of (Page generated in x seconds using x queries), so that it is possible to assess
performance issues on a running basis under multiple environments, which can function as an early warning system in testing new features or modifications—or as a quick way for the team to congratulate themselves when solving a persistent problem.<br class="">
<div class=""><br class="">
</div>
<div class="">Chris Prom</div>
<div class="">University of Illinois</div>
<div class="">
<div class=""><br class="">
</div>
<div class=""><br class="">
<div class=""><br class="">
</div>
<div class="">
<div>
<blockquote type="cite" class="">
<div class="">On Aug 27, 2015, at 10:36 AM, Jason Loeffler <<a title="Cmd+Click or tap to follow the link" href="mailto:j@minorscience.com" class="">j@minorscience.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">Kudos to Yale for engaging Percona. For small and very small organizations with limited or no in-house IT support, this kind of contribution is essential. Having experienced substantial latency on resource trees with only 3,000 records,
I
<div class=""><br class="">
</div>
<div class="">Chris, regarding benchmarking tools, I've used <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__launchpad.net_mysql-2Dtuning-2Dprimer&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=0GZvusRIq5YEacXko6vQC3PGETVe5AhepGb4XsBwVvU&e=" class="">
tuning-primer</a> and <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_akopytov_sysbench&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=uWUvOQVUkzlXblxwbs2YE-K43_t2r0T72oD4hk-abjw&e=" class="">
sysbench</a> alongside MySQLTuner for years with good success. Additionally I've deployed ASpace (and other projects) against
<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__mariadb.org_&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=NDSconlpArQ7ucTO1537AUvUHJVcqxw0BChhHcCtPng&e=" class="">
MariaDB</a> and Percona's own InnoDB drop-in replacement with the <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__www.percona.com_software_mysql-2Ddatabase_percona-2Dserver_xtradb&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=qcTLiPE6RJfYMExBCgXUa9yvYPJLSboGPgzTWbYGzOc&e=" class="">
Xtra DB storage engine</a>. No issues so far. </div>
<div class=""><br class="">
</div>
<div class="">For organizations with dedicated IT support, I highly recommend the
<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__www.percona.com_software_mysql-2Dtools_percona-2Dtoolkit&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=dc8ufJuyqLLtUNbbutaaxjO0vfizSzAz1kOJ8v3e2s0&e=" class="">
Percona toolkit </a>as a standard part of any deployment as a means to collect and analyze information about database-related problems. </div>
<div class=""><br class="">
</div>
<div class="">I'd be happy to describe how I've used these database distributions and tools in the ASpace context and outline the potential benefits. </div>
<div class=""><br class="">
</div>
<div class="">That said, having read the Percona report, I'm not entirely clear that database tuning yields much of a positive impact. The report bears this out. The section on "Alternatives" in the JRuby GitHub page seems most promising. I'm not a Java dev,
but this description seems very similar to using in-memory or file-based cache stores, like Memcache or Redis, in unthreaded applications (e.g. PHP). </div>
<div class="gmail_extra"><br class="" clear="all">
<div class="">
<div class="gmail_signature">
<div dir="ltr" class="">
<div class="">
<div class=""><font class="" style="background-color:rgb(255,255,255)" face="garamond, serif" color="#666666" size="2">Jason Loeffler</font></div>
<div class=""><font class="" face="garamond, serif" color="#666666">Technical Consultant | American Academy in Rome</font></div>
<div class=""><font class="" face="garamond, serif" color="#666666">Principal | Minor Science | Application Development & Metadata Strategy</font></div>
<div class=""><font class="" style="background-color:rgb(255,255,255)" face="garamond, serif" color="#666666" size="2">Brooklyn, New York</font></div>
</div>
<div class=""><br class="">
</div>
</div>
</div>
</div>
<br class="">
<div class="gmail_quote">On Wed, Aug 26, 2015 at 10:47 AM, Chris Fitzpatrick <span dir="ltr" class="">
<<a href="mailto:Chris.Fitzpatrick@lyrasis.org" target="_blank" class="">Chris.Fitzpatrick@lyrasis.org</a>></span> wrote:<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<div dir="ltr" class="">
<div class="" style="font-size:12pt; background-color:rgb(255,255,255); font-family:Calibri,Arial,Helvetica,sans-serif">
<p class=""><br class="">
</p>
<p class=""><br class="">
</p>
<p class="">Hi Maureen,</p>
<p class=""><br class="">
</p>
<p class="">This is excellent. Percona is one of the premier MySQL experts, so this is great feedback.</p>
<p class=""><br class="">
</p>
<p class="">Also, I have a page describing some <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__archivesspace.github.io_archivesspace_user_tuning-2Darchivesspace_&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=w1fr9R-bpxicmWuR4qurQGSHMMP_pxkOZRfMMYKQ4OQ&e=" target="_blank" class="">
MySQL and application tuning suggestions.</a> <br class="">
</p>
<p class="">One thing that I suggest is to run a profiler on the MySQL DB server, which can give you some ideas on where to look for problem areas. I've been using MySQL Tuner for years (
<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__mysqltuner.com_&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=ilPsD8s5bxhnho-xmV_bsfO2IN6FctFU_06Cm4zgxTA&e=" target="_blank" class="">
http://mysqltuner.com/)</a> but maybe there are some other things people are using? Any suggestions would help, and I can add them to the page ( or better yet send me a PR
<br class="">
</p>
<p class="">Also curious about various MySQL distributions ( Oracle, MariaDB, Percona, etc ) people are using? Are people using dedicated db servers, or clusters, or just having the db server on the same box as the application?
</p>
<p class=""><br class="">
</p>
<p class="">Lastly, it would be great if we could start work on supporting other DBs. I think there's a strong interests ( especially from smaller orgs ) for MS Server, but are others wanting Oracle or postgres?
<br class="">
</p>
<p class=""><br class="">
</p>
<p class="">best, chris. <br class="">
</p>
<p class=""><br class="">
</p>
<p class=""><br class="">
</p>
<p class=""><br class="">
</p>
<div class="">
<div name="divtagdefaultwrapper" class="">
<div class=""><font class="" size="2">
<div class="">Chris Fitzpatrick | <font class="" size="2">Developer, ArchivesSpace</font><br class="">
Skype: chrisfitzpat | Phone: <a href="tel:918.236.6048" value="+19182366048" target="_blank" class="">
918.236.6048</a><br class="">
<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__archivesspace.org_&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=L26CpKlTR6X2dmSdnnbzHulcky_6J46BAhQAx2lKhEY&e=" target="_blank" class="">http://archivesspace.org/</a><br class="">
</div>
</font></div>
</div>
</div>
<br class="">
<br class="">
<div class="" style="">
<hr class="" style="display:inline-block; width:98%">
<div dir="ltr" class=""><font class="" style="font-size:11pt" face="Calibri, sans-serif"><b class="">From:</b>
<a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org" target="_blank" class="">
archivesspace_users_group-bounces@lyralists.lyrasis.org</a> <<a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org" target="_blank" class="">archivesspace_users_group-bounces@lyralists.lyrasis.org</a>> on behalf of Callahan, Maureen <<a href="mailto:maureen.callahan@yale.edu" target="_blank" class="">maureen.callahan@yale.edu</a>><br class="">
<b class="">Sent:</b> Wednesday, August 26, 2015 3:36 PM<br class="">
<b class="">To:</b> '<a href="mailto:archivesspace_users_group@lyralists.lyrasis.org" target="_blank" class="">archivesspace_users_group@lyralists.lyrasis.org</a>'<br class="">
<b class="">Subject:</b> [Archivesspace_Users_Group] More ArchivesSpace performance analysis -- this time, the database</font>
<div class=""> </div>
</div>
<div class="">
<div class="">
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
Hi everyone,</div>
<p class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:"Calibri","sans-serif"">
</p>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
As I’ve reported before, we at Yale are in the middle of an aggressive period of analysis to diagnose some of the slow performance we’ve been seeing with ArchivesSpace. As part of this initiative, we’ve contracted with Percona, a firm that specializes in MySQL
database analysis. Their report (attached), includes a number of action items that we believe the ArchivesSpace community may find helpful. We hope that this will result in further improvements to the application.</div>
<p class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:"Calibri","sans-serif"">
</p>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
Best wishes,</div>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
Maureen</div>
<p class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:"Calibri","sans-serif"">
</p>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
Maureen Callahan</div>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
<span class="" style="color:#404040">Archivist, Metadata Specialist</span></div>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
<span class="" style="color:#404040">Manuscripts & Archives</span></div>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
<span class="" style="color:#404040">Yale University Library</span></div>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
<span class="" style="color:#8496b0"><a href="mailto:maureen.callahan@yale.edu" target="_blank" class="" style="color:rgb(5,99,193); text-decoration:underline"><span class="" style="color:#8496b0">maureen.callahan@yale.edu</span></a></span></div>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
<span class="" style="color:#404040"><a href="tel:203.432.3627" value="+12034323627" target="_blank" class="">203.432.3627</a></span></div>
<p class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:"Calibri","sans-serif"">
<span class="" style="color:#7f7f7f"> </span></p>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
<span class="" style="color:#404040">Webpage:</span><span class="" style="color:#7f7f7f">
</span><span class="" style="color:#8496b0"><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__web.library.yale.edu_mssa&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=il5kqPWzNKhOZRu6dbtSlEGZh3ivBOroSclNvHD2kbI&e=" target="_blank" class="" style="color:rgb(5,99,193); text-decoration:underline"><span class="" style="color:#8496b0">web.library.yale.edu/mssa</span></a></span><span class="" style="color:#7f7f7f"></span></div>
<div class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:Calibri,sans-serif">
<span class="" style="color:#404040">Collections:</span><span class="" style="color:#7f7f7f">
</span><span class="" style="color:#8496b0"><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__drs.library.yale.edu&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=A5eKQor7yHrRYelGeaAuYnq_9-BEsZBy6_fzNFCHV3I&e=" target="_blank" class="" style="color:rgb(5,99,193); text-decoration:underline"><span class="" style="color:#8496b0">drs.library.yale.edu</span></a></span><span class="" style="color:#7f7f7f"></span></div>
<p class="" style="margin:0in 0in 0.0001pt; font-size:11pt; font-family:"Calibri","sans-serif"">
</p>
</div>
</div>
</div>
</div>
</div>
<br class="">
_______________________________________________<br class="">
Archivesspace_Users_Group mailing list<br class="">
<a href="mailto:Archivesspace_Users_Group@lyralists.lyrasis.org" class="">Archivesspace_Users_Group@lyralists.lyrasis.org</a><br class="">
<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__lyralists.lyrasis.org_mailman_listinfo_archivesspace-5Fusers-5Fgroup&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=2HqTRngQzTLf7b73TbhCb4ol3DrdTXvz_H1cfi1XlRg&e=" rel="noreferrer" target="_blank" class="">http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group</a><br class="">
<br class="">
</blockquote>
</div>
<br class="">
</div>
</div>
_______________________________________________<br class="">
Archivesspace_Users_Group mailing list<br class="">
<a href="mailto:Archivesspace_Users_Group@lyralists.lyrasis.org" class="">Archivesspace_Users_Group@lyralists.lyrasis.org</a><br class="">
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group<br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>