[Archivesspace_Users_Group] More ArchivesSpace performance analysis -- this time, the database

Thu Aug 27 12:08:14 EDT 2015

While this should not absolve the project from addressing the fundamental issues that report identifies (which are quite concerning, especially the extraordinary number of SQL queries being generated), I second the motion for caching the EAD and html output, with the cache being updated whenever an underlying component of the cached output is updated.

It can be a bit tricky to implement, but many enterprise style web apps have this kind of feature, and there may even be an external library that can be incorporated to do it.  (As a side note, we incorporated caching to archon at a certain point in time, which dramatically cut page load time for long finding aids and EADs where the content had not been modified.  We implemented it so the cache was updated whenever the public page for a modified resource was requested--which was not ideal, but at least contained the problem the initial page load of a modified page.)

Another suggestion I have is to add some performance data to the bottom of each page in the UI (which can be enabled with a configuration setting) along the lines of (Page generated in x seconds using x queries), so that it is possible to assess performance issues on a running basis under multiple environments, which can function as an early warning system in testing new features or modifications—or as a quick way for the team to congratulate themselves when solving a persistent problem.

Chris Prom
University of Illinois

On Aug 27, 2015, at 10:36 AM, Jason Loeffler <j at minorscience.com<mailto:j at minorscience.com>> wrote:

Kudos to Yale for engaging Percona. For small and very small organizations with limited or no in-house IT support, this kind of contribution is essential. Having experienced substantial latency on resource trees with only 3,000 records, I

Chris, regarding benchmarking tools, I've used tuning-primer<https://urldefense.proofpoint.com/v2/url?u=https-3A__launchpad.net_mysql-2Dtuning-2Dprimer&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=0GZvusRIq5YEacXko6vQC3PGETVe5AhepGb4XsBwVvU&e=> and sysbench<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_akopytov_sysbench&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=uWUvOQVUkzlXblxwbs2YE-K43_t2r0T72oD4hk-abjw&e=> alongside MySQLTuner for years with good success. Additionally I've deployed ASpace (and other projects) against MariaDB<https://urldefense.proofpoint.com/v2/url?u=https-3A__mariadb.org_&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=NDSconlpArQ7ucTO1537AUvUHJVcqxw0BChhHcCtPng&e=> and Percona's own InnoDB drop-in replacement with the Xtra DB storage engine<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.percona.com_software_mysql-2Ddatabase_percona-2Dserver_xtradb&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=qcTLiPE6RJfYMExBCgXUa9yvYPJLSboGPgzTWbYGzOc&e=>. No issues so far.

For organizations with dedicated IT support, I highly recommend the Percona toolkit <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.percona.com_software_mysql-2Dtools_percona-2Dtoolkit&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=dc8ufJuyqLLtUNbbutaaxjO0vfizSzAz1kOJ8v3e2s0&e=> as a standard part of any deployment as a means to collect and analyze information about database-related problems.

I'd be happy to describe how I've used these database distributions and tools in the ASpace context and outline the potential benefits.

That said, having read the Percona report, I'm not entirely clear that database tuning yields much of a positive impact. The report bears this out. The section on "Alternatives" in the JRuby GitHub page seems most promising. I'm not a Java dev, but this description seems very similar to using in-memory or file-based cache stores, like Memcache or Redis, in unthreaded applications (e.g. PHP).

Jason Loeffler
Technical Consultant | American Academy in Rome
Principal | Minor Science | Application Development & Metadata Strategy
Brooklyn, New York

On Wed, Aug 26, 2015 at 10:47 AM, Chris Fitzpatrick <Chris.Fitzpatrick at lyrasis.org<mailto:Chris.Fitzpatrick at lyrasis.org>> wrote:

Hi Maureen,

This is excellent. Percona is one of the premier MySQL experts, so this is great feedback.

Also, I have a page describing some MySQL and application tuning suggestions.<https://urldefense.proofpoint.com/v2/url?u=http-3A__archivesspace.github.io_archivesspace_user_tuning-2Darchivesspace_&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=w1fr9R-bpxicmWuR4qurQGSHMMP_pxkOZRfMMYKQ4OQ&e=>

One thing that I suggest is to run a profiler on the MySQL DB server, which can give you some ideas on where to look for problem areas. I've been using MySQL Tuner for years ( http://mysqltuner.com/)<https://urldefense.proofpoint.com/v2/url?u=http-3A__mysqltuner.com_&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=ilPsD8s5bxhnho-xmV_bsfO2IN6FctFU_06Cm4zgxTA&e=> but maybe there are some other things people are using? Any suggestions would help, and I can add them to the page ( or better yet send me a PR

Also curious about various MySQL distributions ( Oracle, MariaDB, Percona, etc ) people are using?  Are people using dedicated db servers, or clusters, or just having the db server on the same box as the application?

Lastly, it would be great if we could start work on supporting other DBs. I think there's a strong interests ( especially from smaller orgs ) for MS Server, but are others wanting Oracle or postgres?

best, chris.

Chris Fitzpatrick | Developer, ArchivesSpace
Skype: chrisfitzpat  | Phone: 918.236.6048<tel:918.236.6048>
http://archivesspace.org/<https://urldefense.proofpoint.com/v2/url?u=http-3A__archivesspace.org_&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=L26CpKlTR6X2dmSdnnbzHulcky_6J46BAhQAx2lKhEY&e=>

________________________________
From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> <archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>> on behalf of Callahan, Maureen <maureen.callahan at yale.edu<mailto:maureen.callahan at yale.edu>>
Sent: Wednesday, August 26, 2015 3:36 PM
To: 'archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>'
Subject: [Archivesspace_Users_Group] More ArchivesSpace performance analysis -- this time, the database

Hi everyone,

As I’ve reported before, we at Yale are in the middle of an aggressive period of analysis to diagnose some of the slow performance we’ve been seeing with ArchivesSpace. As part of this initiative, we’ve contracted with Percona, a firm that specializes in MySQL database analysis. Their report (attached), includes a number of action items that we believe the ArchivesSpace community may find helpful. We hope that this will result in further improvements to the application.

Best wishes,
Maureen

Maureen Callahan
Archivist, Metadata Specialist
Manuscripts & Archives
Yale University Library
maureen.callahan at yale.edu<mailto:maureen.callahan at yale.edu>
203.432.3627<tel:203.432.3627>

Webpage: web.library.yale.edu/mssa<https://urldefense.proofpoint.com/v2/url?u=http-3A__web.library.yale.edu_mssa&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=il5kqPWzNKhOZRu6dbtSlEGZh3ivBOroSclNvHD2kbI&e=>
Collections: drs.library.yale.edu<https://urldefense.proofpoint.com/v2/url?u=http-3A__drs.library.yale.edu&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=A5eKQor7yHrRYelGeaAuYnq_9-BEsZBy6_fzNFCHV3I&e=>

_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group<https://urldefense.proofpoint.com/v2/url?u=http-3A__lyralists.lyrasis.org_mailman_listinfo_archivesspace-5Fusers-5Fgroup&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=jGJMaTc-8I-z6_tkoj_Qyi4UF1KtYBfcz4s2Ly33jmw&m=sYxMTtfZU9rIKQ_pQ1HLrHeYPTcyoGJXudcMXK1pKUM&s=2HqTRngQzTLf7b73TbhCb4ol3DrdTXvz_H1cfi1XlRg&e=>

_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20150827/84475554/attachment.html>