[Archivesspace_Users_Group] performance issues

Tue Sep 16 16:52:09 EDT 2014

Hey Douglas,

So, off the bat I'd suggest upgrading to Java 7, which is recommended version for Aspace. I'd also suggest using Oracle java instead of OpenJDK, as I think most people see much better performance. 

Also, since you have 4G of RAM, you can bump that heap up from 1024m, probably to 2G or even 3G. Without changing the heap, the application will only use that 1G allocated to the java heap. You can do this by setting the ASPACE_JAVA_XMX system variable to '-Xmx2048m'.  Bumping the PermGen to 512m might also help  ( ASPACE_JAVA_MAXPERMSIZE ). 

I'd try those fixes first, and I'm willing to bet you'll see some improvement..

b,chris. 

Chris Fitzpatrick | Developer, ArchivesSpace
Skype: chrisfitzpat  | Phone: 918.236.6048
http://archivesspace.org/

________________________________________
From: archivesspace_users_group-bounces at lyralists.lyrasis.org <archivesspace_users_group-bounces at lyralists.lyrasis.org> on behalf of Douglas James Simmons <dsimmons at siu.edu>
Sent: Tuesday, September 16, 2014 8:07 PM
To: Archivesspace Users Group
Subject: Re: [Archivesspace_Users_Group] performance issues

This is a default install of aspace running Jetty.
The last test migration was a few weeks ago, so I would have thought all indexing was finished long ago.
We have a local mysql: mysql  Ver 14.14 Distrib 5.5.38, for debian-linux-gnu (x86_64) using readline 6.2
We have 27G disk available on the 4-core/4GB Ubuntu KVM VM.

More details below. Thanks very much for looking into this!
Doug Simmon
SIUC Morris Library

JVM

    Runtime
        OpenJDK 64-Bit Server VM (23.25-b01)

    Processors
        4

    Args
        -Dfile.encoding=UTF-8
        -XX:MaxPermSize=256m
        -Xss2m
        -Xmx1024m
        -Darchivesspace-daemon=yes

Solr Java Properties

    ASPACE_LAUNCHER_BASE
        /var/www/archivesspace

    archivesspace-daemon
        yes

    aspace.config.data_directory
        /var/www/archivesspace/data

    aspace.config.frontend_cookie_secret
        417672d64050640f996c29db809ae47f

    aspace.config.public_cookie_secret
        aee9b14a299462d15767861bed03c653

    aspace.config.public_user_secret
        075837e5ea6c7133cf6bffd383d61e6c

    aspace.config.search_user_secret
        e6d00d9adb3df6beb9dcbd4f6459e5a5

    aspace.config.staff_user_secret
        3bc6d8720f849a7a921dae705bd3aeaa

    aspace.launcher.base
        /var/www/archivesspace

    file.encoding
        UTF-8

    file.encoding.pkg
        sun.io

    file.separator
        /

    java.awt.graphicsenv
        sun.awt.X11GraphicsEnvironment

    java.awt.headless
        true

    java.awt.printerjob
        sun.print.PSPrinterJob

    java.class.path
        lib/mysql-connector-java-5.1.28.jar
        lib/common.jar
        lib/jsoup-1.7.2.jar
        launcher/lib/servlet-api-3.0.jar
        launcher/lib/jetty-all-8.1.5.v20120716.jar
        /var/www/archivesspace/gems/gems/jruby-jars-1.7.6/lib/jruby-stdlib-complete-1.7.6.jar
        /var/www/archivesspace/gems/gems/jruby-jars-1.7.6/lib/jruby-core-complete-1.7.6.jar
        /var/www/archivesspace/gems/gems/jruby-rack-1.1.12/lib/jruby-rack-1.1.12.jar

    java.class.version
        50.0

    java.endorsed.dirs
        /usr/lib/jvm/java-6-openjdk-amd64/jre/lib/endorsed

    java.ext.dirs
        /usr/lib/jvm/java-6-openjdk-amd64/jre/lib/ext
        /usr/java/packages/lib/ext

    java.home
        /usr/lib/jvm/java-6-openjdk-amd64/jre

    java.io.tmpdir
        /var/www/archivesspace/data/tmp

    java.library.path
        /usr/lib/jvm/java-6-openjdk-amd64/jre/lib/amd64/server
        /usr/lib/jvm/java-6-openjdk-amd64/jre/lib/amd64
        /usr/lib/jvm/java-6-openjdk-amd64/jre/../lib/amd64
        /usr/java/packages/lib/amd64
        /usr/lib/x86_64-linux-gnu/jni
        /lib/x86_64-linux-gnu
        /usr/lib/x86_64-linux-gnu
        /usr/lib/jni
        /lib
        /usr/lib

    java.net.preferIPv4Stack
        true

    java.runtime.name
        OpenJDK Runtime Environment

    java.runtime.version
        1.6.0_32-b32

    java.specification.name
        Java Platform API Specification

    java.specification.vendor
        Sun Microsystems Inc.

    java.specification.version
        1.6

    java.vendor
        Sun Microsystems Inc.

    java.vendor.url
        http://java.sun.com/

    java.vendor.url.bug
        http://java.sun.com/cgi-bin/bugreport.cgi

    java.version
        1.6.0_32

    java.vm.info
        mixed mode

    java.vm.name
        OpenJDK 64-Bit Server VM

    java.vm.specification.name
        Java Virtual Machine Specification

    java.vm.specification.vendor
        Sun Microsystems Inc.

    java.vm.specification.version
        1.0

    java.vm.vendor
        Sun Microsystems Inc.

    java.vm.version
        23.25-b01

    jruby.management.enabled
        true

    line.separator
        \n

    org.eclipse.jetty.server.handler.LEVEL
        WARN

    org.eclipse.jetty.webapp.LEVEL
        WARN

    os.arch
        amd64

    os.name
        Linux

    os.version
        3.13.0-35-generic

    path.separator
        :

    solr.data.dir
        /var/www/archivesspace/data/solr_index

    solr.solr.home
        /var/www/archivesspace/data/solr_home

    sun.arch.data.model
        64

    sun.boot.class.path
        /usr/lib/jvm/java-6-openjdk-amd64/jre/lib/resources.jar
        /usr/lib/jvm/java-6-openjdk-amd64/jre/lib/rt.jar
        /usr/lib/jvm/java-6-openjdk-amd64/jre/lib/sunrsasign.jar
        /usr/lib/jvm/java-6-openjdk-amd64/jre/lib/jsse.jar
        /usr/lib/jvm/java-6-openjdk-amd64/jre/lib/jce.jar
        /usr/lib/jvm/java-6-openjdk-amd64/jre/lib/charsets.jar
        /usr/lib/jvm/java-6-openjdk-amd64/jre/lib/jfr.jar
        /usr/lib/jvm/java-6-openjdk-amd64/jre/lib/netx.jar
        /usr/lib/jvm/java-6-openjdk-amd64/jre/lib/plugin.jar
        /usr/lib/jvm/java-6-openjdk-amd64/jre/lib/rhino.jar
        /usr/lib/jvm/java-6-openjdk-amd64/jre/classes

    sun.boot.library.path
        /usr/lib/jvm/java-6-openjdk-amd64/jre/lib/amd64

    sun.cpu.endian
        little

    sun.cpu.isalist

    sun.io.unicode.encoding
        UnicodeLittle

    sun.java.command
        org.jruby.Main --disable-gems --1.9 launcher/launcher.rb

    sun.java.launcher
        SUN_STANDARD

    sun.jnu.encoding
        UTF-8

    sun.management.compiler
        HotSpot 64-Bit Tiered Compilers

    sun.os.patch.level
        unknown

    user.country
        US

    user.dir
        /var/www/archivesspace

    user.home
        /root

    user.language
        en

    user.name
        root

    user.timezone
        America/Chicago

-----Original Message-----
From: archivesspace_users_group-bounces at lyralists.lyrasis.org [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] On Behalf Of Chris Fitzpatrick
Sent: Tuesday, September 16, 2014 11:04 AM
To: Archivesspace Users Group (archivesspace_users_group at lyralists.lyrasis.org)
Subject: Re: [Archivesspace_Users_Group] performance issues

Hi Douglas,

Some quesitons: what is the heap space settings? are you running with jetty or tomcat?  and have you made any tweaks to the default setting?

What your seeing in the logs indicates that the system is indexing the records. Did you do a migration recently?

b,chris.

Chris Fitzpatrick | Developer, ArchivesSpace
Skype: chrisfitzpat  | Phone: 918.236.6048 http://archivesspace.org/

________________________________________
From: archivesspace_users_group-bounces at lyralists.lyrasis.org <archivesspace_users_group-bounces at lyralists.lyrasis.org> on behalf of Douglas James Simmons <dsimmons at siu.edu>
Sent: Tuesday, September 16, 2014 5:42 PM
To: Archivesspace Users Group   (archivesspace_users_group at lyralists.lyrasis.org)
Subject: [Archivesspace_Users_Group] performance issues

I've not found the aspace interface to be fast enough for production use. Some of the retrievals it does while browsing take long enough that I wonder if I'm accidentally running in some sort of debug mode. Does such a thing exist? The VM our pilot is on is a 4-core/4gb instance which should be overkill for the six users we have during the pilot. Log spam in general is pretty bad with aspace, another reason I wonder if I'm running in some kind of debug mode.

Frequently I notice that java claims all the cores and the machine really begins to lag. I looked at the solr interface without really knowing what I'm looking for, is there some info there which would help me figure out what is locking up the java process?

At present the system is making log entries like "Indexed 12325 of 37427 archival_object records in repository University Archives" but I didn't delete the index to force a new one and I'm not sure why re-indexing would take so much processor time when we aren't changing anything.

Thanks in advance for any suggestions as to how to diagnose the poor performance.

Doug Simmons
Procedures and Systems Analyst II

MORRIS LIBRARY - SYSTEMS
MAIL CODE 6632
SOUTHERN ILLINOIS UNIVERSITY
605 AGRICULTURE DRIVE
CARBONDALE, ILLINOIS  62901

dsimmons at lib.siu.edu
P: 618/453-1026
F: 618/453-3031
LIB.SIU.EDU

_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group