[Archivesspace_Users_Group] What is archivesspace doing? 100% CPU usage, also quotation marks not migrating

Wed Mar 4 12:41:16 EST 2015

Unless I'm missing something, this isn't just a problem with the migration tool.

The attached screenshots are from a record that I created using AS admin. The same title is being displayed three different ways. I thought I remember this being brought up a while back, but I didn't remember if there was a resolution. Seems like a good time to bring it up again.

-Andrew Holland

From: archivesspace_users_group-bounces at lyralists.lyrasis.org [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] On Behalf Of Nathan Stevens
Sent: Wednesday, March 04, 2015 11:35 AM
To: Archivesspace Users Group
Subject: Re: [Archivesspace_Users_Group] What is archivesspace doing? 100% CPU usage, also quotation marks not migrating

I suspect the issue here is that if the Archon Migration tool is being executed on Windows, then the correct encoding (UTF-8) is not being used.  To resolve just run the migration tool with the following command to set the correct encoding:

 java -Dfile.encoding=UTF-8 -jar archon-migration.war
See attached screenshot showing quotes being displayed correctly after Archon data migration.

On Wed, Mar 4, 2015 at 12:17 PM, Prom, Christopher John <prom at illinois.edu<mailto:prom at illinois.edu>> wrote:
I just ran a test of this, and the Archon data provider is encoding this as following in the JSON stream as escaped (backslashed) quotes:

This is a snippet from a resource record's scope field:

   "Scope": "CollectionMgr.Description.Scope-Archon\n\tHere is an example of bold text: <emph render='bold'>bold text</emph>\n\tHere is an example of italicized text: <emph render='italic'>italicized text</emph>\n\tHere is an example of underlined text: <emph render='underline'>underlined text</emph>\n\tHere is an example of subscript text: <emph render='sub'>subscript text</emph>\n\tHere is an example of superscript text: <emph render='super'>superscript text</emph>\n\tHere is an example of a URL link: <extref href='http://www.archivesspace.org'>http://www.archivesspace.org</extref>\n\tHere<http://www.archivesspace.org%3c/extref%3e/n/tHere> is an example of an e-mail link: <extref='mailto:mailto<mailto:mailto>:test at archivesspace.org<http://archivesspace.org>?subject=Test&body=Test%20message'>test at archivesspace.org<mailto:test at archivesspace.org></a>\n\tHere is an example of a special character (copyright symbol): ©\n\tHere is an example of quoted text: \"Quoted text\".",

So, it would appear that the migration tool is converting them to the entity references or they are being displayed that way for some reason.

Chris Prom

On Mar 4, 2015, at 10:34 AM, Gunnells, Diana <Diana.Gunnells at unco.edu<mailto:Diana.Gunnells at unco.edu>> wrote:

We have seen the same issue regarding quotation marks and italicized words in our migration from Archon. I, too, wondered if there might be something that could be done in the migration tool to fix this issue.

Diana

Diana L. Gunnells
III Data Coordinator
James A. Michener Library
Campus Box 48
University of Northern Colorado
Greeley, Co 80639
970-351-2564<tel:970-351-2564>
diana.gunnells at unco.edu<mailto:diana.gunnells at unco.edu>

From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] On Behalf Of Douglas James Simmons
Sent: Wednesday, March 04, 2015 9:23 AM
To: archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>
Subject: [Archivesspace_Users_Group] What is archivesspace doing? 100% CPU usage, also quotation marks not migrating

Below is an excerpt of the aspace log. The service takes a long time (~10 minutes) to even start, and then it consumes all the cores available. What is it doing? Can I stop it/speed it up?

FYI, I did a migration from archon 3.21-rev2 to aspace 1.0.4 and then upgraded aspace to 1.1.2. I ran it for about 24 hours with Appconfig[:resequence_on_startup] = true and then restarted with this reset to false.

Another minor issue we have is quotations not migrating over correctly. We get collection titles like this in aspace:

"A Historical Look at University Housing" Vertical File Manuscript

I wonder if this might be easily fixed in the migration tool, or if we have to remove the quotes in the data in archon first?

Doug Simmons
Morris Library Systems
SIUC

Mar 2, 2015 12:37:27 PM org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp= path=/update params={} {add=[/repositories/2/archival_objects/40851, /repositories/2/archival_objects/40852, /repositories/2/archival_objects/40853, /repositories/2/archival_objects/40854, /repositories/2/archival_objects/40855, /repositories/2/archival_objects/40856, /repositories/2/archival_objects/40857, /repositories/2/archival_objects/40858, /repositories/2/archival_objects/40859, /repositories/2/archival_objects/40860, ... (25 adds)]} 0 7
Indexed 41050 of 41661 archival_object records in repository University Archives
D, [2015-03-02T12:37:27.618000 #1661] DEBUG -- : Thread-106498: GET /repositories/2/archival_objects?id_set=40926%2C40927%2C40928%2C40929%2C40930%2C40931%2C40932%2C40933%2C40934%2C40935%2C40936%2C40937%2C40938%2C40939%2C40940%2C40941%2C40942%2C40943%2C40944%2C40945%2C40946%2C40947%2C40948%2C40949%2C40950&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classification&resolve%5B%5D=digital_object [session: #<Session:0x2f750ae @store={:user=>"search_indexer", :login_time=>2015-03-02 12:34:08 -0600, :expirable=>false}, @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
D, [2015-03-02T12:37:27.620000 #1661] DEBUG -- : Thread-106498: Post-processed params: {"id_set"=>[40926, 40927, 40928, 40929, 40930, 40931, 40932, 40933, 40934, 40935, 40936, 40937, 40938, 40939, 40940, 40941, 40942, 40943, 40944, 40945, 40946, 40947, 40948, 40949, 40950], "resolve"=>["subjects", "linked_agents", "linked_records", "classification", "digital_object"], :repo_id=>2, "modified_since"=>0}
D, [2015-03-02T12:37:27.628000 #1661] DEBUG -- : Thread-112092: Responded with [200, {"Content-Type"=>"application/json", "Cache-Control"=>"private, must-revalidate, max-age=0", "Content-Length"=>"50105"}, ["[{\"lock_version\":0,\"position\":24,\"publish\":true,\"ref_id\":\"659ea3a9cbf0146d84530fbf0a57db9a\",\"component_id\":\"Folder 25\",\"title\":\"Doctoral Committee Chairman Wendy Jane Broadbooks Statistics and Measurement\",\"display_string\":\"Doctoral Committee Chairman Wendy Jane Broadbooks Statistics and Measurement, 1980-1983\",\"restrictions_apply\":false,\"created_by\":\"adm... in 314.0ms
Mar 2, 2015 12:37:27 PM org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp= path=/update params={} {add=[/repositories/2/archival_objects/40876, /repositories/2/archival_objects/40877, /repositories/2/archival_objects/40878, /repositories/2/archival_objects/40879, /repositories/2/archival_objects/40880, /repositories/2/archival_objects/40881, /repositories/2/archival_objects/40882, /repositories/2/archival_objects/40883, /repositories/2/archival_objects/40884, /repositories/2/archival_objects/40885, ... (25 adds)]} 0 17
Indexed 41075 of 41661 archival_object records in repository University Archives
D, [2015-03-02T12:37:27.722000 #1661] DEBUG -- : Thread-113406: GET /repositories/2/archival_objects?id_set=40951%2C40952%2C40953%2C40954%2C40955%2C40956%2C40957%2C40958%2C40959%2C40960%2C40961%2C40962%2C40963%2C40964%2C40965%2C40966%2C40967%2C40968%2C40969%2C40970%2C40971%2C40972%2C40973%2C40974%2C40975&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classification&resolve%5B%5D=digital_object [session: #<Session:0x3883500e @store={:user=>"search_indexer", :login_time=>2015-03-02 12:34:08 -0600, :expirable=>false}, @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
D, [2015-03-02T12:37:27.724000 #1661] DEBUG -- : Thread-113406: Post-processed params: {"id_set"=>[40951, 40952, 40953, 40954, 40955, 40956, 40957, 40958, 40959, 40960, 40961, 40962, 40963, 40964, 40965, 40966, 40967, 40968, 40969, 40970, 40971, 40972, 40973, 40974, 40975], "resolve"=>["subjects", "linked_agents", "linked_records", "classification", "digital_object"], :repo_id=>2, "modified_since"=>0}
D, [2015-03-02T12:37:27.770000 #1661] DEBUG -- : Thread-111206: Responded with [200, {"Content-Type"=>"application/json", "Cache-Control"=>"private, must-revalidate, max-age=0", "Content-Length"=>"46709"}, ["[{\"lock_version\":0,\"position\":5,\"publish\":true,\"ref_id\":\"5c8c5cd60619e372f3bb9fda9f69aae5\",\"component_id\":\"Folder 6\",\"title\":\"[u]Advisvory Committee[/u], Dale A. Ulrich, Phsical Education Project on Evaluation of Motor Skill Assessment Instrument for Use with Handicapped Students\",\"display_string\":\"[u]Advisvory Committee[/u], Dale A. Ulrich, Phsical Education P... in 275.0ms
Mar 2, 2015 12:37:27 PM org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp= path=/update params={} {add=[/repositories/2/archival_objects/40901, /repositories/2/archival_objects/40902, /repositories/2/archival_objects/40903, /repositories/2/archival_objects/40904, /repositories/2/archival_objects/40905, /repositories/2/archival_objects/40906, /repositories/2/archival_objects/40907, /repositories/2/archival_objects/40908, /repositories/2/archival_objects/40909, /repositories/2/archival_objects/40910, ... (25 adds)]} 0 7
Indexed 41100 of 41661 archival_object records in repository University Archives
D, [2015-03-02T12:37:27.860000 #1661] DEBUG -- : Thread-104680: GET /repositories/2/archival_objects?id_set=40976%2C40977%2C40978%2C40979%2C40980%2C40981%2C40982%2C40983%2C40984%2C40985%2C40986%2C40987%2C40988%2C40989%2C40990%2C40991%2C40992%2C40993%2C40994%2C40995%2C40996%2C40997%2C40998%2C40999%2C41000&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classification&resolve%5B%5D=digital_object [session: #<Session:0x45bb59d @store={:user=>"search_indexer", :login_time=>2015-03-02 12:34:08 -0600, :expirable=>false}, @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
D, [2015-03-02T12:37:27.860000 #1661] DEBUG -- : Thread-106498: Responded with [200, {"Content-Type"=>"application/json", "Cache-Control"=>"private, must-revalidate, max-age=0", "Content-Length"=>"42584"}, ["[{\"lock_version\":0,\"position\":2,\"publish\":true,\"ref_id\":\"92bf78d7ed852be131ce495d923ca07e\",\"component_id\":\"Folder 31\",\"title\":\"Problem Sets at the End of Chapters in RLD Wright's  [u]Understanding Statistics[/u]\",\"display_string\":\"Problem Sets at the End of Chapters in RLD Wright's  [u]Understanding Statistics[/u]\",\"restrictions_apply\":false,\"created_by\":\... in 245.0ms
D, [2015-03-02T12:37:27.867000 #1661] DEBUG -- : Thread-104680: Post-processed params: {"id_set"=>[40976, 40977, 40978, 40979, 40980, 40981, 40982, 40983, 40984, 40985, 40986, 40987, 40988, 40989, 40990, 40991, 40992, 40993, 40994, 40995, 40996, 40997, 40998, 40999, 41000], "resolve"=>["subjects", "linked_agents", "linked_records", "classification", "digital_object"], :repo_id=>2, "modified_since"=>0}
Mar 2, 2015 12:37:27 PM org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp= path=/update params={} {add=[/repositories/2/archival_objects/40926, /repositories/2/archival_objects/40927, /repositories/2/archival_objects/40928, /repositories/2/archival_objects/40929, /repositories/2/archival_objects/40930, /repositories/2/archival_objects/40931, /repositories/2/archival_objects/40932, /repositories/2/archival_objects/40933, /repositories/2/archival_objects/40934, /repositories/2/archival_objects/40935, ... (25 adds)]} 0 6
Indexed 41125 of 41661 archival_object records in repository University Archives
D, [2015-03-02T12:37:27.938000 #1661] DEBUG -- : Thread-112092: GET /repositories/2/archival_objects?id_set=41001%2C41002%2C41003%2C41004%2C41005%2C41006%2C41007%2C41008%2C41009%2C41010%2C41011%2C41012%2C41013%2C41014%2C41015%2C41016%2C41017%2C41018%2C41019%2C41020%2C41021%2C41022%2C41023%2C41024%2C41025&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classification&resolve%5B%5D=digital_object [session: #<Session:0x3fc11c13 @store={:user=>"search_indexer", :login_time=>2015-03-02 12:34:08 -0600, :expirable=>false}, @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
D, [2015-03-02T12:37:27.941000 #1661] DEBUG -- : Thread-112092: Post-processed params: {"id_set"=>[41001, 41002, 41003, 41004, 41005, 41006, 41007, 41008, 41009, 41010, 41011, 41012, 41013, 41014, 41015, 41016, 41017, 41018, 41019, 41020, 41021, 41022, 41023, 41024, 41025], "resolve"=>["subjects", "linked_agents", "linked_records", "classification", "digital_object"], :repo_id=>2, "modified_since"=>0}
D, [2015-03-02T12:37:27.988000 #1661] DEBUG -- : Thread-113406: Responded with [200, {"Content-Type"=>"application/json", "Cache-Control"=>"private, must-revalidate, max-age=0", "Content-Length"=>"45706"}, ["[{\"lock_version\":0,\"position\":9,\"publish\":true,\"ref_id\":\"4812d9379e62e7b8a05317eb34b5d871\",\"component_id\":\"Folder 10\",\"title\":\"Educational Psychology 507: Summary of Models by Number of Estimates and Solution of Regression Equations with Two Predictors\",\"display_string\":\"Educational Psychology 507: Summary of Models by Number of Estimates and Solution of Regres... in 270.0ms
Mar 2, 2015 12:37:28 PM org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp= path=/update params={} {add=[/repositories/2/archival_objects/40951, /repositories/2/archival_objects/40952, /repositories/2/archival_objects/40953, /repositories/2/archival_objects/40954, /repositories/2/archival_objects/40955, /repositories/2/archival_objects/40956, /repositories/2/archival_objects/40957, /repositories/2/archival_objects/40958, /repositories/2/archival_objects/40959, /repositories/2/archival_objects/40960, ... (25 adds)]} 0 7
Indexed 41150 of 41661 archival_object records in repository University Archives
D, [2015-03-02T12:37:28.069000 #1661] DEBUG -- : Thread-106498: GET /repositories/2/archival_objects?id_set=41026%2C41027%2C41028%2C41029%2C41030%2C41031%2C41032%2C41033%2C41034%2C41035%2C41036%2C41037%2C41038%2C41039%2C41040%2C41041%2C41042%2C41043%2C41044%2C41045%2C41046%2C41047%2C41048%2C41049%2C41050&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classification&resolve%5B%5D=digital_object [session: #<Session:0x568e24f8 @store={:user=>"search_indexer", :login_time=>2015-03-02 12:34:08 -0600, :expirable=>false}, @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
D, [2015-03-02T12:37:28.072000 #1661] DEBUG -- : Thread-106498: Post-processed params: {"id_set"=>[41026, 41027, 41028, 41029, 41030, 41031, 41032, 41033, 41034, 41035, 41036, 41037, 41038, 41039, 41040, 41041, 41042, 41043, 41044, 41045, 41046, 41047, 41048, 41049, 41050], "resolve"=>["subjects", "linked_agents", "linked_records", "classification", "digital_object"], :repo_id=>2, "modified_since"=>0}
D, [2015-03-02T12:37:28.127000 #1661] DEBUG -- : Thread-104680: Responded with [200, {"Content-Type"=>"application/json", "Cache-Control"=>"private, must-revalidate, max-age=0", "Content-Length"=>"49421"}, ["[{\"lock_version\":0,\"position\":34,\"publish\":true,\"ref_id\":\"7c64551408ea76652410341c1ea0c5ba\",\"component_id\":\"Folder 35\",\"title\":\"Educational Psychology 507: Sample for Class, Computer Printout #7, SAS, Fall\",\"display_string\":\"Educational Psychology 507: Sample for Class, Computer Printout #7, SAS, Fall, 1987\",\"restrictions_apply\":false,\"created_by\":\"admin\... in 272.0ms
D, [2015-03-02T12:37:28.149000 #1661] DEBUG -- : Thread-112092: Responded with [200, {"Content-Type"=>"application/json", "Cache-Control"=>"private, must-revalidate, max-age=0", "Content-Length"=>"46655"}, ["[{\"lock_version\":0,\"position\":24,\"publish\":true,\"ref_id\":\"2068ac71b12ea2dbb48b35edf6735275\",\"component_id\":\"Folder 25\",\"title\":\"Guidance 507: Computer Problem 3, SAS, One-Way ANOVA with Dummy Coding Models 2 and 3, Use Intercept Models 11 and 12, No Intercept, Spring\",\"display_string\":\"Guidance 507: Computer Problem 3, SAS, One-Way ANOVA with Dummy Coding Model... in 215.0ms
Mar 2, 2015 12:37:28 PM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start commit{flags=0,_version_=0,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
Mar 2, 2015 12:37:28 PM org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp= path=/update params={} {deleteByQuery=primary_type:tree_view AND root_uri:("/repositories/2/resources/276")} 0 0
Mar 2, 2015 12:37:28 PM org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp= path=/update params={} {add=[/repositories/2/archival_objects/40976, /repositories/2/archival_objects/40977, /repositories/2/archival_objects/40978, /repositories/2/archival_objects/40979, /repositories/2/archival_objects/40980, /repositories/2/archival_objects/40981, /repositories/2/archival_objects/40982, /repositories/2/archival_objects/40983, /repositories/2/archival_objects/40984, /repositories/2/archival_objects/40985, ... (25 adds)]} 0 30
Indexed 41175 of 41661 archival_object records in repository University Archives
D, [2015-03-02T12:37:28.237000 #1661] DEBUG -- : Thread-111206: GET /repositories/2/resources/276/tree [session: #<Session:0x4f648888 @store={:user=>"search_indexer", :login_time=>2015-03-02 12:34:08 -0600, :expirable=>false}, @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
D, [2015-03-02T12:37:28.238000 #1661] DEBUG -- : Thread-113406: GET /repositories/2/archival_objects?id_set=41051%2C41052%2C41053%2C41054%2C41055%2C41056%2C41057%2C41058%2C41059%2C41060%2C41061%2C41062%2C41063%2C41064%2C41065%2C41066%2C41067%2C41068%2C41069%2C41070%2C41071%2C41072%2C41073%2C41074%2C41075&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classification&resolve%5B%5D=digital_object [session: #<Session:0x5619daba @store={:user=>"search_indexer", :login_time=>2015-03-02 12:34:08 -0600, :expirable=>false}, @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
D, [2015-03-02T12:37:28.243000 #1661] DEBUG -- : Thread-113406: Post-processed params: {"id_set"=>[41051, 41052, 41053, 41054, 41055, 41056, 41057, 41058, 41059, 41060, 41061, 41062, 41063, 41064, 41065, 41066, 41067, 41068, 41069, 41070, 41071, 41072, 41073, 41074, 41075], "resolve"=>["subjects", "linked_agents", "linked_records", "classification", "digital_object"], :repo_id=>2, "modified_since"=>0}
D, [2015-03-02T12:37:28.245000 #1661] DEBUG -- : Thread-111206: Post-processed params: {:id=>276, :repo_id=>2}

_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

--
Nathan Stevens
Programmer/Analyst
Digital Library Technology Services
New York University

1212-998-2653
ns96 at nyu.edu<mailto:ns96 at nyu.edu>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20150304/638f22ad/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: search_results.JPG
Type: image/jpeg
Size: 22978 bytes
Desc: search_results.JPG
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20150304/638f22ad/attachment.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: details.JPG
Type: image/jpeg
Size: 29870 bytes
Desc: details.JPG
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20150304/638f22ad/attachment-0001.jpe>