[Archivesspace_Users_Group] What is archivesspace doing? 100% CPU usage, also quotation marks not migrating

Wed Mar 4 15:17:11 EST 2015

Chris,

Is this, and the subject heading source migration issue, something that needs to be fixed in a new version of the Archon migration tool?

Matt

From: archivesspace_users_group-bounces at lyralists.lyrasis.org [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] On Behalf Of Prom, Christopher John
Sent: Wednesday, March 04, 2015 11:17 AM
To: Archivesspace Users Group
Subject: Re: [Archivesspace_Users_Group] What is archivesspace doing? 100% CPU usage, also quotation marks not migrating

I just ran a test of this, and the Archon data provider is encoding this as following in the JSON stream as escaped (backslashed) quotes:

This is a snippet from a resource record's scope field:

   "Scope": "CollectionMgr.Description.Scope-Archon\n\tHere is an example of bold text: <emph render='bold'>bold text</emph>\n\tHere is an example of italicized text: <emph render='italic'>italicized text</emph>\n\tHere is an example of underlined text: <emph render='underline'>underlined text</emph>\n\tHere is an example of subscript text: <emph render='sub'>subscript text</emph>\n\tHere is an example of superscript text: <emph render='super'>superscript text</emph>\n\tHere is an example of a URL link: <extref href='http://www.archivesspace.org'>http://www.archivesspace.org</extref>\n\tHere<http://www.archivesspace.org%3c/extref%3e/n/tHere> is an example of an e-mail link: <extref='mailto:mailto:test at archivesspace.org<http://archivesspace.org>?subject=Test&body=Test%20message'>test at archivesspace.org<mailto:test at archivesspace.org></a>\n\tHere is an example of a special character (copyright symbol): (c)\n\tHere is an example of quoted text: \"Quoted text\".",

So, it would appear that the migration tool is converting them to the entity references or they are being displayed that way for some reason.

Chris Prom

On Mar 4, 2015, at 10:34 AM, Gunnells, Diana <Diana.Gunnells at unco.edu<mailto:Diana.Gunnells at unco.edu>> wrote:

We have seen the same issue regarding quotation marks and italicized words in our migration from Archon. I, too, wondered if there might be something that could be done in the migration tool to fix this issue.

Diana

Diana L. Gunnells
III Data Coordinator
James A. Michener Library
Campus Box 48
University of Northern Colorado
Greeley, Co 80639
970-351-2564
diana.gunnells at unco.edu<mailto:diana.gunnells at unco.edu>

From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] On Behalf Of Douglas James Simmons
Sent: Wednesday, March 04, 2015 9:23 AM
To: archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>
Subject: [Archivesspace_Users_Group] What is archivesspace doing? 100% CPU usage, also quotation marks not migrating

Below is an excerpt of the aspace log. The service takes a long time (~10 minutes) to even start, and then it consumes all the cores available. What is it doing? Can I stop it/speed it up?

FYI, I did a migration from archon 3.21-rev2 to aspace 1.0.4 and then upgraded aspace to 1.1.2. I ran it for about 24 hours with Appconfig[:resequence_on_startup] = true and then restarted with this reset to false.

Another minor issue we have is quotations not migrating over correctly. We get collection titles like this in aspace:

"A Historical Look at University Housing" Vertical File Manuscript

I wonder if this might be easily fixed in the migration tool, or if we have to remove the quotes in the data in archon first?

Doug Simmons
Morris Library Systems
SIUC

Mar 2, 2015 12:37:27 PM org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp= path=/update params={} {add=[/repositories/2/archival_objects/40851, /repositories/2/archival_objects/40852, /repositories/2/archival_objects/40853, /repositories/2/archival_objects/40854, /repositories/2/archival_objects/40855, /repositories/2/archival_objects/40856, /repositories/2/archival_objects/40857, /repositories/2/archival_objects/40858, /repositories/2/archival_objects/40859, /repositories/2/archival_objects/40860, ... (25 adds)]} 0 7
Indexed 41050 of 41661 archival_object records in repository University Archives
D, [2015-03-02T12:37:27.618000 #1661] DEBUG -- : Thread-106498: GET /repositories/2/archival_objects?id_set=40926%2C40927%2C40928%2C40929%2C40930%2C40931%2C40932%2C40933%2C40934%2C40935%2C40936%2C40937%2C40938%2C40939%2C40940%2C40941%2C40942%2C40943%2C40944%2C40945%2C40946%2C40947%2C40948%2C40949%2C40950&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classification&resolve%5B%5D=digital_object [session: #<Session:0x2f750ae @store={:user=>"search_indexer", :login_time=>2015-03-02 12:34:08 -0600, :expirable=>false}, @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
D, [2015-03-02T12:37:27.620000 #1661] DEBUG -- : Thread-106498: Post-processed params: {"id_set"=>[40926, 40927, 40928, 40929, 40930, 40931, 40932, 40933, 40934, 40935, 40936, 40937, 40938, 40939, 40940, 40941, 40942, 40943, 40944, 40945, 40946, 40947, 40948, 40949, 40950], "resolve"=>["subjects", "linked_agents", "linked_records", "classification", "digital_object"], :repo_id=>2, "modified_since"=>0}
D, [2015-03-02T12:37:27.628000 #1661] DEBUG -- : Thread-112092: Responded with [200, {"Content-Type"=>"application/json", "Cache-Control"=>"private, must-revalidate, max-age=0", "Content-Length"=>"50105"}, ["[{\"lock_version\":0,\"position\":24,\"publish\":true,\"ref_id\":\"659ea3a9cbf0146d84530fbf0a57db9a\",\"component_id\":\"Folder 25\",\"title\":\"Doctoral Committee Chairman Wendy Jane Broadbooks Statistics and Measurement\",\"display_string\":\"Doctoral Committee Chairman Wendy Jane Broadbooks Statistics and Measurement, 1980-1983\",\"restrictions_apply\":false,\"created_by\":\"adm... in 314.0ms
Mar 2, 2015 12:37:27 PM org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp= path=/update params={} {add=[/repositories/2/archival_objects/40876, /repositories/2/archival_objects/40877, /repositories/2/archival_objects/40878, /repositories/2/archival_objects/40879, /repositories/2/archival_objects/40880, /repositories/2/archival_objects/40881, /repositories/2/archival_objects/40882, /repositories/2/archival_objects/40883, /repositories/2/archival_objects/40884, /repositories/2/archival_objects/40885, ... (25 adds)]} 0 17
Indexed 41075 of 41661 archival_object records in repository University Archives
D, [2015-03-02T12:37:27.722000 #1661] DEBUG -- : Thread-113406: GET /repositories/2/archival_objects?id_set=40951%2C40952%2C40953%2C40954%2C40955%2C40956%2C40957%2C40958%2C40959%2C40960%2C40961%2C40962%2C40963%2C40964%2C40965%2C40966%2C40967%2C40968%2C40969%2C40970%2C40971%2C40972%2C40973%2C40974%2C40975&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classification&resolve%5B%5D=digital_object [session: #<Session:0x3883500e @store={:user=>"search_indexer", :login_time=>2015-03-02 12:34:08 -0600, :expirable=>false}, @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
D, [2015-03-02T12:37:27.724000 #1661] DEBUG -- : Thread-113406: Post-processed params: {"id_set"=>[40951, 40952, 40953, 40954, 40955, 40956, 40957, 40958, 40959, 40960, 40961, 40962, 40963, 40964, 40965, 40966, 40967, 40968, 40969, 40970, 40971, 40972, 40973, 40974, 40975], "resolve"=>["subjects", "linked_agents", "linked_records", "classification", "digital_object"], :repo_id=>2, "modified_since"=>0}
D, [2015-03-02T12:37:27.770000 #1661] DEBUG -- : Thread-111206: Responded with [200, {"Content-Type"=>"application/json", "Cache-Control"=>"private, must-revalidate, max-age=0", "Content-Length"=>"46709"}, ["[{\"lock_version\":0,\"position\":5,\"publish\":true,\"ref_id\":\"5c8c5cd60619e372f3bb9fda9f69aae5\",\"component_id\":\"Folder 6\",\"title\":\"[u]Advisvory Committee[/u], Dale A. Ulrich, Phsical Education Project on Evaluation of Motor Skill Assessment Instrument for Use with Handicapped Students\",\"display_string\":\"[u]Advisvory Committee[/u], Dale A. Ulrich, Phsical Education P... in 275.0ms
Mar 2, 2015 12:37:27 PM org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp= path=/update params={} {add=[/repositories/2/archival_objects/40901, /repositories/2/archival_objects/40902, /repositories/2/archival_objects/40903, /repositories/2/archival_objects/40904, /repositories/2/archival_objects/40905, /repositories/2/archival_objects/40906, /repositories/2/archival_objects/40907, /repositories/2/archival_objects/40908, /repositories/2/archival_objects/40909, /repositories/2/archival_objects/40910, ... (25 adds)]} 0 7
Indexed 41100 of 41661 archival_object records in repository University Archives
D, [2015-03-02T12:37:27.860000 #1661] DEBUG -- : Thread-104680: GET /repositories/2/archival_objects?id_set=40976%2C40977%2C40978%2C40979%2C40980%2C40981%2C40982%2C40983%2C40984%2C40985%2C40986%2C40987%2C40988%2C40989%2C40990%2C40991%2C40992%2C40993%2C40994%2C40995%2C40996%2C40997%2C40998%2C40999%2C41000&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classification&resolve%5B%5D=digital_object [session: #<Session:0x45bb59d @store={:user=>"search_indexer", :login_time=>2015-03-02 12:34:08 -0600, :expirable=>false}, @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
D, [2015-03-02T12:37:27.860000 #1661] DEBUG -- : Thread-106498: Responded with [200, {"Content-Type"=>"application/json", "Cache-Control"=>"private, must-revalidate, max-age=0", "Content-Length"=>"42584"}, ["[{\"lock_version\":0,\"position\":2,\"publish\":true,\"ref_id\":\"92bf78d7ed852be131ce495d923ca07e\",\"component_id\":\"Folder 31\",\"title\":\"Problem Sets at the End of Chapters in RLD Wright's  [u]Understanding Statistics[/u]\",\"display_string\":\"Problem Sets at the End of Chapters in RLD Wright's  [u]Understanding Statistics[/u]\",\"restrictions_apply\":false,\"created_by\":\... in 245.0ms
D, [2015-03-02T12:37:27.867000 #1661] DEBUG -- : Thread-104680: Post-processed params: {"id_set"=>[40976, 40977, 40978, 40979, 40980, 40981, 40982, 40983, 40984, 40985, 40986, 40987, 40988, 40989, 40990, 40991, 40992, 40993, 40994, 40995, 40996, 40997, 40998, 40999, 41000], "resolve"=>["subjects", "linked_agents", "linked_records", "classification", "digital_object"], :repo_id=>2, "modified_since"=>0}
Mar 2, 2015 12:37:27 PM org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp= path=/update params={} {add=[/repositories/2/archival_objects/40926, /repositories/2/archival_objects/40927, /repositories/2/archival_objects/40928, /repositories/2/archival_objects/40929, /repositories/2/archival_objects/40930, /repositories/2/archival_objects/40931, /repositories/2/archival_objects/40932, /repositories/2/archival_objects/40933, /repositories/2/archival_objects/40934, /repositories/2/archival_objects/40935, ... (25 adds)]} 0 6
Indexed 41125 of 41661 archival_object records in repository University Archives
D, [2015-03-02T12:37:27.938000 #1661] DEBUG -- : Thread-112092: GET /repositories/2/archival_objects?id_set=41001%2C41002%2C41003%2C41004%2C41005%2C41006%2C41007%2C41008%2C41009%2C41010%2C41011%2C41012%2C41013%2C41014%2C41015%2C41016%2C41017%2C41018%2C41019%2C41020%2C41021%2C41022%2C41023%2C41024%2C41025&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classification&resolve%5B%5D=digital_object [session: #<Session:0x3fc11c13 @store={:user=>"search_indexer", :login_time=>2015-03-02 12:34:08 -0600, :expirable=>false}, @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
D, [2015-03-02T12:37:27.941000 #1661] DEBUG -- : Thread-112092: Post-processed params: {"id_set"=>[41001, 41002, 41003, 41004, 41005, 41006, 41007, 41008, 41009, 41010, 41011, 41012, 41013, 41014, 41015, 41016, 41017, 41018, 41019, 41020, 41021, 41022, 41023, 41024, 41025], "resolve"=>["subjects", "linked_agents", "linked_records", "classification", "digital_object"], :repo_id=>2, "modified_since"=>0}
D, [2015-03-02T12:37:27.988000 #1661] DEBUG -- : Thread-113406: Responded with [200, {"Content-Type"=>"application/json", "Cache-Control"=>"private, must-revalidate, max-age=0", "Content-Length"=>"45706"}, ["[{\"lock_version\":0,\"position\":9,\"publish\":true,\"ref_id\":\"4812d9379e62e7b8a05317eb34b5d871\",\"component_id\":\"Folder 10\",\"title\":\"Educational Psychology 507: Summary of Models by Number of Estimates and Solution of Regression Equations with Two Predictors\",\"display_string\":\"Educational Psychology 507: Summary of Models by Number of Estimates and Solution of Regres... in 270.0ms
Mar 2, 2015 12:37:28 PM org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp= path=/update params={} {add=[/repositories/2/archival_objects/40951, /repositories/2/archival_objects/40952, /repositories/2/archival_objects/40953, /repositories/2/archival_objects/40954, /repositories/2/archival_objects/40955, /repositories/2/archival_objects/40956, /repositories/2/archival_objects/40957, /repositories/2/archival_objects/40958, /repositories/2/archival_objects/40959, /repositories/2/archival_objects/40960, ... (25 adds)]} 0 7
Indexed 41150 of 41661 archival_object records in repository University Archives
D, [2015-03-02T12:37:28.069000 #1661] DEBUG -- : Thread-106498: GET /repositories/2/archival_objects?id_set=41026%2C41027%2C41028%2C41029%2C41030%2C41031%2C41032%2C41033%2C41034%2C41035%2C41036%2C41037%2C41038%2C41039%2C41040%2C41041%2C41042%2C41043%2C41044%2C41045%2C41046%2C41047%2C41048%2C41049%2C41050&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classification&resolve%5B%5D=digital_object [session: #<Session:0x568e24f8 @store={:user=>"search_indexer", :login_time=>2015-03-02 12:34:08 -0600, :expirable=>false}, @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
D, [2015-03-02T12:37:28.072000 #1661] DEBUG -- : Thread-106498: Post-processed params: {"id_set"=>[41026, 41027, 41028, 41029, 41030, 41031, 41032, 41033, 41034, 41035, 41036, 41037, 41038, 41039, 41040, 41041, 41042, 41043, 41044, 41045, 41046, 41047, 41048, 41049, 41050], "resolve"=>["subjects", "linked_agents", "linked_records", "classification", "digital_object"], :repo_id=>2, "modified_since"=>0}
D, [2015-03-02T12:37:28.127000 #1661] DEBUG -- : Thread-104680: Responded with [200, {"Content-Type"=>"application/json", "Cache-Control"=>"private, must-revalidate, max-age=0", "Content-Length"=>"49421"}, ["[{\"lock_version\":0,\"position\":34,\"publish\":true,\"ref_id\":\"7c64551408ea76652410341c1ea0c5ba\",\"component_id\":\"Folder 35\",\"title\":\"Educational Psychology 507: Sample for Class, Computer Printout #7, SAS, Fall\",\"display_string\":\"Educational Psychology 507: Sample for Class, Computer Printout #7, SAS, Fall, 1987\",\"restrictions_apply\":false,\"created_by\":\"admin\... in 272.0ms
D, [2015-03-02T12:37:28.149000 #1661] DEBUG -- : Thread-112092: Responded with [200, {"Content-Type"=>"application/json", "Cache-Control"=>"private, must-revalidate, max-age=0", "Content-Length"=>"46655"}, ["[{\"lock_version\":0,\"position\":24,\"publish\":true,\"ref_id\":\"2068ac71b12ea2dbb48b35edf6735275\",\"component_id\":\"Folder 25\",\"title\":\"Guidance 507: Computer Problem 3, SAS, One-Way ANOVA with Dummy Coding Models 2 and 3, Use Intercept Models 11 and 12, No Intercept, Spring\",\"display_string\":\"Guidance 507: Computer Problem 3, SAS, One-Way ANOVA with Dummy Coding Model... in 215.0ms
Mar 2, 2015 12:37:28 PM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start commit{flags=0,_version_=0,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
Mar 2, 2015 12:37:28 PM org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp= path=/update params={} {deleteByQuery=primary_type:tree_view AND root_uri:("/repositories/2/resources/276")} 0 0
Mar 2, 2015 12:37:28 PM org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp= path=/update params={} {add=[/repositories/2/archival_objects/40976, /repositories/2/archival_objects/40977, /repositories/2/archival_objects/40978, /repositories/2/archival_objects/40979, /repositories/2/archival_objects/40980, /repositories/2/archival_objects/40981, /repositories/2/archival_objects/40982, /repositories/2/archival_objects/40983, /repositories/2/archival_objects/40984, /repositories/2/archival_objects/40985, ... (25 adds)]} 0 30
Indexed 41175 of 41661 archival_object records in repository University Archives
D, [2015-03-02T12:37:28.237000 #1661] DEBUG -- : Thread-111206: GET /repositories/2/resources/276/tree [session: #<Session:0x4f648888 @store={:user=>"search_indexer", :login_time=>2015-03-02 12:34:08 -0600, :expirable=>false}, @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
D, [2015-03-02T12:37:28.238000 #1661] DEBUG -- : Thread-113406: GET /repositories/2/archival_objects?id_set=41051%2C41052%2C41053%2C41054%2C41055%2C41056%2C41057%2C41058%2C41059%2C41060%2C41061%2C41062%2C41063%2C41064%2C41065%2C41066%2C41067%2C41068%2C41069%2C41070%2C41071%2C41072%2C41073%2C41074%2C41075&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classification&resolve%5B%5D=digital_object [session: #<Session:0x5619daba @store={:user=>"search_indexer", :login_time=>2015-03-02 12:34:08 -0600, :expirable=>false}, @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
D, [2015-03-02T12:37:28.243000 #1661] DEBUG -- : Thread-113406: Post-processed params: {"id_set"=>[41051, 41052, 41053, 41054, 41055, 41056, 41057, 41058, 41059, 41060, 41061, 41062, 41063, 41064, 41065, 41066, 41067, 41068, 41069, 41070, 41071, 41072, 41073, 41074, 41075], "resolve"=>["subjects", "linked_agents", "linked_records", "classification", "digital_object"], :repo_id=>2, "modified_since"=>0}
D, [2015-03-02T12:37:28.245000 #1661] DEBUG -- : Thread-111206: Post-processed params: {:id=>276, :repo_id=>2}

_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20150304/67d74156/attachment.html>