[Archivesspace_Users_Group] What is archivesspace doing? 100% CPU usage, also quotation marks not migrating

Wed Mar 4 12:35:00 EST 2015

I suspect the issue here is that if the Archon Migration tool is being
executed on Windows, then the correct encoding (UTF-8) is not being used.
To resolve just run the migration tool with the following command to set
the correct encoding:

 java -Dfile.encoding=UTF-8 -jar archon-migration.war

See attached screenshot showing quotes being displayed correctly after
Archon data migration.

On Wed, Mar 4, 2015 at 12:17 PM, Prom, Christopher John <prom at illinois.edu>
wrote:

>  I just ran a test of this, and the Archon data provider is encoding this
> as following in the JSON stream as escaped (backslashed) quotes:
>
>  This is a snippet from a resource record's scope field:
>
>     "Scope": "CollectionMgr.Description.Scope-Archon\n\tHere is an
> example of bold text: <emph render='bold'>bold text</emph>\n\tHere is an
> example of italicized text: <emph render='italic'>italicized
> text</emph>\n\tHere is an example of underlined text: <emph
> render='underline'>underlined text</emph>\n\tHere is an example of
> subscript text: <emph render='sub'>subscript text</emph>\n\tHere is an
> example of superscript text: <emph render='super'>superscript
> text</emph>\n\tHere is an example of a URL link: <extref href='
> http://www.archivesspace.org'>
> http://www.archivesspace.org</extref>\n\tHere is an example of an e-mail
> link: <extref='mailto:mailto:test at archivesspace.org
> ?subject=Test&body=Test%20message'>test at archivesspace.org</a>\n\tHere is
> an example of a special character (copyright symbol): ©\n\t*Here is an
> example of quoted text: \"Quoted text\".*",
>
>  So, it would appear that the migration tool is converting them to the
> entity references or they are being displayed that way for some reason.
>
>  Chris Prom
>
>  On Mar 4, 2015, at 10:34 AM, Gunnells, Diana <Diana.Gunnells at unco.edu>
> wrote:
>
>   We have seen the same issue regarding quotation marks and italicized
> words in our migration from Archon. I, too, wondered if there might be
> something that could be done in the migration tool to fix this issue.
>
>
>
> *Diana*
>
>
>
> Diana L. Gunnells
>
> III Data Coordinator
>
> James A. Michener Library
>
> Campus Box 48
>
> University of Northern Colorado
>
> Greeley, Co 80639
>
> 970-351-2564
>
> diana.gunnells at unco.edu
>
>
>
> *From:* archivesspace_users_group-bounces at lyralists.lyrasis.org [
> mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org
> <archivesspace_users_group-bounces at lyralists.lyrasis.org>] *On Behalf Of *Douglas
> James Simmons
> *Sent:* Wednesday, March 04, 2015 9:23 AM
> *To:* archivesspace_users_group at lyralists.lyrasis.org
> *Subject:* [Archivesspace_Users_Group] What is archivesspace doing? 100%
> CPU usage, also quotation marks not migrating
>
>
>
>      Below is an excerpt of the aspace log. The service takes a long time
> (~10 minutes) to even start, and then it consumes all the cores available.
> What is it doing? Can I stop it/speed it up?
>
>
>
> FYI, I did a migration from archon 3.21-rev2 to aspace 1.0.4 and then
> upgraded aspace to 1.1.2. I ran it for about 24 hours with
> Appconfig[:resequence_on_startup] = true and then restarted with this reset
> to false.
>
>
>
> Another minor issue we have is quotations not migrating over correctly. We
> get collection titles like this in aspace:
>
>
>
> "A Historical Look at University Housing" Vertical File
> Manuscript
>
>
>
> I wonder if this might be easily fixed in the migration tool, or if we
> have to remove the quotes in the data in archon first?
>
>
>
> Doug Simmons
>
> Morris Library Systems
>
> SIUC
>
>
>
> Mar 2, 2015 12:37:27 PM
> org.apache.solr.update.processor.LogUpdateProcessor finish
>
> INFO: [collection1] webapp= path=/update params={}
> {add=[/repositories/2/archival_objects/40851,
> /repositories/2/archival_objects/40852,
> /repositories/2/archival_objects/40853,
> /repositories/2/archival_objects/40854,
> /repositories/2/archival_objects/40855,
> /repositories/2/archival_objects/40856,
> /repositories/2/archival_objects/40857,
> /repositories/2/archival_objects/40858,
> /repositories/2/archival_objects/40859,
> /repositories/2/archival_objects/40860, ... (25 adds)]} 0 7
>
> Indexed 41050 of 41661 archival_object records in repository University
> Archives
>
> D, [2015-03-02T12:37:27.618000 #1661] DEBUG -- : Thread-106498: GET
> /repositories/2/archival_objects?id_set=40926%2C40927%2C40928%2C40929%2C40930%2C40931%2C40932%2C40933%2C40934%2C40935%2C40936%2C40937%2C40938%2C40939%2C40940%2C40941%2C40942%2C40943%2C40944%2C40945%2C40946%2C40947%2C40948%2C40949%2C40950&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classification&resolve%5B%5D=digital_object
> [session: #<Session:0x2f750ae @store={:user=>"search_indexer",
> :login_time=>2015-03-02 12:34:08 -0600, :expirable=>false},
> @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
>
> D, [2015-03-02T12:37:27.620000 #1661] DEBUG -- : Thread-106498:
> Post-processed params: {"id_set"=>[40926, 40927, 40928, 40929, 40930,
> 40931, 40932, 40933, 40934, 40935, 40936, 40937, 40938, 40939, 40940,
> 40941, 40942, 40943, 40944, 40945, 40946, 40947, 40948, 40949, 40950],
> "resolve"=>["subjects", "linked_agents", "linked_records",
> "classification", "digital_object"], :repo_id=>2, "modified_since"=>0}
>
> D, [2015-03-02T12:37:27.628000 #1661] DEBUG -- : Thread-112092: Responded
> with [200, {"Content-Type"=>"application/json", "Cache-Control"=>"private,
> must-revalidate, max-age=0", "Content-Length"=>"50105"},
> ["[{\"lock_version\":0,\"position\":24,\"publish\":true,\"ref_id\":\"659ea3a9cbf0146d84530fbf0a57db9a\",\"component_id\":\"Folder
> 25\",\"title\":\"Doctoral Committee Chairman Wendy Jane Broadbooks
> Statistics and Measurement\",\"display_string\":\"Doctoral Committee
> Chairman Wendy Jane Broadbooks Statistics and Measurement,
> 1980-1983\",\"restrictions_apply\":false,\"created_by\":\"adm... in 314.0ms
>
> Mar 2, 2015 12:37:27 PM
> org.apache.solr.update.processor.LogUpdateProcessor finish
>
> INFO: [collection1] webapp= path=/update params={}
> {add=[/repositories/2/archival_objects/40876,
> /repositories/2/archival_objects/40877,
> /repositories/2/archival_objects/40878,
> /repositories/2/archival_objects/40879,
> /repositories/2/archival_objects/40880,
> /repositories/2/archival_objects/40881,
> /repositories/2/archival_objects/40882,
> /repositories/2/archival_objects/40883,
> /repositories/2/archival_objects/40884,
> /repositories/2/archival_objects/40885, ... (25 adds)]} 0 17
>
> Indexed 41075 of 41661 archival_object records in repository University
> Archives
>
> D, [2015-03-02T12:37:27.722000 #1661] DEBUG -- : Thread-113406: GET
> /repositories/2/archival_objects?id_set=40951%2C40952%2C40953%2C40954%2C40955%2C40956%2C40957%2C40958%2C40959%2C40960%2C40961%2C40962%2C40963%2C40964%2C40965%2C40966%2C40967%2C40968%2C40969%2C40970%2C40971%2C40972%2C40973%2C40974%2C40975&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classification&resolve%5B%5D=digital_object
> [session: #<Session:0x3883500e @store={:user=>"search_indexer",
> :login_time=>2015-03-02 12:34:08 -0600, :expirable=>false},
> @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
>
> D, [2015-03-02T12:37:27.724000 #1661] DEBUG -- : Thread-113406:
> Post-processed params: {"id_set"=>[40951, 40952, 40953, 40954, 40955,
> 40956, 40957, 40958, 40959, 40960, 40961, 40962, 40963, 40964, 40965,
> 40966, 40967, 40968, 40969, 40970, 40971, 40972, 40973, 40974, 40975],
> "resolve"=>["subjects", "linked_agents", "linked_records",
> "classification", "digital_object"], :repo_id=>2, "modified_since"=>0}
>
> D, [2015-03-02T12:37:27.770000 #1661] DEBUG -- : Thread-111206: Responded
> with [200, {"Content-Type"=>"application/json", "Cache-Control"=>"private,
> must-revalidate, max-age=0", "Content-Length"=>"46709"},
> ["[{\"lock_version\":0,\"position\":5,\"publish\":true,\"ref_id\":\"5c8c5cd60619e372f3bb9fda9f69aae5\",\"component_id\":\"Folder
> 6\",\"title\":\"[u]Advisvory Committee[/u], Dale A. Ulrich, Phsical
> Education Project on Evaluation of Motor Skill Assessment Instrument for
> Use with Handicapped Students\",\"display_string\":\"[u]Advisvory
> Committee[/u], Dale A. Ulrich, Phsical Education P... in 275.0ms
>
> Mar 2, 2015 12:37:27 PM
> org.apache.solr.update.processor.LogUpdateProcessor finish
>
> INFO: [collection1] webapp= path=/update params={}
> {add=[/repositories/2/archival_objects/40901,
> /repositories/2/archival_objects/40902,
> /repositories/2/archival_objects/40903,
> /repositories/2/archival_objects/40904,
> /repositories/2/archival_objects/40905,
> /repositories/2/archival_objects/40906,
> /repositories/2/archival_objects/40907,
> /repositories/2/archival_objects/40908,
> /repositories/2/archival_objects/40909,
> /repositories/2/archival_objects/40910, ... (25 adds)]} 0 7
>
> Indexed 41100 of 41661 archival_object records in repository University
> Archives
>
> D, [2015-03-02T12:37:27.860000 #1661] DEBUG -- : Thread-104680: GET
> /repositories/2/archival_objects?id_set=40976%2C40977%2C40978%2C40979%2C40980%2C40981%2C40982%2C40983%2C40984%2C40985%2C40986%2C40987%2C40988%2C40989%2C40990%2C40991%2C40992%2C40993%2C40994%2C40995%2C40996%2C40997%2C40998%2C40999%2C41000&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classification&resolve%5B%5D=digital_object
> [session: #<Session:0x45bb59d @store={:user=>"search_indexer",
> :login_time=>2015-03-02 12:34:08 -0600, :expirable=>false},
> @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
>
> D, [2015-03-02T12:37:27.860000 #1661] DEBUG -- : Thread-106498: Responded
> with [200, {"Content-Type"=>"application/json", "Cache-Control"=>"private,
> must-revalidate, max-age=0", "Content-Length"=>"42584"},
> ["[{\"lock_version\":0,\"position\":2,\"publish\":true,\"ref_id\":\"92bf78d7ed852be131ce495d923ca07e\",\"component_id\":\"Folder
> 31\",\"title\":\"Problem Sets at the End of Chapters in RLD Wright's
> [u]Understanding Statistics[/u]\",\"display_string\":\"Problem Sets at the
> End of Chapters in RLD Wright's  [u]Understanding
> Statistics[/u]\",\"restrictions_apply\":false,\"created_by\":\... in 245.0ms
>
> D, [2015-03-02T12:37:27.867000 #1661] DEBUG -- : Thread-104680:
> Post-processed params: {"id_set"=>[40976, 40977, 40978, 40979, 40980,
> 40981, 40982, 40983, 40984, 40985, 40986, 40987, 40988, 40989, 40990,
> 40991, 40992, 40993, 40994, 40995, 40996, 40997, 40998, 40999, 41000],
> "resolve"=>["subjects", "linked_agents", "linked_records",
> "classification", "digital_object"], :repo_id=>2, "modified_since"=>0}
>
> Mar 2, 2015 12:37:27 PM
> org.apache.solr.update.processor.LogUpdateProcessor finish
>
> INFO: [collection1] webapp= path=/update params={}
> {add=[/repositories/2/archival_objects/40926,
> /repositories/2/archival_objects/40927,
> /repositories/2/archival_objects/40928,
> /repositories/2/archival_objects/40929,
> /repositories/2/archival_objects/40930,
> /repositories/2/archival_objects/40931,
> /repositories/2/archival_objects/40932,
> /repositories/2/archival_objects/40933,
> /repositories/2/archival_objects/40934,
> /repositories/2/archival_objects/40935, ... (25 adds)]} 0 6
>
> Indexed 41125 of 41661 archival_object records in repository University
> Archives
>
> D, [2015-03-02T12:37:27.938000 #1661] DEBUG -- : Thread-112092: GET
> /repositories/2/archival_objects?id_set=41001%2C41002%2C41003%2C41004%2C41005%2C41006%2C41007%2C41008%2C41009%2C41010%2C41011%2C41012%2C41013%2C41014%2C41015%2C41016%2C41017%2C41018%2C41019%2C41020%2C41021%2C41022%2C41023%2C41024%2C41025&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classification&resolve%5B%5D=digital_object
> [session: #<Session:0x3fc11c13 @store={:user=>"search_indexer",
> :login_time=>2015-03-02 12:34:08 -0600, :expirable=>false},
> @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
>
> D, [2015-03-02T12:37:27.941000 #1661] DEBUG -- : Thread-112092:
> Post-processed params: {"id_set"=>[41001, 41002, 41003, 41004, 41005,
> 41006, 41007, 41008, 41009, 41010, 41011, 41012, 41013, 41014, 41015,
> 41016, 41017, 41018, 41019, 41020, 41021, 41022, 41023, 41024, 41025],
> "resolve"=>["subjects", "linked_agents", "linked_records",
> "classification", "digital_object"], :repo_id=>2, "modified_since"=>0}
>
> D, [2015-03-02T12:37:27.988000 #1661] DEBUG -- : Thread-113406: Responded
> with [200, {"Content-Type"=>"application/json", "Cache-Control"=>"private,
> must-revalidate, max-age=0", "Content-Length"=>"45706"},
> ["[{\"lock_version\":0,\"position\":9,\"publish\":true,\"ref_id\":\"4812d9379e62e7b8a05317eb34b5d871\",\"component_id\":\"Folder
> 10\",\"title\":\"Educational Psychology 507: Summary of Models by Number of
> Estimates and Solution of Regression Equations with Two
> Predictors\",\"display_string\":\"Educational Psychology 507: Summary of
> Models by Number of Estimates and Solution of Regres... in 270.0ms
>
> Mar 2, 2015 12:37:28 PM
> org.apache.solr.update.processor.LogUpdateProcessor finish
>
> INFO: [collection1] webapp= path=/update params={}
> {add=[/repositories/2/archival_objects/40951,
> /repositories/2/archival_objects/40952,
> /repositories/2/archival_objects/40953,
> /repositories/2/archival_objects/40954,
> /repositories/2/archival_objects/40955,
> /repositories/2/archival_objects/40956,
> /repositories/2/archival_objects/40957,
> /repositories/2/archival_objects/40958,
> /repositories/2/archival_objects/40959,
> /repositories/2/archival_objects/40960, ... (25 adds)]} 0 7
>
> Indexed 41150 of 41661 archival_object records in repository University
> Archives
>
> D, [2015-03-02T12:37:28.069000 #1661] DEBUG -- : Thread-106498: GET
> /repositories/2/archival_objects?id_set=41026%2C41027%2C41028%2C41029%2C41030%2C41031%2C41032%2C41033%2C41034%2C41035%2C41036%2C41037%2C41038%2C41039%2C41040%2C41041%2C41042%2C41043%2C41044%2C41045%2C41046%2C41047%2C41048%2C41049%2C41050&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classification&resolve%5B%5D=digital_object
> [session: #<Session:0x568e24f8 @store={:user=>"search_indexer",
> :login_time=>2015-03-02 12:34:08 -0600, :expirable=>false},
> @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
>
> D, [2015-03-02T12:37:28.072000 #1661] DEBUG -- : Thread-106498:
> Post-processed params: {"id_set"=>[41026, 41027, 41028, 41029, 41030,
> 41031, 41032, 41033, 41034, 41035, 41036, 41037, 41038, 41039, 41040,
> 41041, 41042, 41043, 41044, 41045, 41046, 41047, 41048, 41049, 41050],
> "resolve"=>["subjects", "linked_agents", "linked_records",
> "classification", "digital_object"], :repo_id=>2, "modified_since"=>0}
>
> D, [2015-03-02T12:37:28.127000 #1661] DEBUG -- : Thread-104680: Responded
> with [200, {"Content-Type"=>"application/json", "Cache-Control"=>"private,
> must-revalidate, max-age=0", "Content-Length"=>"49421"},
> ["[{\"lock_version\":0,\"position\":34,\"publish\":true,\"ref_id\":\"7c64551408ea76652410341c1ea0c5ba\",\"component_id\":\"Folder
> 35\",\"title\":\"Educational Psychology 507: Sample for Class, Computer
> Printout #7, SAS, Fall\",\"display_string\":\"Educational Psychology 507:
> Sample for Class, Computer Printout #7, SAS, Fall,
> 1987\",\"restrictions_apply\":false,\"created_by\":\"admin\... in 272.0ms
>
> D, [2015-03-02T12:37:28.149000 #1661] DEBUG -- : Thread-112092: Responded
> with [200, {"Content-Type"=>"application/json", "Cache-Control"=>"private,
> must-revalidate, max-age=0", "Content-Length"=>"46655"},
> ["[{\"lock_version\":0,\"position\":24,\"publish\":true,\"ref_id\":\"2068ac71b12ea2dbb48b35edf6735275\",\"component_id\":\"Folder
> 25\",\"title\":\"Guidance 507: Computer Problem 3, SAS, One-Way ANOVA with
> Dummy Coding Models 2 and 3, Use Intercept Models 11 and 12, No Intercept,
> Spring\",\"display_string\":\"Guidance 507: Computer Problem 3, SAS,
> One-Way ANOVA with Dummy Coding Model... in 215.0ms
>
> Mar 2, 2015 12:37:28 PM org.apache.solr.update.DirectUpdateHandler2 commit
>
> INFO: start
> commit{flags=0,_version_=0,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
>
> Mar 2, 2015 12:37:28 PM
> org.apache.solr.update.processor.LogUpdateProcessor finish
>
> INFO: [collection1] webapp= path=/update params={}
> {deleteByQuery=primary_type:tree_view AND
> root_uri:("/repositories/2/resources/276")} 0 0
>
> Mar 2, 2015 12:37:28 PM
> org.apache.solr.update.processor.LogUpdateProcessor finish
>
> INFO: [collection1] webapp= path=/update params={}
> {add=[/repositories/2/archival_objects/40976,
> /repositories/2/archival_objects/40977,
> /repositories/2/archival_objects/40978,
> /repositories/2/archival_objects/40979,
> /repositories/2/archival_objects/40980,
> /repositories/2/archival_objects/40981,
> /repositories/2/archival_objects/40982,
> /repositories/2/archival_objects/40983,
> /repositories/2/archival_objects/40984,
> /repositories/2/archival_objects/40985, ... (25 adds)]} 0 30
>
> Indexed 41175 of 41661 archival_object records in repository University
> Archives
>
> D, [2015-03-02T12:37:28.237000 #1661] DEBUG -- : Thread-111206: GET
> /repositories/2/resources/276/tree [session: #<Session:0x4f648888
> @store={:user=>"search_indexer", :login_time=>2015-03-02 12:34:08 -0600,
> :expirable=>false},
> @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
>
> D, [2015-03-02T12:37:28.238000 #1661] DEBUG -- : Thread-113406: GET
> /repositories/2/archival_objects?id_set=41051%2C41052%2C41053%2C41054%2C41055%2C41056%2C41057%2C41058%2C41059%2C41060%2C41061%2C41062%2C41063%2C41064%2C41065%2C41066%2C41067%2C41068%2C41069%2C41070%2C41071%2C41072%2C41073%2C41074%2C41075&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classification&resolve%5B%5D=digital_object
> [session: #<Session:0x5619daba @store={:user=>"search_indexer",
> :login_time=>2015-03-02 12:34:08 -0600, :expirable=>false},
> @id="0e5b404f4d9d2df00dc325899fe7360a39d3b95afd1fb0c42a671c5a028a06fd">]
>
> D, [2015-03-02T12:37:28.243000 #1661] DEBUG -- : Thread-113406:
> Post-processed params: {"id_set"=>[41051, 41052, 41053, 41054, 41055,
> 41056, 41057, 41058, 41059, 41060, 41061, 41062, 41063, 41064, 41065,
> 41066, 41067, 41068, 41069, 41070, 41071, 41072, 41073, 41074, 41075],
> "resolve"=>["subjects", "linked_agents", "linked_records",
> "classification", "digital_object"], :repo_id=>2, "modified_since"=>0}
>
> D, [2015-03-02T12:37:28.245000 #1661] DEBUG -- : Thread-111206:
> Post-processed params: {:id=>276, :repo_id=>2}
>
>
>   _______________________________________________
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group at lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>
>
>
> _______________________________________________
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group at lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>
>

-- 
Nathan Stevens
Programmer/Analyst
Digital Library Technology Services
New York University

1212-998-2653
ns96 at nyu.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20150304/18420da9/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ArchonQuoteEncoding.png
Type: image/png
Size: 26831 bytes
Desc: not available
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20150304/18420da9/attachment.png>