[Archivesspace_Users_Group] hELP - No record found after test migration from v1.4.2 to 1.5.0
Maderik, Rachel A
maderikra at vmi.edu
Thu Sep 15 12:03:58 EDT 2016
Yes, sorry I didn't clarify, I ran the tests on the newest version, 1.5.1.
From: archivesspace_users_group-bounces at lyralists.lyrasis.org [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] On Behalf Of Mang Sun
Sent: Thursday, September 15, 2016 11:54 AM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org>
Subject: Re: [Archivesspace_Users_Group] hELP - No record found after test migration from v1.4.2 to 1.5.0
Patrick,
I did test with 1.5.0 and 1.5.1 and ran into the same problem.
Mang
On 9/15/2016 10:39 AM, Galligan, Patrick wrote:
Rachel,
It was a known issue that 1.5.0 had indexing problems. These have generally been fixed in 1.5.1. I'd suggest using that version for your test migration.
Patrick Galligan
Rockefeller Archive Center
Assistant Digital Archivist
914-366-6386
From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] On Behalf Of Maderik, Rachel A
Sent: Thursday, September 15, 2016 11:30 AM
To: 'Archivesspace Users Group'
Subject: Re: [Archivesspace_Users_Group] hELP - No record found after test migration from v1.4.2 to 1.5.0
I was able to replicate this problem using Rice's data. I set up a fresh instance running 1.4.2 using their database dump from that version, and it built the index fine (took 10-15 minutes). When I migrated it to 1.5, the indexer seems to get hung up after indexing the top containers (which finished about 2 hrs ago). Below is an error message that appears right after it finishes the top containers (and continues to appear repeatedly in the log file). It seems to be related to indexing archival objects, though I can't say I understand anything more than that:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~ Indexed 12884 top_container records in 600 seconds ~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Sep 15, 2016 9:33:57 AM org.eclipse.jetty.server.handler.ContextHandler$Context log
INFO: DEBUG: resetting rack response due exception
Failure in periodic indexer worker thread: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
...
<body>
<div id="summary">
<h1>Internal Server Error <span>(500)</span></h1>
<table class="meta">
<tr>
<th>Request Method:</th>
<td>GET</td>
</tr>
<tr>
<th>Request URL:</th<URL:%3c/th>>
<td>http://localhost:8089/repositories/2/archival_objects?id_set=1%2C2%2C3%2C4%2C5%2C6%2C7%2C8%2C9%2C10%2C11%2C12%2C13%2C14%2C15%2C16%2C17%2C18%2C19%2C20%2C21%2C22%2C23%2C24%2C25&resolve%5B%5D=location_profile&resolve%5B%5D=container_profile&resolve%5B%5D=container_locations&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classifications&resolve%5B%5D=digital_object<http:////localhost:8089/repositories/2/archival_objects?id_set=1%2C2%2C3%2C4%2C5%2C6%2C7%2C8%2C9%2C10%2C11%2C12%2C13%2C14%2C15%2C16%2C17%2C18%2C19%2C20%2C21%2C22%2C23%2C24%2C25&resolve%5B%5D=location_profile&resolve%5B%5D=container_profile&resolve%5B%5D=container_locations&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classifications&resolve%5B%5D=digital_object></td>
</tr>
</table>
</div>
<div id="info">
<p>(NoMethodError) undefined method `related_records' for nil:NilClass</p>
</div>
<div id="explanation">
<p>
You're seeing this error because you use <code>JRuby::Rack::ErrorApp::ShowStatus</code>.
</p>
</div>
</body>
-------------------------------
Rachel Maderik
Systems and Technology Librarian
501D Preston Library
Virginia Military Institute
Lexington, VA 24450
540-464-7572
From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] On Behalf Of Mang Sun
Sent: Wednesday, September 14, 2016 3:00 PM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>>
Subject: Re: [Archivesspace_Users_Group] hELP - No record found after test migration from v1.4.2 to 1.5.0
Christine and other AS community members who have successfully migrated AS 1.4.2 to AS 1.5.x,
We are trying to draw attention and seek help from AS development team and other community members who have successfully migrated AS 1.4.2 to AS 1.5.x. If necessary, we are willing to provide our MySQL dump file for you to test.
Test migration from AS 1.4.2 to AS1.5.x really drives us mad. Though we don't quite see any (out of mem) error in the log, we still blindly pump up heap size (Xmx )to 6GB, stack size set o 3MB and rerun rebuild index or the complete migration steps, the problem is still there - Solr seemingly very silently stops writing or committing index files when the number of docs in Solr reaches to about 15,xxx as opposed to 500,xxx indexed docs found in our 1.4.2 instance, though thereafter we still can see the " end_commit_flush" message pop up in the log from time to time .Our test box environments are
1. Redhat 6.8 ,JAVA build 1.7.0_67 . MySQL 5.1.7 and 8GB memory
2.. Redhat 7.2 Virtual Machine,NYSQL 5.5.45, java 1.80_65,and 4GB.
We also can confirm database side migration is largely done with new tables and fields added, existing records converted and top container records added. The problem is sticking to rebuilding the whole index - Solr stops writing and committing index prematurely after the number of docs grows to a small number(12xxx-15xxx) though commit flush can still be seen thereafter .
Now we have exhausted ideas and means in figuring out why this "no record found " problem that seemingly only happens to us.
Other experiment we did is export EAD file of one resource record from our production and load it into a 1.4.2 test instance and then upgrade this test instance to 1.5.x. this time the resource record can be indexed and appear(some issue with the Name ) in the test 1.5.x instance.
I think there is some fundamental but very sneaky issue with 1.5.x indexing (rebuild index). The challenge is that no explicit error message that could be easily found in the log that can give us some hint and therefore this is a nightmare given the long waiting time before you can confirm a successful indexing or not. May be a standalone tool allowing user to manually rebuild index could be very helpful and straightforward, and less arcane.
As mentioned earlier, records have been converted and migrated but not indexed, what else we tried is use direct route path (can be inferred from our production) to pull records out and update them by toggling through Suppress/Unsuppress or Publisesh/Unpublished , then we can get those updated records indexed. in the following diagram, we can see 3 resources, 2 agent records and some other record types highlighted in yellow appear in the types field, as opposed to the types (not highlighted) originally indexed and generated by the migration process. Also in the same diagram, the number of indexed docs is too small (we can see about half million docs in our 1.4.2 production instance).Also we found a side issue with agent record, no matter what you set for Publish, Solr only sees it unpublished.But At least, this experiment can confirm there is not obvious problem with dynamic indexing.
If AS developers or other AS users want to reproduce the issue or give a try for a solution, we can provide our v1.4.2 database dump or the v1.5.x database dump.
Thank you very much.
Mang Sun
Rice U.
[cid:image001.png at 01D20F49.3CA5BE10]
On 9/9/2016 11:55 AM, Christine Di Bella wrote:
Mang,
How long ago did you start the reindex and how big is your database? The v1.5.1 upgrade improved indexing speed, especially for large databases, but the indexing does still take some time. People on this list can probably give you a sense of how long it took them for different sizes of databases. It's probably been most common to run it overnight or over a weekend, when use of the system is low.
(Some people were large databases were experiencing problems with indexing taking an excessive amount of time or not completing at all, which was one of the reasons for the quick release of v1.5.1.)
Christine
Christine Di Bella
Community Outreach Manager
christine.dibella at lyrasis.org<mailto:christine.dibella at lyrasis.org>
800.999.8558 x2905
678-235-2905
cdibella13 (Skype)
[cid:image003.png at 01CE734E.FD759D30]
From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] On Behalf Of Mang Sun
Sent: Friday, September 9, 2016 12:31 PM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org><mailto:archivesspace_users_group at lyralists.lyrasis.org>
Subject: Re: [Archivesspace_Users_Group] No record found after test migration from v1.4.2 to 1.5.0
I even didn't copy the data folder of the v1.4.2 into v.1.5.0. but even after I follow the recommended way by copying the data folder and removing those two subfolders mentioned by Joshua the problem is still there.Peeping into Solr schema, I found field "types" only have the following four types but not resource and digital_object,subject and agent . The missing types may explain why we find no records under every record type tab. Puzzled~ Mang
* 12608
o top_container<http://archives-test.library.rice.edu:8090/#/collection1/query?q=types:%22top_container%22>
* 1,169
o tree_view<http://archives-test.library.rice.edu:8090/#/collection1/query?q=types:%22tree_view%22>
* 75
o archival_object<http://archives-test.library.rice.edu:8090/#/collection1/query?q=types:%22archival_object%22>
* 1
o repository<http://archives-test.library.rice.edu:8090/#/collection1/query?q=types:%22repository%22>
On 9/9/2016 8:52 AM, Sally Vermaaten wrote:
Hi Mang Sun,
I agree with Joshua - this sounds to me like an indexing problem to me. In the course of upgrading to 1.5.0 did you delete the data/solr_index/index directory and all files in the data/indexer_state directory? This is one of the recommended steps in the instructions for upgrading to v.1.5.0<https://github.com/archivesspace/archivesspace/blob/master/UPGRADING_1.5.0.md>. We accidentally missed this step once in a running a test upgrade and definitely experienced some indexing problems though I don't remember the details. As it was a test upgrade (and it sounds like that's what you're doing as well) we just started afresh and ran the upgrade process over again. When we included that step, things were fine.
On a related note - there's a 1.5.1 release of ArchivesSpace<https://github.com/archivesspace/archivesspace/releases/tag/v1.5.1> that includes several fixes for bugs that were found in the weeks after 1.5.0 was released. If are looking to update to a 1.5.x version, you may want to consider upgrading to 1.5.1 one rather than 1.5.0.
Good luck,
Sally
On Fri, Sep 9, 2016 at 9:50 AM, Joshua D. Shaw <Joshua.D.Shaw at dartmouth.edu<mailto:Joshua.D.Shaw at dartmouth.edu>> wrote:
I believe the suggestion for migration to 1.5+ was to delete the existing index and indexer states which will trigger a complete reindex.
Best,
Joshua
On Sep 9, 2016, at 8:20 AM, "Mang Sun" <mang.sun at rice.edu<mailto:mang.sun at rice.edu>> wrote:
> i
>
> After we migrate our v.1.4.2 test instance to v1.5.0, we can't find any records but just no record in Staff interface and public interface.
>
> But if we open a migrated record (saycollection record) using the direct route path to that record and resave it, this record but only this one starts to show up under Collections when browsing collections. This problem does not come with any new record created within v.1.5.0. Any suggestion?
>
> Thank you.
>
> Mang Sun
>
> Rice U.
>
>
>
> _______________________________________________
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
--
Sally Vermaaten
Project Manager, Archival Systems
New York University Libraries
1-212-992-6259
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20160915/6d5ddd63/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 41892 bytes
Desc: image001.png
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20160915/6d5ddd63/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 7645 bytes
Desc: image002.png
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20160915/6d5ddd63/attachment-0001.png>
More information about the Archivesspace_Users_Group
mailing list