[Archivesspace_Users_Group] hELP - No record found after test migration from v1.4.2 to 1.5.0

Mang Sun mang.sun at rice.edu
Thu Sep 15 11:53:55 EDT 2016


Patrick,

I did test with 1.5.0 and 1.5.1 and ran into  the same problem.


Mang



On 9/15/2016 10:39 AM, Galligan, Patrick wrote:
>
> Rachel,
>
> It was a known issue that 1.5.0 had indexing problems. These have 
> generally been fixed in 1.5.1. I’d suggest using that version for your 
> test migration.
>
> Patrick Galligan
>
> Rockefeller Archive Center
>
> Assistant Digital Archivist
>
> 914-366-6386
>
> *From:*archivesspace_users_group-bounces at lyralists.lyrasis.org 
> [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] *On 
> Behalf Of *Maderik, Rachel A
> *Sent:* Thursday, September 15, 2016 11:30 AM
> *To:* 'Archivesspace Users Group'
> *Subject:* Re: [Archivesspace_Users_Group] hELP - No record found 
> after test migration from v1.4.2 to 1.5.0
>
> I was able to replicate this problem using Rice’s data. I set up a 
> fresh instance running 1.4.2 using their database dump from that 
> version, and it built the index fine (took 10-15 minutes). When I 
> migrated it to 1.5, the indexer seems to get hung up after indexing 
> the top containers (which finished about 2 hrs ago). Below is an error 
> message that appears right after it finishes the top containers (and 
> continues to appear repeatedly in the log file). It seems to be 
> related to indexing archival objects, though I can’t say I understand 
> anything more than that:
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> ~~~ Indexed 12884 top_container records in 600 seconds ~~~
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> Sep 15, 2016 9:33:57 AM 
> org.eclipse.jetty.server.handler.ContextHandler$Context log
>
> INFO: DEBUG: resetting rack response due exception
>
> Failure in periodic indexer worker thread: <!DOCTYPE HTML PUBLIC 
> "-//W3C//DTD HTML 4.01 Transitional//EN" 
> "http://www.w3.org/TR/html4/loose.dtd">
>
>>
> <body>
>
> <div id="summary">
>
> <h1>Internal Server Error <span>(500)</span></h1>
>
> <table class="meta">
>
> <tr>
>
> <th>Request Method:</th>
>
> <td>GET</td>
>
> </tr>
>
> <tr>
>
> <th>Request URL:</th <URL:%3c/th>>
>
> <td>http:&#x2F;&#x2F;localhost:8089&#x2F;repositories&#x2F;2&#x2F;archival_objects?id_set=1%2C2%2C3%2C4%2C5%2C6%2C7%2C8%2C9%2C10%2C11%2C12%2C13%2C14%2C15%2C16%2C17%2C18%2C19%2C20%2C21%2C22%2C23%2C24%2C25&resolve%5B%5D=location_profile&resolve%5B%5D=container_profile&resolve%5B%5D=container_locations&resolve%5B%5D=subjects&resolve%5B%5D=linked_agents&resolve%5B%5D=linked_records&resolve%5B%5D=classifications&resolve%5B%5D=digital_object</td>
>
> </tr>
>
> </table>
>
> </div>
>
> <div id="info">
>
> <p>(NoMethodError) undefined method `related_records&#x27; for 
> nil:NilClass</p>
>
> </div>
>
> <div id="explanation">
>
> <p>
>
> You're seeing this error because you use 
> <code>JRuby::Rack::ErrorApp::ShowStatus</code>.
>
> </p>
>
> </div>
>
> </body>
>
> -------------------------------
>
> Rachel Maderik
>
> Systems and Technology Librarian
>
> 501D Preston Library
>
> Virginia Military Institute
>
> Lexington, VA 24450
>
> 540-464-7572
>
> *From:*archivesspace_users_group-bounces at lyralists.lyrasis.org 
> <mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> 
> [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] *On 
> Behalf Of *Mang Sun
> *Sent:* Wednesday, September 14, 2016 3:00 PM
> *To:* Archivesspace Users Group 
> <archivesspace_users_group at lyralists.lyrasis.org 
> <mailto:archivesspace_users_group at lyralists.lyrasis.org>>
> *Subject:* Re: [Archivesspace_Users_Group] hELP - No record found 
> after test migration from v1.4.2 to 1.5.0
>
> Christine and other AS community members who have successfully 
> migrated AS 1.4.2 to AS 1.5.x,
>
> We are trying to  draw attention and seek help from AS development 
> team and other community members who have successfully migrated AS 
> 1.4.2 to AS 1.5.x. If necessary, we are willing to  provide our MySQL 
> dump file for you to test.
>
> Test migration from AS 1.4.2 to AS1.5.x really drives us mad. Though 
> we don't quite see any (out of mem) error in the log, we still blindly 
> pump up heap size (Xmx )to 6GB, stack size set o 3MB and rerun rebuild 
> index or the complete migration steps, the problem is still there - 
> Solr seemingly very silently stops writing or committing index files 
> when the number of docs in Solr reaches to  about 15,xxx as opposed to 
> 500,xxx indexed docs found  in our 1.4.2 instance, though thereafter 
> we still can see the " end_commit_flush" message pop up in the log  
> from time to time .Our test box environments are
>
> 1. Redhat 6.8 ,JAVA build 1.7.0_67 . MySQL 5.1.7 and 8GB memory
>  2.. Redhat 7.2 Virtual Machine,NYSQL 5.5.45, java 1.80_65,and 4GB.
>  We also can  confirm database side migration  is largely done with 
> new tables and fields added, existing records converted and top 
> container records added. The problem is sticking to rebuilding the 
> whole index - Solr  stops writing and committing  index prematurely 
> after  the number of docs  grows to a small number(12xxx-15xxx) though 
> commit  flush can still be seen thereafter .
>
> Now we have exhausted ideas and means in figuring out why this "no 
> record found " problem that seemingly only  happens to us.
>
> Other experiment we did is export EAD file of one resource record from 
> our production and load it into a 1.4.2 test instance and then upgrade 
> this test instance to 1.5.x. this time the resource record can be 
> indexed and appear(some issue with the Name )  in the test 1.5.x instance.
>
> I think there is some fundamental but very sneaky issue with 1.5.x 
> indexing (rebuild index). The challenge is that no explicit error 
> message that could be easily found in the log that can give us some 
> hint and therefore this is a nightmare given the long waiting time 
> before you can confirm a successful indexing or not.  May be a 
> standalone tool allowing user to manually rebuild index could be very 
> helpful and straightforward, and less arcane.
>
> As mentioned earlier, records have been converted and migrated but not 
> indexed, what else we tried is use direct route path (can be inferred 
> from our production) to pull records out and update them by toggling 
> through   Suppress/Unsuppress or Publisesh/Unpublished , then we can 
> get those updated records indexed. in the following diagram, we can 
> see 3 resources, 2 agent records and some other record types 
> highlighted in yellow appear in the types field, as opposed to the 
> types (not highlighted) originally indexed and generated by the 
> migration process. Also in the same diagram, the number of indexed 
> docs is too small (we can see about half million docs in our 1.4.2 
> production instance).Also we found a side  issue with agent record, no 
> matter what you set for Publish, Solr only sees it unpublished.But   
> At least, this experiment can confirm there is not obvious problem 
> with dynamic indexing.
>
> If AS developers or other AS users want to reproduce the issue or give 
> a try for a solution, we can provide our v1.4.2 database dump or the 
> v1.5.x database dump.
>
> Thank you very much.
>
> Mang Sun
>
> Rice U.
>
> On 9/9/2016 11:55 AM, Christine Di Bella wrote:
>
>     Mang,
>
>     How long ago did you start the reindex and how big is your
>     database? The v1.5.1 upgrade improved indexing speed, especially
>     for large databases, but the indexing does still take some time.
>     People on this list can probably give you a sense of how long it
>     took them for different sizes of databases. It’s probably been
>     most common to run it overnight or over a weekend, when use of the
>     system is low.
>
>     (Some people were large databases were experiencing problems with
>     indexing taking an excessive amount of time or not completing at
>     all, which was one of the reasons for the quick release of v1.5.1.)
>
>     Christine
>
>     Christine Di Bella
>
>     Community Outreach Manager
>
>     christine.dibella at lyrasis.org <mailto:christine.dibella at lyrasis.org>
>
>     800.999.8558 x2905
>
>     678-235-2905
>
>     cdibella13 (Skype)
>
>     cid:image003.png at 01CE734E.FD759D30
>
>     *From:*archivesspace_users_group-bounces at lyralists.lyrasis.org
>     <mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>
>     [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org]
>     *On Behalf Of *Mang Sun
>     *Sent:* Friday, September 9, 2016 12:31 PM
>     *To:* Archivesspace Users Group
>     <archivesspace_users_group at lyralists.lyrasis.org>
>     <mailto:archivesspace_users_group at lyralists.lyrasis.org>
>     *Subject:* Re: [Archivesspace_Users_Group] No record found after
>     test migration from v1.4.2 to 1.5.0
>
>
>     I even  didn't copy the data folder of the v1.4.2 into v.1.5.0.
>     but even after I follow the recommended way by copying the data
>     folder and removing those two subfolders mentioned by Joshua the
>     problem is still there.Peeping into Solr schema, I found field
>     "types" only have  the following four types but not resource and
>     digital_object,subject and agent . The missing types may explain
>     why we find no records under every record type tab. Puzzled~ Mang
>
>     ·12608
>
>     otop_container
>     <http://archives-test.library.rice.edu:8090/#/collection1/query?q=types:%22top_container%22>
>
>     ·1,169
>
>     otree_view
>     <http://archives-test.library.rice.edu:8090/#/collection1/query?q=types:%22tree_view%22>
>
>     ·75
>
>     oarchival_object
>     <http://archives-test.library.rice.edu:8090/#/collection1/query?q=types:%22archival_object%22>
>
>     ·1
>
>     orepository
>     <http://archives-test.library.rice.edu:8090/#/collection1/query?q=types:%22repository%22>
>
>     On 9/9/2016 8:52 AM, Sally Vermaaten wrote:
>
>         Hi Mang Sun,
>
>         I agree with Joshua - this sounds to me like an indexing
>         problem to me. In the course of upgrading to 1.5.0 did you
>         delete the data/solr_index/index directory and all files in
>         the data/indexer_state directory? This is one of the
>         recommended steps in the instructions for upgrading to v.1.5.0
>         <https://github.com/archivesspace/archivesspace/blob/master/UPGRADING_1.5.0.md>.
>         We accidentally missed this step once in a running a test
>         upgrade and definitely experienced some indexing problems
>         though I don't remember the details. As it was a test upgrade
>         (and it sounds like that's what you're doing as well) we just
>         started afresh and ran the upgrade process over again. When we
>         included that step, things were fine.
>
>         On a related note - there's a 1.5.1 release of ArchivesSpace
>         <https://github.com/archivesspace/archivesspace/releases/tag/v1.5.1> that
>         includes several fixes for bugs that were found in the weeks
>         after 1.5.0 was released. If are looking to update to a 1.5.x
>         version, you may want to consider upgrading to 1.5.1 one
>         rather than 1.5.0.
>
>         Good luck,
>
>         Sally
>
>         On Fri, Sep 9, 2016 at 9:50 AM, Joshua D. Shaw
>         <Joshua.D.Shaw at dartmouth.edu
>         <mailto:Joshua.D.Shaw at dartmouth.edu>> wrote:
>
>             I believe the suggestion for migration to 1.5+ was to
>             delete the existing index and indexer states which will
>             trigger a complete reindex.
>
>             Best,
>             Joshua
>
>
>
>
>
>             On Sep 9, 2016, at 8:20 AM, "Mang Sun" <mang.sun at rice.edu
>             <mailto:mang.sun at rice.edu>> wrote:
>
>             > i
>             >
>             > After we migrate our v.1.4.2 test instance to v1.5.0, we
>             can't find any records but just no record in Staff
>             interface and public interface.
>             >
>             > But if we open a migrated record (saycollection record)
>             using the direct route path to that record and resave it,
>             this record but only this one starts to show up under
>             Collections when browsing collections. This problem does
>             not come with any new record created within v.1.5.0. Any
>             suggestion?
>             >
>             > Thank you.
>             >
>             > Mang Sun
>             >
>             > Rice U.
>             >
>             >
>             >
>             > _______________________________________________
>             > Archivesspace_Users_Group mailing list
>             > Archivesspace_Users_Group at lyralists.lyrasis.org
>             <mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
>             >
>             http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>             _______________________________________________
>             Archivesspace_Users_Group mailing list
>             Archivesspace_Users_Group at lyralists.lyrasis.org
>             <mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
>             http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>
>
>
>         -- 
>
>         Sally Vermaaten
>         Project Manager, Archival Systems
>         New York University Libraries
>
>         1-212-992-6259
>
>
>
>
>         _______________________________________________
>
>         Archivesspace_Users_Group mailing list
>
>         Archivesspace_Users_Group at lyralists.lyrasis.org
>         <mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
>
>         http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>
>
>
>     _______________________________________________
>
>     Archivesspace_Users_Group mailing list
>
>     Archivesspace_Users_Group at lyralists.lyrasis.org
>     <mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
>
>     http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>
>
>
> _______________________________________________
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group at lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20160915/0cddb1cc/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 41892 bytes
Desc: not available
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20160915/0cddb1cc/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 7645 bytes
Desc: not available
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20160915/0cddb1cc/attachment-0001.png>


More information about the Archivesspace_Users_Group mailing list