[Archivesspace_Users_Group] out of memory errors during reindexing, 1.5 RC3

Thu Jul 21 20:53:38 EDT 2016

Just wanted to confirm Mark Cooper's note from earlier in the thread.
Deleting and rebuilding the index is recommended here
<https://groups.google.com/forum/#!topic/archivesspace/uqDXPUv-yCo>.

As a result, I ran a couple of test migrations to 1.5 yesterday (~275,000
records) with no issues.

On Thu, Jul 21, 2016 at 8:19 PM, Kevin Clair <Kevin.Clair at du.edu> wrote:

> To contribute another data point, this happens with ours as well, but we
> only have 130,000 archival objects in our database so typically it will
> finish indexing those but then run out of memory when it moves on to the
> digital objects. Upon restarting AS once the index will finish normally.
> Our Solr index has a little over 200,000 documents, which sounds perhaps
> like it’s the lower limit at which 1.5 starts to have memory issues with
> the initial index. I haven’t been copying over the /data directory at all
> when doing test upgrades.  -k
>
>
>
> *From: *<archivesspace_users_group-bounces at lyralists.lyrasis.org> on
> behalf of Patrick Galligan <PGalligan at rockarch.org>
> *Reply-To: *Archivesspace Group <
> archivesspace_users_group at lyralists.lyrasis.org>
> *Date: *Thursday, July 21, 2016 at 1:48 PM
> *To: *Archivesspace Group <archivesspace_users_group at lyralists.lyrasis.org
> >
> *Subject: *Re: [Archivesspace_Users_Group] out of memory errors during
> reindexing, 1.5 RC3
>
>
>
> Hi all,
>
>
>
> I’ve just tried to index again, with no luck. Same error that we’re
> running out of memory, despite high memory allocations.
>
>
>
> This is a pretty big blocker for us. We were hoping to migrate to 1.5 in
> early August, but we obviously can’t if the index never finishes
> successfully.
>
>
>
> Patrick Galligan
>
> Rockefeller Archive Center
>
> Assistant Digital Archivist
>
> 914-366-6386
>
>
>
> *From:* archivesspace_users_group-bounces at lyralists.lyrasis.org [mailto:
> archivesspace_users_group-bounces at lyralists.lyrasis.org] *On Behalf Of *Dallas
> Pillen
> *Sent:* Wednesday, July 20, 2016 3:50 PM
> *To:* Archivesspace Users Group
> *Subject:* Re: [Archivesspace_Users_Group] out of memory errors during
> reindexing, 1.5 RC3
>
>
>
> Hi all,
>
>
>
> Just want to jump in to say that we have been experiencing the same issue
> during a test migration to 1.5 RC3.
>
>
>
> For context, we're migrating 2,910 EADs and in the process creating about
> 850,000 archival objects and 43,000 top containers. We have also not used
> the barcoder plugin (though we have added temporary fake barcodes to top
> containers in our EADs).
>
>
>
> Dallas
>
>
>
> On Wed, Jul 20, 2016 at 3:15 PM, Vandermillen, Michael <
> michael_vandermillen at harvard.edu> wrote:
>
> Thanks Noah! Yes, it always happens in the middle of archival objects,
> then starts over. We never ran the barcode tool, so that’s not the issue,
> at least for us. Hopefully we can get more info from Lyrasis.
>
>
>
> Michael
>
>
>
> *From:* archivesspace_users_group-bounces at lyralists.lyrasis.org [mailto:
> archivesspace_users_group-bounces at lyralists.lyrasis.org] *On Behalf Of *Noah
> Huffman
> *Sent:* Wednesday, July 20, 2016 3:08 PM
> *To:* Archivesspace Users Group
> *Subject:* Re: [Archivesspace_Users_Group] out of memory errors during
> reindexing, 1.5 RC3
>
>
>
> Michael,
>
>
>
> I’ve experienced this exact problem when running test migrations to
> 1.5RC3.  Increasing the heap size never helped either.  From examining my
> log file (archivesspace.out), it looks like the re-indexing process runs
> out of memory (typically when indexing archival object records) and then
> just starts indexing at archival object #1 again.  It seems to continue in
> this loop indefinitely.  To successfully migrate to 1.5RC3, I had to stop
> and start the app several times until it finally indexed all the archival
> objects.
>
>
>
> No matter how high I set the heap size, the app seems to jump up to that
> limit shortly after starting it.
>
>
>
> I communicated this problem to Lyrasis and there was some suspicion it was
> caused by running the barcoder plugin over my entire repository.
>
>
>
> FWIW, we’ve also experience out of memory issues in our production
> instance during periodic indexing (not a complete re-index).  It seems like
> we have to restart the app every couple of weeks due to memory issues.
>
>
>
> I’m curious if others have experienced these same memory problems too.
> Thanks for raising the issue.
>
>
>
> -Noah
>
>
>
> ================
>
> Noah Huffman
>
> Archivist for Metadata, Systems, and Digital Records
>
> David M. Rubenstein Rare Book & Manuscript Library
>
> Duke University | 919-660-5982
>
> http://library.duke.edu/rubenstein/
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__library.duke.edu_rubenstein_&d=CwMFAg&c=WO-RGvefibhHBZq3fL85hQ&r=VrQdYlh6WQ9nrFC-sjqrXhpO3c_dIW1HO3Q189A7J80&m=X71gxo3lvFklEbvZRx8LX86CpdM-tILccnang2Hetx4&s=MrGtvBJTB3c-Xfhv3l51heHE0bRimo5uYXOjQCIoU2k&e=>
>
>
>
>
>
> *From:* archivesspace_users_group-bounces at lyralists.lyrasis.org [
> mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org
> <archivesspace_users_group-bounces at lyralists.lyrasis.org>] *On Behalf Of *Vandermillen,
> Michael
> *Sent:* Wednesday, July 20, 2016 2:42 PM
> *To:* Archivesspace Users Group (
> archivesspace_users_group at lyralists.lyrasis.org) <
> archivesspace_users_group at lyralists.lyrasis.org>
> *Subject:* [Archivesspace_Users_Group] out of memory errors during
> reindexing, 1.5 RC3
>
>
>
> Hi,
>
>
>
> As we import more and more finding aids into our archivesspace test
> instance, I get out of memory errors when trying to reindex (by deleting
> indexer_state and solr_index):
>
>
>
> SEVERE: auto commit error...:java.lang.IllegalStateException: this writer
> hit an OutOfMemoryError; cannot commit
>
>
>
> For 1.4x (and 1.5 RC 1), we had aspace heap set at the default (1024m) and
> could successfully reindex, after loading 4000+ finding aids. We have now
> loaded 5000+ finding aids, but even when I increase available heap to 8 gig
> (more doesn’t seem reasonable), I still get out of memory errors during a
> reindex (using 1.5 RC3).
>
>
>
> Has anyone else run into this? Any suggestions?
>
>
>
> Michael
>
>
>
> Michael Vandermillen
>
> Digital Library Software Engineer
>
> Harvard University IT
>
>
>
>
>
>
> _______________________________________________
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group at lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>
>
>
>
>
> --
>
>
> *Dallas Pillen *Assistant Archivist for Metadata and Digital Projects
>
>
>
>   Bentley Historical Library <http://bentley.umich.edu/>
>
>   1150 Beal Avenue
>
>   Ann Arbor, Michigan 48109-2113
>
>   734.647.3559
>
>   Twitter <https://twitter.com/umichBentley> Facebook
> <https://www.facebook.com/bentleyhistoricallibrary>
>
> _______________________________________________
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group at lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20160721/9bc3b779/attachment.html>