From kbreneman at ubalt.edu Thu Jan 5 20:32:25 2023 From: kbreneman at ubalt.edu (Kyle Breneman) Date: Fri, 6 Jan 2023 01:32:25 +0000 Subject: [Archivesspace_Users_Group] analytics? In-Reply-To: References: Message-ID: We're also using Google Analytics with ArchivesSpace. I'm attaching a PDF of our custom dashboard (in UA) which shows events tracking views of 1) digital object records, 2) digital objects, and 3) resource records. You can also see user search terms in this dashboard. (Unfortunately, GA4 does not currently support adding the custom query parameter used by ArchivesSpace, so no more search term tracking with GA4!) Kyle Breneman Integrated Digital Services Librarian The University of Baltimore kbreneman at ubalt.edu I believe in freedom of thought and freedom of speech. Do you? From: archivesspace_users_group-bounces at lyralists.lyrasis.org On Behalf Of Joshua D. Shaw Sent: Tuesday, December 20, 2022 12:42 PM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] analytics? [EXTERNAL EMAIL: This message originated from a non-UBalt email system. Hover over any links before clicking and use caution when opening attachments.] HI Jennifer Yep. You need a plugin to do it, but its a super simple plugin. Here's what we use at Dartmouth. https://github.com/dartmouth-dltg/aspace_analytics as an example. Joshua ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org > on behalf of Sharp, Jennifer > Sent: Tuesday, December 20, 2022 12:11 PM To: Archivesspace Users Group > Subject: [Archivesspace_Users_Group] analytics? Hi, Does anyone use Google Analytics to count how many times people look at your finding aids (using the PUI)? Is this even possible? Thanks, Jennifer --- Jennifer Sharp, MSI Archivist, Hartford History Center Hartford Public Library jsharp at hplct.org o: 860.695.6332 c: 860.929.6915 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Analytics ArchivesSpace - Master Omnibus 20220901-20221130.pdf Type: application/pdf Size: 168953 bytes Desc: Analytics ArchivesSpace - Master Omnibus 20220901-20221130.pdf URL: From laurie.arp at lyrasis.org Fri Jan 6 08:32:25 2023 From: laurie.arp at lyrasis.org (Laurie Arp) Date: Fri, 6 Jan 2023 13:32:25 +0000 Subject: [Archivesspace_Users_Group] =?windows-1252?q?It_Really_Does_Take_?= =?windows-1252?q?a_Village_=96_Learn_About_It_Takes_a_Village_=26_How_Sam?= =?windows-1252?q?vera_and_Fedora_Are_Using_It?= Message-ID: Greetings, Happy new year! We are excited to open registration to webinars highlighting the use of the It Takes a Village (ITAV) OSS Sustainability Guidebook and Toolkit. We are very pleased to highlight members of the community presenting their experiences. Session Description Long-term sustainability planning is something that poses a unique challenge for many open-source programs. These programs are often funded and supported in ways that do not give appropriate consideration to the long-term sustainability of the applications and their communities. The It Takes a Village (ITAV) framework was created to give OSS communities, like Samvera and Fedora, the tools and information needed to begin the planning process for their futures. Through a series of guided activities, ITAV presents program participants with the opportunity to look both inward and outward to evaluate what is working and what is not and how to take actionable steps forward. Speakers This session will outline the ITAV framework and explain the process for using the toolkit activities. Megan Forbes will provide background and process info. Panelists include Heather Greer Klein, Community Manager at Samvera and Arran Griffith, Program Manager at Fedora. Heather and Arran will share their experiences participating in ITAV and showcase activities they have used. They will highlight and share the lessons learned while undergoing the process for each of their own program?s sustainability planning. Dates/Times You can attend either session: * 2/16/23 at 7 am pacific/8 am mountain/9 am central/10 am eastern time * 2/23/23 at 11 am pacific/noon mountain/1 pm central/2 pm eastern time Registration You can easily register for these free sessions here. Feel free to share this announcement with your networks. Please do not hesitate to let me know if you have any questions. Best wishes, Laurie Gemmill Arp Director, DuraSpace Community Supported Programs laurie.arp at lyrasis.org 800.999.8558 x 2908 [Logo Description automatically generated with low confidence] Pronouns: she/her/hers -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 32028 bytes Desc: image001.png URL: From andrew-holland at uiowa.edu Fri Jan 6 09:49:24 2023 From: andrew-holland at uiowa.edu (Holland, Andrew S) Date: Fri, 6 Jan 2023 14:49:24 +0000 Subject: [Archivesspace_Users_Group] [External] Re: Preferred Citation Notes In-Reply-To: References: Message-ID: Thank you for the response, Blake. This section of the docs explains kind of how preferred citation is used, but my issue is that the citation doesn?t display to the public. We have information in the preferred citation field but you can?t see that unless you?re viewing the resource in the admin interface. https://aspace.lib.uiowa.edu/repositories/4/resources/2762 Abstract, Arrangement, Scope, etc. all display under their own header, but there is no preferred citation section even though we have that filled out. I can?t figure out why that wouldn?t display. -Andrew From: Blake Carver Sent: Thursday, December 15, 2022 11:48 AM To: Archivesspace Users Group Subject: [External] Re: [Archivesspace_Users_Group] Preferred Citation Notes I think the docs explain it here? https://archivesspace.atlassian.net/wiki/spaces/ADC/pages/103209762/Public+Interface+Documentation+for+staff#PublicInterfaceDocumentation(forstaff)-CitationactionbuttonandthePreferredCitationnote From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Holland, Andrew S Sent: Thursday, December 15, 2022 8:25 AM To: Archivesspace Users Group Subject: [Archivesspace_Users_Group] Preferred Citation Notes Hello all, I can seem to figure out how to get our notes put in to the Preferred Citation field to show up on the public site. Can anyone help me? Thanks, Andrew Holland University of Iowa Libraries -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 227A033FD7374F6BBA8449C0D573662F.png Type: image/png Size: 159 bytes Desc: 227A033FD7374F6BBA8449C0D573662F.png URL: From blake.carver at lyrasis.org Fri Jan 6 09:58:06 2023 From: blake.carver at lyrasis.org (Blake Carver) Date: Fri, 6 Jan 2023 14:58:06 +0000 Subject: [Archivesspace_Users_Group] [External] Re: Preferred Citation Notes In-Reply-To: References: Message-ID: I think it just shows up in the button thingy there on the top? There is a JIRA in about notes in general here https://archivesspace.atlassian.net/browse/ANW-968 And there's a comment there that calls out the preferred cite as well, though I'm not quite sure how to read it "The default would also include all notes expect ?preferred citation?." I THINK that means all notes but NOT preferred cite? ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Holland, Andrew S Sent: Friday, January 6, 2023 9:49 AM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] [External] Re: Preferred Citation Notes Thank you for the response, Blake. This section of the docs explains kind of how preferred citation is used, but my issue is that the citation doesn?t display to the public. We have information in the preferred citation field but you can?t see that unless you?re viewing the resource in the admin interface. https://aspace.lib.uiowa.edu/repositories/4/resources/2762 Abstract, Arrangement, Scope, etc. all display under their own header, but there is no preferred citation section even though we have that filled out. I can?t figure out why that wouldn?t display. -Andrew From: Blake Carver Sent: Thursday, December 15, 2022 11:48 AM To: Archivesspace Users Group Subject: [External] Re: [Archivesspace_Users_Group] Preferred Citation Notes I think the docs explain it here? https://archivesspace.atlassian.net/wiki/spaces/ADC/pages/103209762/Public+Interface+Documentation+for+staff#PublicInterfaceDocumentation(forstaff)-CitationactionbuttonandthePreferredCitationnote From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Holland, Andrew S Sent: Thursday, December 15, 2022 8:25 AM To: Archivesspace Users Group Subject: [Archivesspace_Users_Group] Preferred Citation Notes Hello all, I can seem to figure out how to get our notes put in to the Preferred Citation field to show up on the public site. Can anyone help me? Thanks, Andrew Holland University of Iowa Libraries -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 227A033FD7374F6BBA8449C0D573662F.png Type: image/png Size: 159 bytes Desc: 227A033FD7374F6BBA8449C0D573662F.png URL: From saue at d.umn.edu Fri Jan 6 10:01:16 2023 From: saue at d.umn.edu (Shana Aue) Date: Fri, 6 Jan 2023 09:01:16 -0600 Subject: [Archivesspace_Users_Group] [External] Re: Preferred Citation Notes In-Reply-To: References: Message-ID: That's how ours is -- the preferred citation only appears when you click the "Citation" button at the top of the page. It doesn't have its own heading in the main part of the page. On Fri, Jan 6, 2023 at 8:58 AM Blake Carver wrote: > I think it just shows up in the button thingy there on the top? > > There is a JIRA in about notes in general here > https://archivesspace.atlassian.net/browse/ANW-968 > And there's a comment there that calls out the preferred cite as well, > though I'm not quite sure how to read it > > "The default would also include all notes expect ?preferred citation?." > > I THINK that means all notes but NOT preferred cite? > ------------------------------ > *From:* archivesspace_users_group-bounces at lyralists.lyrasis.org < > archivesspace_users_group-bounces at lyralists.lyrasis.org> on behalf of > Holland, Andrew S > *Sent:* Friday, January 6, 2023 9:49 AM > *To:* Archivesspace Users Group < > archivesspace_users_group at lyralists.lyrasis.org> > *Subject:* Re: [Archivesspace_Users_Group] [External] Re: Preferred > Citation Notes > > > Thank you for the response, Blake. This section of the docs explains kind > of how preferred citation is used, but my issue is that the citation > doesn?t display to the public. We have information in the preferred > citation field but you can?t see that unless you?re viewing the resource in > the admin interface. > > > > https://aspace.lib.uiowa.edu/repositories/4/resources/2762 > > > > Abstract, Arrangement, Scope, etc. all display under their own header, but > there is no preferred citation section even though we have that filled out. > I can?t figure out why that wouldn?t display. -Andrew > > > > > > > > *From: *Blake Carver > *Sent: *Thursday, December 15, 2022 11:48 AM > *To: *Archivesspace Users Group > > *Subject: *[External] Re: [Archivesspace_Users_Group] Preferred Citation > Notes > > > > I think the docs explain it here? > > > > > https://archivesspace.atlassian.net/wiki/spaces/ADC/pages/103209762/Public+Interface+Documentation+for+staff#PublicInterfaceDocumentation(forstaff)-CitationactionbuttonandthePreferredCitationnote > > > > *From:* archivesspace_users_group-bounces at lyralists.lyrasis.org < > archivesspace_users_group-bounces at lyralists.lyrasis.org> on behalf of > Holland, Andrew S > *Sent:* Thursday, December 15, 2022 8:25 AM > *To:* Archivesspace Users Group < > archivesspace_users_group at lyralists.lyrasis.org> > *Subject:* [Archivesspace_Users_Group] Preferred Citation Notes > > > > Hello all, > > > > I can seem to figure out how to get our notes put in to the Preferred > Citation field to show up on the public site. Can anyone help me? > > > > Thanks, > > > > Andrew Holland > > University of Iowa Libraries > > > > > > > > > _______________________________________________ > Archivesspace_Users_Group mailing list > Archivesspace_Users_Group at lyralists.lyrasis.org > http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group > -- Shana Aue Archives Assistant | Kathryn A. Martin Library Archives and Special Collections https://lib.d.umn.edu/research-collections/archives University of Minnesota Duluth | d.umn.edu saue at d.umn.edu | 218-726-7314 Pronouns: she, her, hers *We collectively acknowledge that the University of Minnesota Duluth is located on the traditional, ancestral, and contemporary lands of Indigenous people. The University resides on land that was cared for and called home by the Anishinaabe people, and the Dakota people before them, from time immemorial. Ceded by the Anishinaabe in an 1854 treaty, this land holds great historical, spiritual, and personal significance for its original stewards, the Native nations and peoples of this region. We recognize and continually support and advocate for the sovereignty of the Native nations in this territory and beyond. By offering this land acknowledgment, we affirm tribal sovereignty and will work to hold the University of Minnesota Duluth accountable to American Indian peoples and nations.* -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 227A033FD7374F6BBA8449C0D573662F.png Type: image/png Size: 159 bytes Desc: not available URL: From leah.tams at duke.edu Fri Jan 6 10:05:50 2023 From: leah.tams at duke.edu (Leah Tams) Date: Fri, 6 Jan 2023 15:05:50 +0000 Subject: [Archivesspace_Users_Group] [External] Re: Preferred Citation Notes In-Reply-To: References: Message-ID: Confirming that if I click the Citation button/book icon in the PUI, I see what I assume is the preferred citation. Screenshot attached. Best, Leah she/her/hers MSLS in Archives & Records Management Accessions Coordinator Rubenstein Rare Book & Manuscript Library Duke University leah.tams at duke.edu | leahtams.org From: archivesspace_users_group-bounces at lyralists.lyrasis.org On Behalf Of Blake Carver Sent: Friday, January 6, 2023 9:58 AM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] [External] Re: Preferred Citation Notes I think it just shows up in the button thingy there on the top? There is a JIRA in about notes in general here https://archivesspace.atlassian.net/browse/ANW-968 And there's a comment there that calls out the preferred cite as well, though I'm not quite sure how to read it "The default would also include all notes expect "preferred citation"." I THINK that means all notes but NOT preferred cite? ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org > on behalf of Holland, Andrew S > Sent: Friday, January 6, 2023 9:49 AM To: Archivesspace Users Group > Subject: Re: [Archivesspace_Users_Group] [External] Re: Preferred Citation Notes Thank you for the response, Blake. This section of the docs explains kind of how preferred citation is used, but my issue is that the citation doesn't display to the public. We have information in the preferred citation field but you can't see that unless you're viewing the resource in the admin interface. https://aspace.lib.uiowa.edu/repositories/4/resources/2762 Abstract, Arrangement, Scope, etc. all display under their own header, but there is no preferred citation section even though we have that filled out. I can't figure out why that wouldn't display. -Andrew From: Blake Carver Sent: Thursday, December 15, 2022 11:48 AM To: Archivesspace Users Group Subject: [External] Re: [Archivesspace_Users_Group] Preferred Citation Notes I think the docs explain it here? https://archivesspace.atlassian.net/wiki/spaces/ADC/pages/103209762/Public+Interface+Documentation+for+staff#PublicInterfaceDocumentation(forstaff)-CitationactionbuttonandthePreferredCitationnote From: archivesspace_users_group-bounces at lyralists.lyrasis.org > on behalf of Holland, Andrew S > Sent: Thursday, December 15, 2022 8:25 AM To: Archivesspace Users Group > Subject: [Archivesspace_Users_Group] Preferred Citation Notes Hello all, I can seem to figure out how to get our notes put in to the Preferred Citation field to show up on the public site. Can anyone help me? Thanks, Andrew Holland University of Iowa Libraries -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.png Type: image/png Size: 183 bytes Desc: image002.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Iowa ASPUI.PNG Type: image/png Size: 20576 bytes Desc: Iowa ASPUI.PNG URL: From andrew-holland at uiowa.edu Fri Jan 6 10:24:40 2023 From: andrew-holland at uiowa.edu (Holland, Andrew S) Date: Fri, 6 Jan 2023 15:24:40 +0000 Subject: [Archivesspace_Users_Group] [External] Re: Preferred Citation Notes In-Reply-To: References: Message-ID: Thank you. Yes, I see it in the pop up window. It just seemed so odd to me that it wouldn?t display with the rest of the notes. I?m sure there?s a very good reason for the extra overhead. Maybe I?ll try to write a lightweight plugin to insert that note in with all his friends. Thanks for your responses and the link to that ticket. I read it the same way as ?except preferred citation.? Not sure why PC gets singled out the way it is. Maybe one of those great mysteries of life. From: Leah Tams Sent: Friday, January 6, 2023 9:06 AM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] [External] Re: Preferred Citation Notes Confirming that if I click the Citation button/book icon in the PUI, I see what I assume is the preferred citation. Screenshot attached. Best, Leah she/her/hers MSLS in Archives & Records Management Accessions Coordinator Rubenstein Rare Book & Manuscript Library Duke University leah.tams at duke.edu | leahtams.org From: archivesspace_users_group-bounces at lyralists.lyrasis.org On Behalf Of Blake Carver Sent: Friday, January 6, 2023 9:58 AM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] [External] Re: Preferred Citation Notes I think it just shows up in the button thingy there on the top? There is a JIRA in about notes in general here https://archivesspace.atlassian.net/browse/ANW-968 And there's a comment there that calls out the preferred cite as well, though I'm not quite sure how to read it "The default would also include all notes expect ?preferred citation?." I THINK that means all notes but NOT preferred cite? From: archivesspace_users_group-bounces at lyralists.lyrasis.org > on behalf of Holland, Andrew S > Sent: Friday, January 6, 2023 9:49 AM To: Archivesspace Users Group > Subject: Re: [Archivesspace_Users_Group] [External] Re: Preferred Citation Notes Thank you for the response, Blake. This section of the docs explains kind of how preferred citation is used, but my issue is that the citation doesn?t display to the public. We have information in the preferred citation field but you can?t see that unless you?re viewing the resource in the admin interface. https://aspace.lib.uiowa.edu/repositories/4/resources/2762 Abstract, Arrangement, Scope, etc. all display under their own header, but there is no preferred citation section even though we have that filled out. I can?t figure out why that wouldn?t display. -Andrew From: Blake Carver Sent: Thursday, December 15, 2022 11:48 AM To: Archivesspace Users Group Subject: [External] Re: [Archivesspace_Users_Group] Preferred Citation Notes I think the docs explain it here? https://archivesspace.atlassian.net/wiki/spaces/ADC/pages/103209762/Public+Interface+Documentation+for+staff#PublicInterfaceDocumentation(forstaff)-CitationactionbuttonandthePreferredCitationnote From: archivesspace_users_group-bounces at lyralists.lyrasis.org > on behalf of Holland, Andrew S > Sent: Thursday, December 15, 2022 8:25 AM To: Archivesspace Users Group > Subject: [Archivesspace_Users_Group] Preferred Citation Notes Hello all, I can seem to figure out how to get our notes put in to the Preferred Citation field to show up on the public site. Can anyone help me? Thanks, Andrew Holland University of Iowa Libraries -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 89178947C7DC4060B22610966FA3ECB3.png Type: image/png Size: 141 bytes Desc: 89178947C7DC4060B22610966FA3ECB3.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: F672D05B47444CF2A0E653E49A755E01.png Type: image/png Size: 203 bytes Desc: F672D05B47444CF2A0E653E49A755E01.png URL: From kstrosch at wellesley.edu Fri Jan 6 12:26:43 2023 From: kstrosch at wellesley.edu (Kara Hart) Date: Fri, 6 Jan 2023 12:26:43 -0500 Subject: [Archivesspace_Users_Group] Contents not displaying, Top Containers not searchable and errors in log Message-ID: My Archives staff are reporting issues with a newly added collection's content (series, files, etc) not displaying in the PUI. Also when creating top containers for some records, they do not show up in Manage Top Containers search results when searching by Resources title. I can search by container name and see a result but no resource is listed. I'm seeing the below errors and warnings repeated in my log file. Optimistic locking error but no record to look at? And then a few related to the index_state. Is this an indexing problem, should I fire off a full reindexing? Is there something bigger happening? Thanks for any advice, Kara W, [2023-01-06T12:21:32.637746 #2142] WARN -- : Thread-16226: Got an optimistic locking error when updating user: Attempt to update object did not result in a single row modification (SQL: UPDATE `user` SET `system_mtime` = '2023-01-06 17:21:32', `user_mtime` = '2023-01-06 17:21:32', `lock_version` = 34391 WHERE ((`id` = 2) AND (`lock_version` = 34390)) LIMIT 1) E, [2023-01-06T12:21:32.661982 #2142] ERROR -- : Thread-2296: org/jruby/RubyIO.java:2674:in `readline' /var/local/archivesspace/data/tmp/jetty-0.0.0.0-8091-indexer.war-_aspace-indexer-any-/webapp/WEB-INF/app/lib/index_state.rb:31:in `block in get_last_mtime' org/jruby/RubyIO.java:1171:in `open' /var/local/archivesspace/data/tmp/jetty-0.0.0.0-8091-indexer.war-_aspace-indexer-any-/webapp/WEB-INF/app/lib/index_state.rb:30:in `get_last_mtime' /var/local/archivesspace/data/tmp/jetty-0.0.0.0-8091-indexer.war-_aspace-indexer-any-/webapp/WEB-INF/app/lib/periodic_indexer.rb:89:in `run_index_round' /var/local/archivesspace/data/tmp/jetty-0.0.0.0-8091-indexer.war-_aspace-indexer-any-/webapp/WEB-INF/app/lib/periodic_indexer.rb:210:in `run' /var/local/archivesspace/data/tmp/jetty-0.0.0.0-8091-indexer.war-_aspace-indexer-any-/webapp/WEB-INF/app/main.rb:45:in `block in main' E, [2023-01-06T12:21:32.662160 #2142] ERROR -- : Thread-2296: # E, [2023-01-06T12:21:32.669214 #2142] ERROR -- : Thread-2016: org/jruby/RubyIO.java:2674:in `readline' /var/local/archivesspace/data/tmp/jetty-0.0.0.0-8091-indexer.war-_aspace-indexer-any-/webapp/WEB-INF/app/lib/index_state.rb:31:in `block in get_last_mtime' org/jruby/RubyIO.java:1171:in `open' /var/local/archivesspace/data/tmp/jetty-0.0.0.0-8091-indexer.war-_aspace-indexer-any-/webapp/WEB-INF/app/lib/index_state.rb:30:in `get_last_mtime' /var/local/archivesspace/data/tmp/jetty-0.0.0.0-8091-indexer.war-_aspace-indexer-any-/webapp/WEB-INF/app/lib/periodic_indexer.rb:89:in `run_index_round' /var/local/archivesspace/data/tmp/jetty-0.0.0.0-8091-indexer.war-_aspace-indexer-any-/webapp/WEB-INF/app/lib/periodic_indexer.rb:210:in `run' /var/local/archivesspace/data/tmp/jetty-0.0.0.0-8091-indexer.war-_aspace-indexer-any-/webapp/WEB-INF/app/main.rb:32:in `block in main' E, [2023-01-06T12:21:32.669343 #2142] ERROR -- : Thread-2016: # ~~~~~ Kara S. Hart Systems Librarian - Library & Technology Services - Wellesley College *** Many people want your password so they can steal your information. If an email asks you to send your password or directs you to a page that is NOT in the wellesley.edu domain, DO NOT provide your password. LTS will never ask for your passwords -- when in doubt, contact the LTS Help Desk. -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew.morrison at bodleian.ox.ac.uk Mon Jan 9 07:20:54 2023 From: andrew.morrison at bodleian.ox.ac.uk (Andrew Morrison) Date: Mon, 9 Jan 2023 12:20:54 +0000 Subject: [Archivesspace_Users_Group] Contents not displaying, Top Containers not searchable and errors in log In-Reply-To: References: Message-ID: You might find some tips by reading this thread from a little over a year ago: http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/2021-December/008848.html In your case, the affected user is the one with the ID of 2, which is the "search_indexer" user. The indexer has to log in to the backend to get new/modified records to index them. Part of the logging-in process is updating the user table. So, if that is being blocked, new/modified records won't appear in the PUI. Andrew. On 06/01/2023 17:26, Kara Hart wrote: > My Archives staff are reporting issues with a newly added collection's > content (series, files, etc) not displaying in the PUI. > Also when creating top containers for some records, they do not show > up in Manage Top Containers search results when searching by Resources > title.? I can search by container name and see a result but no > resource is listed. > > > I'm seeing the below errors and warnings repeated in my log file. > Optimistic locking error but no record to look at?? And then a few > related to the index_state. > Is this an indexing problem, should I fire off a full reindexing?? Is > there something bigger happening? > Thanks for any advice, > Kara > > W, [2023-01-06T12:21:32.637746 #2142] ?WARN -- : Thread-16226: Got an > optimistic locking error when updating user: Attempt to update object > did not result in a single row modification (SQL: UPDATE `user` SET > `system_mtime` = '2023-01-06 17:21:32', `user_mtime` = '2023-01-06 > 17:21:32', `lock_version` = 34391 WHERE ((`id` = 2) AND > (`lock_version` = 34390)) LIMIT 1) > E, [2023-01-06T12:21:32.661982 #2142] ERROR -- : Thread-2296: > org/jruby/RubyIO.java:2674:in `readline' > /var/local/archivesspace/data/tmp/jetty-0.0.0.0-8091-indexer.war-_aspace-indexer-any-/webapp/WEB-INF/app/lib/index_state.rb:31:in > `block in get_last_mtime' > org/jruby/RubyIO.java:1171:in `open' > /var/local/archivesspace/data/tmp/jetty-0.0.0.0-8091-indexer.war-_aspace-indexer-any-/webapp/WEB-INF/app/lib/index_state.rb:30:in > `get_last_mtime' > /var/local/archivesspace/data/tmp/jetty-0.0.0.0-8091-indexer.war-_aspace-indexer-any-/webapp/WEB-INF/app/lib/periodic_indexer.rb:89:in > `run_index_round' > /var/local/archivesspace/data/tmp/jetty-0.0.0.0-8091-indexer.war-_aspace-indexer-any-/webapp/WEB-INF/app/lib/periodic_indexer.rb:210:in > `run' > /var/local/archivesspace/data/tmp/jetty-0.0.0.0-8091-indexer.war-_aspace-indexer-any-/webapp/WEB-INF/app/main.rb:45:in > `block in main' > E, [2023-01-06T12:21:32.662160 #2142] ERROR -- : Thread-2296: > # > E, [2023-01-06T12:21:32.669214 #2142] ERROR -- : Thread-2016: > org/jruby/RubyIO.java:2674:in `readline' > /var/local/archivesspace/data/tmp/jetty-0.0.0.0-8091-indexer.war-_aspace-indexer-any-/webapp/WEB-INF/app/lib/index_state.rb:31:in > `block in get_last_mtime' > org/jruby/RubyIO.java:1171:in `open' > /var/local/archivesspace/data/tmp/jetty-0.0.0.0-8091-indexer.war-_aspace-indexer-any-/webapp/WEB-INF/app/lib/index_state.rb:30:in > `get_last_mtime' > /var/local/archivesspace/data/tmp/jetty-0.0.0.0-8091-indexer.war-_aspace-indexer-any-/webapp/WEB-INF/app/lib/periodic_indexer.rb:89:in > `run_index_round' > /var/local/archivesspace/data/tmp/jetty-0.0.0.0-8091-indexer.war-_aspace-indexer-any-/webapp/WEB-INF/app/lib/periodic_indexer.rb:210:in > `run' > /var/local/archivesspace/data/tmp/jetty-0.0.0.0-8091-indexer.war-_aspace-indexer-any-/webapp/WEB-INF/app/main.rb:32:in > `block in main' > E, [2023-01-06T12:21:32.669343 #2142] ERROR -- : Thread-2016: > # > > ~~~~~ Kara S. Hart > > Systems Librarian - Library & Technology Services > - > Wellesley College > > *** Many people want your password so they can steal your information. > If an email asks you to send your password or directs you to a page > that is NOT in the wellesley.edu ?domain, DO NOT > provide your password. LTS will never ask for your passwords -- when > in doubt, contact the LTS Help Desk. > > _______________________________________________ > Archivesspace_Users_Group mailing list > Archivesspace_Users_Group at lyralists.lyrasis.org > http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group -------------- next part -------------- An HTML attachment was scrubbed... URL: From Corey.Schmidt at uga.edu Mon Jan 9 11:25:47 2023 From: Corey.Schmidt at uga.edu (Corey Schmidt) Date: Mon, 9 Jan 2023 16:25:47 +0000 Subject: [Archivesspace_Users_Group] Get Archival Objects for a Top Container Using the API Message-ID: Dear all, How would I go about getting all of the associated archival objects for a top container using the API? Would doing something like the following capture all archival objects: search_aos = client.get_paged(f'repositories/4/search', params={'q': f'{topcontainer_barcode}', 'type': ['archival_object']}) ao_results = [result for result in search_aos] The above works (returns 27 archival objects, exactly what is shown in ASpace), but I am concerned this won't get everything or potentially get unassociated archival objects. I tried a different query, one based on how the staff interface displays linked records in a top container view, but this kind of query returns all archival objects for a repository, not the ones connected to the top container: search_aos = client.get_paged(f'repositories/4/search', params={'filter_term[]': f'top_container_uri_u_sstr: {topccontainer_uri}', 'type': ['archival_object']}) Any help/advice is appreciated! Thanks, Corey Corey Schmidt Special Collections Libraries | Project Management Librarian/Archivist Corey.Schmidt at uga.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ASpace-ao_tc-results.png Type: image/png Size: 128045 bytes Desc: ASpace-ao_tc-results.png URL: From Jessica.Crouch at lyrasis.org Wed Jan 11 11:38:25 2023 From: Jessica.Crouch at lyrasis.org (Jessica Crouch) Date: Wed, 11 Jan 2023 16:38:25 +0000 Subject: [Archivesspace_Users_Group] Webinar Reminder: Local Contexts Initiative and Integration Plugin for ArchivesSpace In-Reply-To: References: Message-ID: Dear ArchivesSpace Users, Join ArchivesSpace community members and Local Contexts project members for a webinar next Wednesday, January 18, 2023, at 2:00pm ET/11:00am PT, on the Local Contexts initiative and the new ArchivesSpace plugin developed by ArchivesSpace community member Joshua Shaw that integrates the Traditional Knowledge and Biocultural Labels in ArchivesSpace. Date: January 18, 2023 Time: 2:00pm ? 3:00pm ET (11:00am ? 12:00pm PT) Where: Zoom Registration: https://lyrasis.zoom.us/webinar/register/WN_xzG-JH8GRtGd8-Xfed0xJQ This webinar will be recorded and made available on the ArchivesSpace YouTube channel. Webinar description: In this webinar, Jane Anderson of New York University and Co-Director of Local Contexts will give an introduction to the Local Contexts initiative and newly launched Local Contexts Hub. Local Contexts is an initiative offering digital strategies for Indigenous communities, institutions, and researchers through the TK (Traditional Knowledge) & BC (Biocultural) Labels and Notices. Local Contexts is focused on increasing Indigenous involvement in data governance through the integration of Indigenous values into data systems. After, ArchivesSpace community member Joshua Shaw will demonstrate the Local Contexts integration plugin he developed that integrates Local Contexts Labels and Notices with ArchivesSpace. Feedback from the ArchivesSpace community would be appreciated to ensure that the plugin works for a variety of institutions. Feature requests, bug reports, or suggestions for changes are encouraged. Exports (EAD, EAD3, and MARCXML) are of special interest and suggestions for improvement would be very welcome. Dr. Jane Anderson is an Associate Professor at New York University in Lenapehoking (New York) and Global Fellow in the Engelberg Center for Innovation Law and Policy in the Law School at NYU. Jane has a Ph.D. in Law from the Law School at University of New South Wales in Australia. Their work is focused on intellectual and cultural property law, Indigenous rights, and the protection of Indigenous/traditional knowledge, cultural heritage, and data. Joshua Shaw is a web and application developer at Dartmouth Library and works with a variety of applications including ArchivesSpace, Omeka-S, Drupal, and other digital collection software. He has been working with ArchivesSpace since 2014 and is a member of the Core Committers group. Recently, Joshua has been working on ArchivesSpace plugins focused on issues related to reparative description. This webinar is being held to further the ArchivesSpace anti-racism and inclusion initiatives identified by the community to support and amplify those that document and empower individuals, organizations and communities that work to make the world a more inclusive place. To learn more, visit our wiki at https://archivesspace.atlassian.net/wiki/spaces/ADC/pages/1737261057/Anti-racism+and+Inclusion+Initiatives. Jessica Dowd Crouch Community Engagement Coordinator for ArchivesSpace jessica.crouch at lyrasis.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt.lawson at uni.edu Thu Jan 12 10:40:17 2023 From: matt.lawson at uni.edu (Matthew Lawson) Date: Thu, 12 Jan 2023 09:40:17 -0600 Subject: [Archivesspace_Users_Group] Changing PUI inheritance Message-ID: My goal was to change PUI inheritance in the config.rb so that the ?Preferred Citation? text is inherited by child records So I uncommented this section at bottom and added the last property myself which I found at https://gist.github.com/fordmadox/6f91ee378ae9d8a471bb8720fd6f2bd9 I then removed the files in /data/indexer_pui_state and restarted the service. But the public and staff websites never came back up. And I noticed that even after a half hour indexer_pui_state was not repopulating as I thought it should. So I reverted to my old config.rb file and restarted. Some indexer files came back but the public website gave a (rails error warning?) *We're sorry, but something went wrong.* I have now reverted to a save point but do not know how to fix record inheritance. I am on 3.1.1 AppConfig[:record_inheritance] = { :archival_object => { :inherited_fields => [ { :property => 'title', :inherit_directly => false }, { :property => 'component_id', :inherit_directly => false }, { :property => 'language', :inherit_directly => false }, { :property => 'dates', :inherit_directly => false }, { :property => 'extents', :inherit_directly => false }, { :property => 'linked_agents', :inherit_if => proc {|json| json.select {|j| j['role'] == 'creator'} }, :inherit_directly => false }, { :property => 'notes', :inherit_if => proc {|json| json.select {|j| j['type'] == 'accessrestrict'} }, :inherit_directly => false }, { :property => 'notes', :inherit_if => proc {|json| json.select {|j| j['type'] == 'scopecontent'} }, :inherit_directly => false }, { :property => 'notes', :inherit_if => proc { |json| json.select {|j| j['type'] == 'prefercite'} }, :inherit_directly => true }, ] } } -- Matt Lawson Cloud Systems Application Administrator IT :: Network & Infrastructure Services :: Specialized Systems 319-273-3931 ITTC 117 / Cedar Falls, IA 50614 uni.edu / *#1UNI* [image: UNI / University of Northern Iowa] -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt.lawson at uni.edu Thu Jan 12 10:52:08 2023 From: matt.lawson at uni.edu (Matthew Lawson) Date: Thu, 12 Jan 2023 09:52:08 -0600 Subject: [Archivesspace_Users_Group] Advice requested on updating public texts with yml Message-ID: I have modified the title, among other things, in a new en.yml file that I want to be visible to the public. But even after a restart, the public site continues to display the old info. I renamed the old en.yml files so as not to lose them and I know via grep results that there can be no other hiding place for where the public website data is coming from. Is there a better way to force Ruby to display current info? $/usr/local/archivesspace$ grep -rnw '/usr/local/archivesspace' -e 'Archives at UNI' /usr/local/archivesspace/locales/public/locales/public/en.jan.yml:24: title: Archives at UNI /usr/local/archivesspace/locales/public/locales/public/en.jan.yml:32: welcome_page_title: Archives at UNI /usr/local/archivesspace/locales/public/public/en.jan.yml:24: title: Archives at UNI /usr/local/archivesspace/locales/public/public/en.jan.yml:32: welcome_page_title: Archives at UNI /usr/local/archivesspace/locales/public/en.jan.yml:24: title: Archives at UNI /usr/local/archivesspace/locales/public/en.jan.yml:32: welcome_page_title: Archives at UNI grep: /usr/local/archivesspace/data/tmp/index_batch_: Permission denied grep: /usr/local/archivesspace/data/tmp/index_batch_: Permission denied /usr/local/archivesspace/plugins/local/public/locales/en.jan.yml:24: title: Archives at UNI /usr/local/archivesspace/plugins/local/public/locales/en.jan.yml:32: welcome_page_title: Archives at UNI $/usr/local/archivesspace$ grep -rnw '/usr/local/archivesspace' -e 'Archives and Finding Aids' /usr/local/archivesspace/locales/public/locales/public/en.yml:24: title: Archives and Finding Aids /usr/local/archivesspace/locales/public/locales/public/en.yml:30: welcome_page_title: Archives and Finding Aids /usr/local/archivesspace/locales/public/public/en.yml:24: title: Archives and Finding Aids /usr/local/archivesspace/locales/public/public/en.yml:31: welcome_page_title: Archives and Finding Aids /usr/local/archivesspace/locales/public/en.yml:24: title: Archives and Finding Aids /usr/local/archivesspace/locales/public/en.yml:30: welcome_page_title: Archives and Finding Aids grep: /usr/local/archivesspace/data/tmp/index_batch_: Permission denied /usr/local/archivesspace/plugins/local/public/locales/en.yml:24: title: Archives and Finding Aids /usr/local/archivesspace/plugins/local/public/locales/en.yml:33: welcome_page_title: Archives and Finding Aids -- Matt Lawson Cloud Systems Application Administrator IT :: Network & Infrastructure Services :: Specialized Systems 319-273-3931 ITTC 117 / Cedar Falls, IA 50614 uni.edu / *#1UNI* [image: UNI / University of Northern Iowa] -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew-holland at uiowa.edu Thu Jan 12 11:19:12 2023 From: andrew-holland at uiowa.edu (Holland, Andrew S) Date: Thu, 12 Jan 2023 16:19:12 +0000 Subject: [Archivesspace_Users_Group] [External] Advice requested on updating public texts with yml In-Reply-To: References: Message-ID: I?m not sure that I completely understand what all is going on with this, but the permissions denied on /data/tmp/* seems like the first thing you might want to address. You could see how it goes after you change the permissions on that directory? Andrew Holland (h/h/h) Administrative Lead, LIT 5023 Main Library, Iowa City, Iowa 52242 Office: 319-335-1193 lib.uiowa.edu [The University of Iowa Logo] From: archivesspace_users_group-bounces at lyralists.lyrasis.org On Behalf Of Matthew Lawson Sent: Thursday, January 12, 2023 9:52 AM To: Archivesspace Users Group Subject: [External] [Archivesspace_Users_Group] Advice requested on updating public texts with yml I have modified the title, among other things, in a new en.yml file that I want to be visible to the public. But even after a restart, the public site continues to display the old info. I renamed the old en.yml files so as not to lose them and I know via grep results that there can be no other hiding place for where the public website data is coming from. Is there a better way to force Ruby to display current info? $/usr/local/archivesspace$ grep -rnw '/usr/local/archivesspace' -e 'Archives at UNI' /usr/local/archivesspace/locales/public/locales/public/en.jan.yml:24: title: Archives at UNI /usr/local/archivesspace/locales/public/locales/public/en.jan.yml:32: welcome_page_title: Archives at UNI /usr/local/archivesspace/locales/public/public/en.jan.yml:24: title: Archives at UNI /usr/local/archivesspace/locales/public/public/en.jan.yml:32: welcome_page_title: Archives at UNI /usr/local/archivesspace/locales/public/en.jan.yml:24: title: Archives at UNI /usr/local/archivesspace/locales/public/en.jan.yml:32: welcome_page_title: Archives at UNI grep: /usr/local/archivesspace/data/tmp/index_batch_: Permission denied grep: /usr/local/archivesspace/data/tmp/index_batch_: Permission denied /usr/local/archivesspace/plugins/local/public/locales/en.jan.yml:24: title: Archives at UNI /usr/local/archivesspace/plugins/local/public/locales/en.jan.yml:32: welcome_page_title: Archives at UNI $/usr/local/archivesspace$ grep -rnw '/usr/local/archivesspace' -e 'Archives and Finding Aids' /usr/local/archivesspace/locales/public/locales/public/en.yml:24: title: Archives and Finding Aids /usr/local/archivesspace/locales/public/locales/public/en.yml:30: welcome_page_title: Archives and Finding Aids /usr/local/archivesspace/locales/public/public/en.yml:24: title: Archives and Finding Aids /usr/local/archivesspace/locales/public/public/en.yml:31: welcome_page_title: Archives and Finding Aids /usr/local/archivesspace/locales/public/en.yml:24: title: Archives and Finding Aids /usr/local/archivesspace/locales/public/en.yml:30: welcome_page_title: Archives and Finding Aids grep: /usr/local/archivesspace/data/tmp/index_batch_: Permission denied /usr/local/archivesspace/plugins/local/public/locales/en.yml:24: title: Archives and Finding Aids /usr/local/archivesspace/plugins/local/public/locales/en.yml:33: welcome_page_title: Archives and Finding Aids -- Matt Lawson Cloud Systems Application Administrator IT :: Network & Infrastructure Services :: Specialized Systems 319-273-3931 ITTC 117 / Cedar Falls, IA 50614 uni.edu / #1UNI [UNI / University of Northern Iowa] -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 1161 bytes Desc: image001.png URL: From philip.webster at sheffield.ac.uk Thu Jan 12 11:23:46 2023 From: philip.webster at sheffield.ac.uk (Philip Webster) Date: Thu, 12 Jan 2023 16:23:46 -0000 Subject: [Archivesspace_Users_Group] ArchivesSpace backups on 3.2 failing In-Reply-To: References: Message-ID: <34e701d926a2$3f495bf0$bddc13d0$@sheffield.ac.uk> Hi Scott, what was the solution for this in the end? We?ve just upgraded to 3.2.0 at Sheffield and we?re seeing the same thing. Regards, Phil From: archivesspace_users_group-bounces at lyralists.lyrasis.org On Behalf Of RENTON Scott Sent: 24 May 2022 17:31 To: archivesspace_users_group at lyralists.lyrasis.org Subject: Re: [Archivesspace_Users_Group] ArchivesSpace backups on 3.2 failing Thanks Andrew. Yeah, here it is (it's just the default solrconfig.xml from archivesspace) Cheers Scott ========== Scott Renton Digital Library Development & Systems Floor F East Argyle House 515219 _____ From: archivesspace_users_group-bounces at lyralists.lyrasis.org > on behalf of Andrew Morrison > Sent: 24 May 2022 16:56 To: archivesspace_users_group at lyralists.lyrasis.org > Subject: Re: [Archivesspace_Users_Group] ArchivesSpace backups on 3.2 failing This email was sent to you by someone outside the University. You should only click on links or attachments if you are certain that the email is genuine and the content is safe. Does your external Solr's "archivesspace" core define a requestHandler whose class is "solr.ReplicationHandler" in its solrconfig.xml? Is the name "/replication"? Andrew. On 24/05/2022 16:42, RENTON Scott wrote: Hi folks I upgraded an instance to v.3.2.0 on Friday, but notice that backups are failing (to do with the new external SOLR?). This is using /apps/archivesspace/scripts/backup.sh --mysqldump --output ${BACKUP_DIR}/archivesspace_backup_$DATE.zip as normal. Has the script changed substantially? We were on 2.8.1 previously. Error looks like this: Loading ArchivesSpace configuration file from path: /apps/archivesspace/config/config.rb WARNING: The parameter 'pui_email_delivery_method' was already set WARNING: The parameter 'pui_email_perform_deliveries' was already set WARNING: The parameter 'pui_email_raise_delivery_errors' was already set Loading ArchivesSpace configuration file from path: /apps/archivesspace/config/config.rb WARNING: The parameter 'pui_email_delivery_method' was already set WARNING: The parameter 'pui_email_perform_deliveries' was already set WARNING: The parameter 'pui_email_raise_delivery_errors' was already set 2022-05-24 14:32:48 +0100: Writing backup to /apps/backups/archivesspace_backup_220524_143245.zip ERROR: Solr snapshot failed (Solr snapshot failed: Problem when getting snapshot details: Error 404 Not Found

HTTP ERROR 404 Not Found

URI:/replication
STATUS:404
MESSAGE:Not Found
SERVLET:-
Otherwise the SOLR seems to be working fine, so I'm assuming my config is ok: AppConfig[:solr_url] = "http://localhost:8983/solr/archivesspace" AppConfig[:solr_verify_checksums] = false AppConfig[:data_directory] = File.join("/apps/solr/server/solr/archivesspace/data") I did wonder if it was the location of the data_directory that was the issue- it had been away from the solr core, but even after repointing that (above), I get the above error. Cheers Scott ========== Scott Renton Digital Library Development & Systems Floor F East Argyle House 515219 The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th? ann an Oilthigh Dh?n ?ideann, cl?raichte an Alba, ?ireamh cl?raidh SC005336. _______________________________________________ Archivesspace_Users_Group mailing list Archivesspace_Users_Group at lyralists.lyrasis.org http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group -------------- next part -------------- An HTML attachment was scrubbed... URL: From blake.carver at lyrasis.org Thu Jan 12 14:08:48 2023 From: blake.carver at lyrasis.org (Blake Carver) Date: Thu, 12 Jan 2023 19:08:48 +0000 Subject: [Archivesspace_Users_Group] Changing PUI inheritance In-Reply-To: References: Message-ID: Best to stick with what's in the official ArchivesSpace config.rb https://github.com/archivesspace/archivesspace/blob/master/common/config/config-defaults.rb#L389-L434 ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Matthew Lawson Sent: Thursday, January 12, 2023 10:40 AM To: Archivesspace Users Group Subject: [Archivesspace_Users_Group] Changing PUI inheritance My goal was to change PUI inheritance in the config.rb so that the ?Preferred Citation? text is inherited by child records So I uncommented this section at bottom and added the last property myself which I found at https://gist.github.com/fordmadox/6f91ee378ae9d8a471bb8720fd6f2bd9 I then removed the files in /data/indexer_pui_state and restarted the service. But the public and staff websites never came back up. And I noticed that even after a half hour indexer_pui_state was not repopulating as I thought it should. So I reverted to my old config.rb file and restarted. Some indexer files came back but the public website gave a (rails error warning?) We're sorry, but something went wrong. I have now reverted to a save point but do not know how to fix record inheritance. I am on 3.1.1 AppConfig[:record_inheritance] = { :archival_object => { :inherited_fields => [ { :property => 'title', :inherit_directly => false }, { :property => 'component_id', :inherit_directly => false }, { :property => 'language', :inherit_directly => false }, { :property => 'dates', :inherit_directly => false }, { :property => 'extents', :inherit_directly => false }, { :property => 'linked_agents', :inherit_if => proc {|json| json.select {|j| j['role'] == 'creator'} }, :inherit_directly => false }, { :property => 'notes', :inherit_if => proc {|json| json.select {|j| j['type'] == 'accessrestrict'} }, :inherit_directly => false }, { :property => 'notes', :inherit_if => proc {|json| json.select {|j| j['type'] == 'scopecontent'} }, :inherit_directly => false }, { :property => 'notes', :inherit_if => proc { |json| json.select {|j| j['type'] == 'prefercite'} }, :inherit_directly => true }, ] } } -- Matt Lawson Cloud Systems Application Administrator IT :: Network & Infrastructure Services :: Specialized Systems 319-273-3931 ITTC 117 / Cedar Falls, IA 50614 uni.edu / #1UNI [UNI / University of Northern Iowa] -------------- next part -------------- An HTML attachment was scrubbed... URL: From blake.carver at lyrasis.org Thu Jan 12 14:11:39 2023 From: blake.carver at lyrasis.org (Blake Carver) Date: Thu, 12 Jan 2023 19:11:39 +0000 Subject: [Archivesspace_Users_Group] [External] Advice requested on updating public texts with yml In-Reply-To: References: Message-ID: Agreeing with Andrew on that permissions error, might be important, might not be. For that path though, I think you want it more like /public/locales/en.yml I think you have an extra /public/ in there? ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Holland, Andrew S Sent: Thursday, January 12, 2023 11:19 AM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] [External] Advice requested on updating public texts with yml I?m not sure that I completely understand what all is going on with this, but the permissions denied on /data/tmp/* seems like the first thing you might want to address. You could see how it goes after you change the permissions on that directory? Andrew Holland (h/h/h) Administrative Lead, LIT 5023 Main Library, Iowa City, Iowa 52242 Office: 319-335-1193 lib.uiowa.edu [The University of Iowa Logo] From: archivesspace_users_group-bounces at lyralists.lyrasis.org On Behalf Of Matthew Lawson Sent: Thursday, January 12, 2023 9:52 AM To: Archivesspace Users Group Subject: [External] [Archivesspace_Users_Group] Advice requested on updating public texts with yml I have modified the title, among other things, in a new en.yml file that I want to be visible to the public. But even after a restart, the public site continues to display the old info. I renamed the old en.yml files so as not to lose them and I know via grep results that there can be no other hiding place for where the public website data is coming from. Is there a better way to force Ruby to display current info? $/usr/local/archivesspace$ grep -rnw '/usr/local/archivesspace' -e 'Archives at UNI' /usr/local/archivesspace/locales/public/locales/public/en.jan.yml:24: title: Archives at UNI /usr/local/archivesspace/locales/public/locales/public/en.jan.yml:32: welcome_page_title: Archives at UNI /usr/local/archivesspace/locales/public/public/en.jan.yml:24: title: Archives at UNI /usr/local/archivesspace/locales/public/public/en.jan.yml:32: welcome_page_title: Archives at UNI /usr/local/archivesspace/locales/public/en.jan.yml:24: title: Archives at UNI /usr/local/archivesspace/locales/public/en.jan.yml:32: welcome_page_title: Archives at UNI grep: /usr/local/archivesspace/data/tmp/index_batch_: Permission denied grep: /usr/local/archivesspace/data/tmp/index_batch_: Permission denied /usr/local/archivesspace/plugins/local/public/locales/en.jan.yml:24: title: Archives at UNI /usr/local/archivesspace/plugins/local/public/locales/en.jan.yml:32: welcome_page_title: Archives at UNI $/usr/local/archivesspace$ grep -rnw '/usr/local/archivesspace' -e 'Archives and Finding Aids' /usr/local/archivesspace/locales/public/locales/public/en.yml:24: title: Archives and Finding Aids /usr/local/archivesspace/locales/public/locales/public/en.yml:30: welcome_page_title: Archives and Finding Aids /usr/local/archivesspace/locales/public/public/en.yml:24: title: Archives and Finding Aids /usr/local/archivesspace/locales/public/public/en.yml:31: welcome_page_title: Archives and Finding Aids /usr/local/archivesspace/locales/public/en.yml:24: title: Archives and Finding Aids /usr/local/archivesspace/locales/public/en.yml:30: welcome_page_title: Archives and Finding Aids grep: /usr/local/archivesspace/data/tmp/index_batch_: Permission denied /usr/local/archivesspace/plugins/local/public/locales/en.yml:24: title: Archives and Finding Aids /usr/local/archivesspace/plugins/local/public/locales/en.yml:33: welcome_page_title: Archives and Finding Aids -- Matt Lawson Cloud Systems Application Administrator IT :: Network & Infrastructure Services :: Specialized Systems 319-273-3931 ITTC 117 / Cedar Falls, IA 50614 uni.edu / #1UNI [UNI / University of Northern Iowa] -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 1161 bytes Desc: image001.png URL: From shand at ap.org Wed Jan 18 09:48:59 2023 From: shand at ap.org (Hand, Sarit) Date: Wed, 18 Jan 2023 14:48:59 +0000 Subject: [Archivesspace_Users_Group] ISO DAMS Info Message-ID: Hello all! I was given a short list of DAMS to consider, and I was wondering if anyone in the ArchivesSpace community is familiar with any of them. If so, do any integrate in any capacity with ASpace? Any other information that might be pertinent would be great. I am happy to get on a call to chat as well. Asset Bank IntelligenceBank MediaValet NetX Picturepark ResourceSpace Terentia Widen Thank you in advance for any assistance you can provide. Cheers, [cid:image001.png at 01D92B22.1605C6A0] Sarit Hand Digital Archivist AP Corporate Archives 200 Liberty Street New York, NY 10281 T 212.621.7035 shand at ap.org ap.org [cid:image002.png at 01D92B22.1605C6A0] The information contained in this communication is intended for the use of the designated recipients named above. If the reader of this communication is not the intended recipient, you are hereby notified that you have received this communication in error, and that any review, dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify The Associated Press immediately by telephone at +1-212-621-1500 and delete this email. Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 1094 bytes Desc: image001.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.png Type: image/png Size: 5548 bytes Desc: image002.png URL: From ctorre at clarkart.edu Wed Jan 18 11:44:15 2023 From: ctorre at clarkart.edu (Colin Torre) Date: Wed, 18 Jan 2023 16:44:15 +0000 Subject: [Archivesspace_Users_Group] ISO DAMS Info In-Reply-To: References: Message-ID: Hi Sarit, I have some familiarity with a few of these platforms. Last year, the Clark Art Institute conducted an open search for a DAMS; I led the search and demo'd NetX, ResourceSpace, and Widen. The working group here ultimately selected ResourceSpace (the Clark is a fine arts museum and it was essential that the DAMS have a plug-in or data transfer option with TMS; of the DAMS we reviewed, only NetX and ResourceSpace met that standard). I am not aware of any DAMS that have developed ArchivesSpace functionality (aside from linking to an appropriate URL as a Digital Object). That's something I'd like to explore as we implement ResourceSpace, and I'd be curious to see the responses to your query here. Best, Colin Torre From: archivesspace_users_group-bounces at lyralists.lyrasis.org On Behalf Of Hand, Sarit Sent: Wednesday, January 18, 2023 9:49 AM To: Archivesspace Users Group ; archivesspace_small_archives_users_group at lyralists.lyrasis.org Subject: [Archivesspace_Users_Group] ISO DAMS Info CAUTION: This email originated from outside of The Clark organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. If there are any questions, please contact the I.T. Department. Hello all! I was given a short list of DAMS to consider, and I was wondering if anyone in the ArchivesSpace community is familiar with any of them. If so, do any integrate in any capacity with ASpace? Any other information that might be pertinent would be great. I am happy to get on a call to chat as well. Asset Bank IntelligenceBank MediaValet NetX Picturepark ResourceSpace Terentia Widen Thank you in advance for any assistance you can provide. Cheers, [cid:image001.png at 01D92B30.8BBD64D0] Sarit Hand Digital Archivist AP Corporate Archives 200 Liberty Street New York, NY 10281 T 212.621.7035 shand at ap.org ap.org [cid:image002.png at 01D92B30.8BBD64D0] The information contained in this communication is intended for the use of the designated recipients named above. If the reader of this communication is not the intended recipient, you are hereby notified that you have received this communication in error, and that any review, dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify The Associated Press immediately by telephone at +1-212-621-1500 and delete this email. Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 1094 bytes Desc: image001.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.png Type: image/png Size: 5548 bytes Desc: image002.png URL: From mladair at umich.edu Wed Jan 18 14:23:14 2023 From: mladair at umich.edu (Matthew Adair) Date: Wed, 18 Jan 2023 14:23:14 -0500 Subject: [Archivesspace_Users_Group] ASpace database issues Message-ID: Version 2.5.2 of ASpace. Version 8.0.30 of MySQL We get the following error in the log file: "The following MySQL database tables are not set to use UTF-8 for their character encoding:" and then it lists all of the database tables, and fails with an unable to connect to database error Encoding on the database and all of the tables are utf8mb3_general_ci This problem seems to have just cropped up and there have been no changes that I am aware of. Thoughts? -Matt ---- *Matt Adair* Archivist for Digital Imaging and Infrastructure Bentley Historical Library 1150 Beal Avenue Ann Arbor, Michigan 48109-2113 734-647-3537 http://bentley.umich.edu @UmichBentley *The Bentley Historical Library acknowledges that coerced cessions of land by the Anishnaabeg and Wyandot made the University of Michigan possible, and we seek to reaffirm the ancestral and contemporary ties of these peoples to the lands where the University now stands.* -------------- next part -------------- An HTML attachment was scrubbed... URL: From hanstra at nd.edu Wed Jan 18 15:17:55 2023 From: hanstra at nd.edu (Tom Hanstra) Date: Wed, 18 Jan 2023 15:17:55 -0500 Subject: [Archivesspace_Users_Group] ASpace database issues In-Reply-To: References: Message-ID: Is there any way that your version of MySQL got upgraded? I see this in the MySQL documentation: https://dev.mysql.com/doc/refman/8.0/en/charset-unicode-utf8.html which indicates that utf8mb3 is deprecated after Mysql 8.0.28. I don't have any experience to say that this is where the problem is. I was just looking around because I don't want to end up in a similar situation. Tom On Wed, Jan 18, 2023 at 2:23 PM Matthew Adair wrote: > Version 2.5.2 of ASpace. Version 8.0.30 of MySQL > > We get the following error in the log file: > "The following MySQL database tables are not set to use UTF-8 for their > character encoding:" > and then it lists all of the database tables, and fails with an unable to > connect to database error > > Encoding on the database and all of the tables are utf8mb3_general_ci > > This problem seems to have just cropped up and there have been no > changes that I am aware of. > > Thoughts? > -Matt > > ---- > *Matt Adair* > Archivist for Digital Imaging and Infrastructure > > > Bentley Historical Library > 1150 Beal Avenue > Ann Arbor, Michigan 48109-2113 > 734-647-3537 > http://bentley.umich.edu > @UmichBentley > > *The Bentley Historical Library acknowledges that coerced cessions of land > by the Anishnaabeg and Wyandot made the University of Michigan possible, > and we seek to reaffirm the ancestral and contemporary ties of these > peoples to the lands where the University now stands.* > _______________________________________________ > Archivesspace_Users_Group mailing list > Archivesspace_Users_Group at lyralists.lyrasis.org > http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group > -- *Tom Hanstra* *Sr. Systems Administrator* hanstra at nd.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From ph448 at cam.ac.uk Wed Jan 18 16:15:42 2023 From: ph448 at cam.ac.uk (Peter Heiner) Date: Wed, 18 Jan 2023 21:15:42 +0000 Subject: [Archivesspace_Users_Group] ASpace database issues In-Reply-To: References: Message-ID: <20230118211542.dybjiuazz6aifklh@sparkly> Matthew Adair wrote on 2023-01-18 14:23:14: > Version 2.5.2 of ASpace. Version 8.0.30 of MySQL A couple of versions prior to yours MySQL started explicitly reporting 'utf8mb3' where in the past it would say 'utf8'. That's because 'utf8' is currently an alias for 'utf8mb3' and they want to deprecate 'utf8mb3' and make 'utf8' an alias of 'utf8mb4' instead. ASpace 2.5.2 only recognises 'utf8' as an UTF-8 character set, this was fixed in ASpace 2.7.1. I would consider upgrading to a more recent version if possible. > We get the following error in the log file: > "The following MySQL database tables are not set to use UTF-8 for their > character encoding:" > and then it lists all of the database tables, and fails with an unable to > connect to database error > > Encoding on the database and all of the tables are utf8mb3_general_ci Short of the ASpace upgrade (or a database downgrade that might be even more painful) the correct solution is to convert table data to 'utf8mb4', see https://dev.mysql.com/doc/refman/8.0/en/charset-unicode-conversion.html for details on how to do that. Hope that helps, p From Jessica.Crouch at lyrasis.org Wed Jan 18 10:51:50 2023 From: Jessica.Crouch at lyrasis.org (Jessica Crouch) Date: Wed, 18 Jan 2023 15:51:50 +0000 Subject: [Archivesspace_Users_Group] Training Announcement: ArchivesSpace Application Overview and Basic Record Creation Message-ID: Dear ArchivesSpace users, While our community-wide trainings provide virtual learning opportunities on a variety of topics, we recognize attending the live version of these trainings is difficult for users outside the time zones represented in North America. To better facilitate access to training for some other parts of our community, we will be offering a free training on February 21, 2023, from 1:00pm-4pm Australian Eastern Daylight Time (AEDT). This training will provide an overview of the ArchivesSpace application and basic record creation. During this training, trainers from the ArchivesSpace Trainers Corps will cover some of the most frequently used areas of the ArchivesSpace application and will provide adequate time for Q&A. A full agenda will be provided to attendees in advance of the training. We hope the feedback we receive from this training will inform future ArchivesSpace training opportunities in a variety of time zones. ArchivesSpace Application Overview and Basic Record Creation Date: February 21, 2023 Time: 1:00pm-4:00pm Australian Eastern Daylight Time (find your local time*) Trainers: Corey Schmidt (University of Georgia) and Jessica Crouch (ArchivesSpace) Registration: https://lyrasis.zoom.us/webinar/register/WN_uwnVJgYbRaaHbxiBWzVOSA *Please note this training occurs in the evening, Monday, February 20th for some time zones. Anyone using ArchivesSpace may register for this training. This training will be recorded. The recording will be made available via the ArchivesSpace website and YouTube channel. You do not need to register for the training to have access to the recordings. We understand this training is outside regular business hours for users in the United States and we ask users not to register if they do not plan to attend live. While this training does not assume any prior experience with ArchivesSpace, attendees are expected to have an understanding of archival workflows and terminology. Due to the large number of attendees, trainers will not be able to troubleshoot individual implementations of ArchivesSpace or provide additional archival instruction beyond the use of ArchivesSpace. For more information about ArchivesSpace, visit our Getting Started webpage or email ArchivesSpaceHome at lyrasis.org Best, Jessica Jessica Dowd Crouch Community Engagement Coordinator for ArchivesSpace jessica.crouch at lyrasis.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From mladair at umich.edu Thu Jan 19 09:07:45 2023 From: mladair at umich.edu (Matthew Adair) Date: Thu, 19 Jan 2023 09:07:45 -0500 Subject: [Archivesspace_Users_Group] ASpace database issues In-Reply-To: References: Message-ID: Indeed, university IT updated our MySQL server from 5.x to 8.x. But this was back in early November. Everything had been working fine since then. ---- *Matt Adair* Archivist for Digital Imaging and Infrastructure Bentley Historical Library 1150 Beal Avenue Ann Arbor, Michigan 48109-2113 734-647-3537 http://bentley.umich.edu @UmichBentley *The Bentley Historical Library acknowledges that coerced cessions of land by the Anishnaabeg and Wyandot made the University of Michigan possible, and we seek to reaffirm the ancestral and contemporary ties of these peoples to the lands where the University now stands.* On Wed, Jan 18, 2023 at 3:18 PM Tom Hanstra wrote: > Is there any way that your version of MySQL got upgraded? I see this in > the MySQL documentation: > > https://dev.mysql.com/doc/refman/8.0/en/charset-unicode-utf8.html > > which indicates that utf8mb3 is deprecated after Mysql 8.0.28. > > I don't have any experience to say that this is where the problem is. I > was just looking around because I don't want to end up in a similar > situation. > > Tom > > On Wed, Jan 18, 2023 at 2:23 PM Matthew Adair wrote: > >> Version 2.5.2 of ASpace. Version 8.0.30 of MySQL >> >> We get the following error in the log file: >> "The following MySQL database tables are not set to use UTF-8 for their >> character encoding:" >> and then it lists all of the database tables, and fails with an unable to >> connect to database error >> >> Encoding on the database and all of the tables are utf8mb3_general_ci >> >> This problem seems to have just cropped up and there have been no >> changes that I am aware of. >> >> Thoughts? >> -Matt >> >> ---- >> *Matt Adair* >> Archivist for Digital Imaging and Infrastructure >> >> >> Bentley Historical Library >> 1150 Beal Avenue >> Ann Arbor, Michigan 48109-2113 >> 734-647-3537 >> http://bentley.umich.edu >> @UmichBentley >> >> *The Bentley Historical Library acknowledges that coerced cessions of >> land by the Anishnaabeg and Wyandot made the University of Michigan >> possible, and we seek to reaffirm the ancestral and contemporary ties of >> these peoples to the lands where the University now stands.* >> _______________________________________________ >> Archivesspace_Users_Group mailing list >> Archivesspace_Users_Group at lyralists.lyrasis.org >> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group >> > > > -- > *Tom Hanstra* > *Sr. Systems Administrator* > hanstra at nd.edu > > > _______________________________________________ > Archivesspace_Users_Group mailing list > Archivesspace_Users_Group at lyralists.lyrasis.org > http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mladair at umich.edu Thu Jan 19 09:14:39 2023 From: mladair at umich.edu (Matthew Adair) Date: Thu, 19 Jan 2023 09:14:39 -0500 Subject: [Archivesspace_Users_Group] ASpace database issues In-Reply-To: <20230118211542.dybjiuazz6aifklh@sparkly> References: <20230118211542.dybjiuazz6aifklh@sparkly> Message-ID: Thanks Peter. I was wondering if the age of our installation might have been the issue. I was even trying to poke through the code to see what it was looking for in the check to see if there was that sort of mis-match between what ASpac was looking for and what MySQL was returning. Fortunately, this is not our production server. It is a sandbox / testing server. It sounds like the upgrade to our university managed MySQL server (in conjunction with the old version of ASpace) is the root of the issue. If I were to hazard a guess, we haven't had to reboot the sandbox server since the MySQL upgrade and recent system reboot just brought the issue to light. Matt ---- *Matt Adair* Archivist for Digital Imaging and Infrastructure Bentley Historical Library 1150 Beal Avenue Ann Arbor, Michigan 48109-2113 734-647-3537 http://bentley.umich.edu @UmichBentley *The Bentley Historical Library acknowledges that coerced cessions of land by the Anishnaabeg and Wyandot made the University of Michigan possible, and we seek to reaffirm the ancestral and contemporary ties of these peoples to the lands where the University now stands.* On Wed, Jan 18, 2023 at 4:15 PM Peter Heiner wrote: > Matthew Adair wrote on 2023-01-18 14:23:14: > > Version 2.5.2 of ASpace. Version 8.0.30 of MySQL > > A couple of versions prior to yours MySQL started explicitly reporting > 'utf8mb3' where in the past it would say 'utf8'. That's because 'utf8' is > currently an alias for 'utf8mb3' and they want to deprecate 'utf8mb3' and > make > 'utf8' an alias of 'utf8mb4' instead. > ASpace 2.5.2 only recognises 'utf8' as an UTF-8 character set, this was > fixed > in ASpace 2.7.1. I would consider upgrading to a more recent version if > possible. > > > We get the following error in the log file: > > "The following MySQL database tables are not set to use UTF-8 for their > > character encoding:" > > and then it lists all of the database tables, and fails with an unable to > > connect to database error > > > > Encoding on the database and all of the tables are utf8mb3_general_ci > > Short of the ASpace upgrade (or a database downgrade that might be even > more > painful) the correct solution is to convert table data to 'utf8mb4', see > https://dev.mysql.com/doc/refman/8.0/en/charset-unicode-conversion.html > for details on how to do that. > > Hope that helps, > p > _______________________________________________ > Archivesspace_Users_Group mailing list > Archivesspace_Users_Group at lyralists.lyrasis.org > http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group > -------------- next part -------------- An HTML attachment was scrubbed... URL: From atruitt at ohiohistory.org Thu Jan 19 10:10:27 2023 From: atruitt at ohiohistory.org (Aimee Truitt) Date: Thu, 19 Jan 2023 15:10:27 +0000 Subject: [Archivesspace_Users_Group] Training Announcement: ArchivesSpace Application Overview and Basic Record Creation In-Reply-To: References: Message-ID: Hello Everyone, Please see the e-mail below for a community-wide training opportunity on ASpace. This webinar will cover application overview and basic record creation training. I will be attending and the training will be recorded, so I can share it with anyone who is interested, but unable to attend on February 21st. Best Wishes, Aimee From: archivesspace_member_reps-bounces at lyralists.lyrasis.org On Behalf Of Jessica Crouch Sent: Wednesday, January 18, 2023 10:52 AM To: Archivesspace Users Group ; Archivesspace Member Reps ; Archivesspace Small Archives Users Group ; archivesspace_bot_members at lyralists.lyrasis.org; archivesspace_tac at lyralists.lyrasis.org; archivesspace_uac at lyralists.lyrasis.org; ARCHIVISTS-collectmgmt at ConnectedCommunity.org; ica-l at mailman.srv.ualberta.ca Subject: [Archivesspace_member_reps] Training Announcement: ArchivesSpace Application Overview and Basic Record Creation Dear ArchivesSpace users, While our community-wide trainings provide virtual learning opportunities on a variety of topics, we recognize attending the live version of these trainings is difficult for users outside the time zones represented in North America. To better facilitate access to training for some other parts of our community, we will be offering a free training on February 21, 2023, from 1:00pm-4pm Australian Eastern Daylight Time (AEDT). This training will provide an overview of the ArchivesSpace application and basic record creation. During this training, trainers from the ArchivesSpace Trainers Corps will cover some of the most frequently used areas of the ArchivesSpace application and will provide adequate time for Q&A. A full agenda will be provided to attendees in advance of the training. We hope the feedback we receive from this training will inform future ArchivesSpace training opportunities in a variety of time zones. ArchivesSpace Application Overview and Basic Record Creation Date: February 21, 2023 Time: 1:00pm-4:00pm Australian Eastern Daylight Time (find your local time*) Trainers: Corey Schmidt (University of Georgia) and Jessica Crouch (ArchivesSpace) Registration: https://lyrasis.zoom.us/webinar/register/WN_uwnVJgYbRaaHbxiBWzVOSA *Please note this training occurs in the evening, Monday, February 20th for some time zones. Anyone using ArchivesSpace may register for this training. This training will be recorded. The recording will be made available via the ArchivesSpace website and YouTube channel. You do not need to register for the training to have access to the recordings. We understand this training is outside regular business hours for users in the United States and we ask users not to register if they do not plan to attend live. While this training does not assume any prior experience with ArchivesSpace, attendees are expected to have an understanding of archival workflows and terminology. Due to the large number of attendees, trainers will not be able to troubleshoot individual implementations of ArchivesSpace or provide additional archival instruction beyond the use of ArchivesSpace. For more information about ArchivesSpace, visit our Getting Started webpage or email ArchivesSpaceHome at lyrasis.org Best, Jessica Jessica Dowd Crouch Community Engagement Coordinator for ArchivesSpace jessica.crouch at lyrasis.org -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: ATT00001.txt URL: From Jessica.Crouch at lyrasis.org Thu Jan 19 11:17:49 2023 From: Jessica.Crouch at lyrasis.org (Jessica Crouch) Date: Thu, 19 Jan 2023 16:17:49 +0000 Subject: [Archivesspace_Users_Group] Training Announcement: ArchivesSpace Application Overview and Basic Record Creation In-Reply-To: References: Message-ID: Hi Aimee and other members of the ArchivesSpace user group listserv, I?m glad you are interested in ArchivesSpace training opportunities. I do want to point out that this training is being offered on February 21st from 1:00pm-4:00pm Australian Eastern Daylight Time (find your local time). This means this training will be offered outside regular business hours on the night of February 20th for those of us in the continental United States. This training was developed to better facilitate access to training for our community members who live outside the time zones we typically offer trainings. Since space is limited, we ask that users in the continental US consider refraining from registering for this training to allow other members of our community to attend. If you do plan to register, we ask that you only register if you intend to participate in the training live. As we mentioned in the announcement, this training will be recorded. The recording will be made available via the ArchivesSpace website and YouTube channel. You do not need to register for the training to have access to the recordings. I hope that information helps. If anyone has additional questions, please feel free to reach out at ArchivesSpaceHome at lyrasis.org. Best, Jessica Jessica Dowd Crouch Community Engagement Coordinator for ArchivesSpace jessica.crouch at lyrasis.org From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Aimee Truitt Date: Thursday, January 19, 2023 at 10:10 AM To: Archivesspace Users Group Subject: [Archivesspace_Users_Group] Training Announcement: ArchivesSpace Application Overview and Basic Record Creation Hello Everyone, Please see the e-mail below for a community-wide training opportunity on ASpace. This webinar will cover application overview and basic record creation training. I will be attending and the training will be recorded, so I can share it with anyone who is interested, but unable to attend on February 21st. Best Wishes, Aimee From: archivesspace_member_reps-bounces at lyralists.lyrasis.org On Behalf Of Jessica Crouch Sent: Wednesday, January 18, 2023 10:52 AM To: Archivesspace Users Group ; Archivesspace Member Reps ; Archivesspace Small Archives Users Group ; archivesspace_bot_members at lyralists.lyrasis.org; archivesspace_tac at lyralists.lyrasis.org; archivesspace_uac at lyralists.lyrasis.org; ARCHIVISTS-collectmgmt at ConnectedCommunity.org; ica-l at mailman.srv.ualberta.ca Subject: [Archivesspace_member_reps] Training Announcement: ArchivesSpace Application Overview and Basic Record Creation Dear ArchivesSpace users, While our community-wide trainings provide virtual learning opportunities on a variety of topics, we recognize attending the live version of these trainings is difficult for users outside the time zones represented in North America. To better facilitate access to training for some other parts of our community, we will be offering a free training on February 21, 2023, from 1:00pm-4pm Australian Eastern Daylight Time (AEDT). This training will provide an overview of the ArchivesSpace application and basic record creation. During this training, trainers from the ArchivesSpace Trainers Corps will cover some of the most frequently used areas of the ArchivesSpace application and will provide adequate time for Q&A. A full agenda will be provided to attendees in advance of the training. We hope the feedback we receive from this training will inform future ArchivesSpace training opportunities in a variety of time zones. ArchivesSpace Application Overview and Basic Record Creation Date: February 21, 2023 Time: 1:00pm-4:00pm Australian Eastern Daylight Time (find your local time*) Trainers: Corey Schmidt (University of Georgia) and Jessica Crouch (ArchivesSpace) Registration: https://lyrasis.zoom.us/webinar/register/WN_uwnVJgYbRaaHbxiBWzVOSA *Please note this training occurs in the evening, Monday, February 20th for some time zones. Anyone using ArchivesSpace may register for this training. This training will be recorded. The recording will be made available via the ArchivesSpace website and YouTube channel. You do not need to register for the training to have access to the recordings. We understand this training is outside regular business hours for users in the United States and we ask users not to register if they do not plan to attend live. While this training does not assume any prior experience with ArchivesSpace, attendees are expected to have an understanding of archival workflows and terminology. Due to the large number of attendees, trainers will not be able to troubleshoot individual implementations of ArchivesSpace or provide additional archival instruction beyond the use of ArchivesSpace. For more information about ArchivesSpace, visit our Getting Started webpage or email ArchivesSpaceHome at lyrasis.org Best, Jessica Jessica Dowd Crouch Community Engagement Coordinator for ArchivesSpace jessica.crouch at lyrasis.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From Jessica.Crouch at lyrasis.org Fri Jan 20 09:38:25 2023 From: Jessica.Crouch at lyrasis.org (Jessica Crouch) Date: Fri, 20 Jan 2023 14:38:25 +0000 Subject: [Archivesspace_Users_Group] Training Announcement: Registration open for Spring 2023 trainings In-Reply-To: References: Message-ID: Dear ArchivesSpace users, Registration is now open for our Spring community-wide trainings. To learn more about upcoming community-wide trainings or view information about past trainings, visit https://archivesspace.org/using-archivesspace/trainings. The first training offered this year will be on using Agents, Subjects and Classifications in ArchivesSpace. This training will be offered on March 29, 2023, from 1pm-4pm ET/10am-1pm PT. If you are interested in attending this training, you can register at https://www.lyrasis.org/ASpaceEventMarch. In this training, users will learn how to create, edit, and manage agent and subject records, link agent and subject records to material description records, and create and manage classifications in ArchivesSpace. The second training offered this year will be on using the ArchivesSpace Public User Interface (PUI). This training will be offered on April 19, 2023, from 1pm-4pm ET/10am-1pm PT. If you are interested in attending this training, you can register at https://www.lyrasis.org/ASpaceEventApril. In this training, users will learn how to navigate the ArchivesSpace Public User Interface, use subject and agent records to link related material, how to use classification to group related material and how to make customizations to the Public User Interface to improve the user experience for staff and researchers. Both trainings assume a basic understanding of archival workflows and metadata formats as well as experience navigating the ArchivesSpace staff user interface. If you plan to attend our community-wide trainings without the required background, we encourage you to review the recorded ArchivesSpace basics training available on our Getting Started webpage. These trainings are open to anyone using or interested in ArchivesSpace but ArchivesSpace members are eligible for a discount. If you qualify for the member discount, you will need to reach out to ArchivesSpaceHome at lyrasis.org to receive a member promo code before registering. If you have any questions, please email ArchivesSpaceHome at lyrasis.org. Jessica Dowd Crouch Community Engagement Coordinator for ArchivesSpace jessica.crouch at lyrasis.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From snewhouse at sciencehistory.org Mon Jan 23 17:05:00 2023 From: snewhouse at sciencehistory.org (Newhouse, Sarah) Date: Mon, 23 Jan 2023 22:05:00 +0000 Subject: [Archivesspace_Users_Group] MARC XML import References: <0bc9ca69-de9a-4006-bb1e-b786269e2036.5bc9e4a4-201a-4113-8500-06887445b2b4.de270528-5784-4fef-9eb5-f5c517900c33@emailsignatures365.codetwo.com> <0bc9ca69-de9a-4006-bb1e-b786269e2036.1fe061b8-c04b-4022-ab6c-ea3cb89cd359.e762100f-1501-47ef-8b05-bb5b263406ef@emailsignatures365.codetwo.com> Message-ID: Hi all, I would really appreciate another set of eyes on these two errors I'm getting while testing ingest of MARC XML (resource records) from our library catalog into 3.2: 1. lang_materials/0/language_and_script/language : Property is required but was missing I've tried a few different things here and can't pinpoint the problem. There isn't a MARC field that maps to this ASpace field in the most recent import map (unless I'm misreading it, which is entirely possible - it's been a long day), but native ASpace resource records exported as MARCXML will import back in with language and script data, so I can tell there is a MARC field that maps to this. Based on MARC XML from resource records created in ASpace, I tried adding language info in 040$a or 041$b, but neither resulted in a successful import, just the same error message. Is there a MARC field I'm missing in the import map and/or the ASpace-generated MARC? The Help Center does say this field can be imported as MARC XML, but doesn't say how. 2. notes/0/subnotes/0/content : Property is required but was missing I see that other folks have encountered this error with an index number to point them to the note that's missing content, but lacking one here I'm not sure where to start. And looking at my sample files with my fallible human eyes I don't see any empty fields. Has anyone run into this before? Is this just pointing at my missing language field? If there's a resource or guide I've missed - especially for interpreting import job errors -- please point me to it! MARC XML is attached (with the added 040 and 041 fields) in case it's helpful. Thank you! __________________________________ Sarah Newhouse (she, her, hers) Digital Preservation Archivist Othmer Library of Chemical History t. +1.215.873.8249 Science History Institute Chemistry * Engineering * Life Sciences 315 Chestnut Street * Philadelphia, PA 19106 * U.S.A. Learn about the scientific discoveries that changed our world at sciencehistory.org/learn -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 3536combedleno.xml Type: application/xml Size: 5656 bytes Desc: 3536combedleno.xml URL: From Joshua.D.Shaw at dartmouth.edu Tue Jan 24 06:56:51 2023 From: Joshua.D.Shaw at dartmouth.edu (Joshua D. Shaw) Date: Tue, 24 Jan 2023 11:56:51 +0000 Subject: [Archivesspace_Users_Group] External Solr - Memory Allocation? Message-ID: Hey all We're about to jump to v3.3.1 and I'm wondering if anyone has any suggestions for memory allocation for Solr? Currently we're running 6GB for the entire suite in v3.1.1 and are looking to keep the same overall memory footprint. Wondering if something like a 75/25 split (ie 4GB for AS and 2GB for Solr) would be a reasonable allocation? Or are people finding that Solr is more demanding? Thanks! Joshua ___________________ Joshua Shaw (he, him) Library Web & Application Developer Digital Library Technologies Group Dartmouth College 603.646.0405 -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.harrington at lyrasis.org Tue Jan 24 11:13:20 2023 From: brian.harrington at lyrasis.org (Brian Harrington) Date: Tue, 24 Jan 2023 16:13:20 +0000 Subject: [Archivesspace_Users_Group] MARC XML import In-Reply-To: References: <0bc9ca69-de9a-4006-bb1e-b786269e2036.5bc9e4a4-201a-4113-8500-06887445b2b4.de270528-5784-4fef-9eb5-f5c517900c33@emailsignatures365.codetwo.com> <0bc9ca69-de9a-4006-bb1e-b786269e2036.1fe061b8-c04b-4022-ab6c-ea3cb89cd359.e762100f-1501-47ef-8b05-bb5b263406ef@emailsignatures365.codetwo.com> Message-ID: Hi Sarah, 1. The MARCXML importer gets the language from 008/35-37. I?m not sure what happened to it, but the 008 in your record looks short. Looking at the code, if the language positions are blank the language is set to undefined. But that would only work if there are actual blanks in those positions. 2. Underneath the error messages, you can see a representation of the JSON record that ASpace is trying to load. If you look at ?notes?=>, the first one (position 0) has an empty "subnotes"=>[{"jsonmodel_type"=>"note_text"}]. You can see this is an odd note with a label of ?Publication date?. In this case, ASpace is trying to create a date note from the 260$c, which is blank in your case, so the it creates a bad note. However, it?s only trying to create that note because it can?t get the date from the 008. ASpace only recognizes 008 dates if 008/06 is an i, k, or s, and you have a q there. It probably should recognize the q, but the quick fix would be to change that to an i or an s. I hope this helps. Brian -- Brian Harrington (he/him) Data Migration Specialist LYRASIS brian.harrington at lyrasis.org From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Newhouse, Sarah Date: Monday, January 23, 2023 at 5:05 PM To: Archivesspace Users Group Subject: [Archivesspace_Users_Group] MARC XML import Hi all, I would really appreciate another set of eyes on these two errors I?m getting while testing ingest of MARC XML (resource records) from our library catalog into 3.2: 1. lang_materials/0/language_and_script/language : Property is required but was missing I?ve tried a few different things here and can?t pinpoint the problem. There isn?t a MARC field that maps to this ASpace field in the most recent import map (unless I?m misreading it, which is entirely possible ? it?s been a long day), but native ASpace resource records exported as MARCXML will import back in with language and script data, so I can tell there is a MARC field that maps to this. Based on MARC XML from resource records created in ASpace, I tried adding language info in 040$a or 041$b, but neither resulted in a successful import, just the same error message. Is there a MARC field I?m missing in the import map and/or the ASpace-generated MARC? The Help Center does say this field can be imported as MARC XML, but doesn?t say how. 2. notes/0/subnotes/0/content : Property is required but was missing I see that other folks have encountered this error with an index number to point them to the note that?s missing content, but lacking one here I?m not sure where to start. And looking at my sample files with my fallible human eyes I don?t see any empty fields. Has anyone run into this before? Is this just pointing at my missing language field? If there?s a resource or guide I?ve missed ? especially for interpreting import job errors -- please point me to it! MARC XML is attached (with the added 040 and 041 fields) in case it?s helpful. Thank you! __________________________________ Sarah Newhouse (she, her, hers) Digital Preservation Archivist Othmer Library of Chemical History t. +1.215.873.8249 Science History Institute Chemistry ? Engineering ? Life Sciences 315 Chestnut Street ? Philadelphia, PA 19106 ? U.S.A. Learn about the scientific discoveries that changed our world at sciencehistory.org/learn -------------- next part -------------- An HTML attachment was scrubbed... URL: From snewhouse at sciencehistory.org Tue Jan 24 12:31:11 2023 From: snewhouse at sciencehistory.org (Newhouse, Sarah) Date: Tue, 24 Jan 2023 17:31:11 +0000 Subject: [Archivesspace_Users_Group] MARC XML import In-Reply-To: References: <0bc9ca69-de9a-4006-bb1e-b786269e2036.5bc9e4a4-201a-4113-8500-06887445b2b4.de270528-5784-4fef-9eb5-f5c517900c33@emailsignatures365.codetwo.com> <0bc9ca69-de9a-4006-bb1e-b786269e2036.1fe061b8-c04b-4022-ab6c-ea3cb89cd359.e762100f-1501-47ef-8b05-bb5b263406ef@emailsignatures365.codetwo.com> Message-ID: Aha! Thank you. I'll investigate 008 with our cataloging librarians. __________________________________ Sarah Newhouse (she, her, hers) Digital Preservation Archivist Othmer Library of Chemical History t. +1.215.873.8249 Science History Institute Chemistry * Engineering * Life Sciences 315 Chestnut Street * Philadelphia, PA 19106 * U.S.A. Learn about the scientific discoveries that changed our world at sciencehistory.org/learn From: archivesspace_users_group-bounces at lyralists.lyrasis.org On Behalf Of Brian Harrington Sent: Tuesday, January 24, 2023 11:13 AM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] MARC XML import Hi Sarah, 1. The MARCXML importer gets the language from 008/35-37. I'm not sure what happened to it, but the 008 in your record looks short. Looking at the code, if the language positions are blank the language is set to undefined. But that would only work if there are actual blanks in those positions. 2. Underneath the error messages, you can see a representation of the JSON record that ASpace is trying to load. If you look at "notes"=>, the first one (position 0) has an empty "subnotes"=>[{"jsonmodel_type"=>"note_text"}]. You can see this is an odd note with a label of "Publication date". In this case, ASpace is trying to create a date note from the 260$c, which is blank in your case, so the it creates a bad note. However, it's only trying to create that note because it can't get the date from the 008. ASpace only recognizes 008 dates if 008/06 is an i, k, or s, and you have a q there. It probably should recognize the q, but the quick fix would be to change that to an i or an s. I hope this helps. Brian -- Brian Harrington (he/him) Data Migration Specialist LYRASIS brian.harrington at lyrasis.org From: archivesspace_users_group-bounces at lyralists.lyrasis.org > on behalf of Newhouse, Sarah > Date: Monday, January 23, 2023 at 5:05 PM To: Archivesspace Users Group > Subject: [Archivesspace_Users_Group] MARC XML import Hi all, I would really appreciate another set of eyes on these two errors I'm getting while testing ingest of MARC XML (resource records) from our library catalog into 3.2: 1. lang_materials/0/language_and_script/language : Property is required but was missing I've tried a few different things here and can't pinpoint the problem. There isn't a MARC field that maps to this ASpace field in the most recent import map (unless I'm misreading it, which is entirely possible - it's been a long day), but native ASpace resource records exported as MARCXML will import back in with language and script data, so I can tell there is a MARC field that maps to this. Based on MARC XML from resource records created in ASpace, I tried adding language info in 040$a or 041$b, but neither resulted in a successful import, just the same error message. Is there a MARC field I'm missing in the import map and/or the ASpace-generated MARC? The Help Center does say this field can be imported as MARC XML, but doesn't say how. 2. notes/0/subnotes/0/content : Property is required but was missing I see that other folks have encountered this error with an index number to point them to the note that's missing content, but lacking one here I'm not sure where to start. And looking at my sample files with my fallible human eyes I don't see any empty fields. Has anyone run into this before? Is this just pointing at my missing language field? If there's a resource or guide I've missed - especially for interpreting import job errors -- please point me to it! MARC XML is attached (with the added 040 and 041 fields) in case it's helpful. Thank you! __________________________________ Sarah Newhouse (she, her, hers) Digital Preservation Archivist Othmer Library of Chemical History t. +1.215.873.8249 Science History Institute Chemistry * Engineering * Life Sciences 315 Chestnut Street * Philadelphia, PA 19106 * U.S.A. Learn about the scientific discoveries that changed our world at sciencehistory.org/learn -------------- next part -------------- An HTML attachment was scrubbed... URL: From heberlei at princeton.edu Tue Jan 24 13:14:01 2023 From: heberlei at princeton.edu (Regine I. Heberlein) Date: Tue, 24 Jan 2023 18:14:01 +0000 Subject: [Archivesspace_Users_Group] MARC XML import Message-ID: <5906A179-08D2-4E4A-A966-B8FCF1DD4ECD@princeton.edu> Hi Sarah, Following up on Brian?s suggestion that your 008 looks short?depending on how your MARC-XML is being created, you may want to make sure that the blanks aren?t being collapsed by some sort of whitespace normalization setting. I know I?ve run into this with oXygen before. Hope this helps, Regine Regine Heberlein (she/her) Library IT Data Analyst 609-258-6156 heberlei at princeton.edu **My working day may not be your working day. Please do not feel obliged to reply to this email outside of your normal working hours.** From: on behalf of "Newhouse, Sarah" Reply-To: Archivesspace Users Group Date: Tuesday, January 24, 2023 at 12:32 PM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] MARC XML import Aha! Thank you. I?ll investigate 008 with our cataloging librarians. __________________________________ Sarah Newhouse (she, her, hers) Digital Preservation Archivist Othmer Library of Chemical History t. +1.215.873.8249 Science History Institute Chemistry ? Engineering ? Life Sciences 315 Chestnut Street ? Philadelphia, PA 19106 ? U.S.A. Learn about the scientific discoveries that changed our world at sciencehistory.org/learn From: archivesspace_users_group-bounces at lyralists.lyrasis.org On Behalf Of Brian Harrington Sent: Tuesday, January 24, 2023 11:13 AM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] MARC XML import Hi Sarah, 1. The MARCXML importer gets the language from 008/35-37. I?m not sure what happened to it, but the 008 in your record looks short. Looking at the code, if the language positions are blank the language is set to undefined. But that would only work if there are actual blanks in those positions. 2. Underneath the error messages, you can see a representation of the JSON record that ASpace is trying to load. If you look at ?notes?=>, the first one (position 0) has an empty "subnotes"=>[{"jsonmodel_type"=>"note_text"}]. You can see this is an odd note with a label of ?Publication date?. In this case, ASpace is trying to create a date note from the 260$c, which is blank in your case, so the it creates a bad note. However, it?s only trying to create that note because it can?t get the date from the 008. ASpace only recognizes 008 dates if 008/06 is an i, k, or s, and you have a q there. It probably should recognize the q, but the quick fix would be to change that to an i or an s. I hope this helps. Brian -- Brian Harrington (he/him) Data Migration Specialist LYRASIS brian.harrington at lyrasis.org From: archivesspace_users_group-bounces at lyralists.lyrasis.org > on behalf of Newhouse, Sarah > Date: Monday, January 23, 2023 at 5:05 PM To: Archivesspace Users Group > Subject: [Archivesspace_Users_Group] MARC XML import Hi all, I would really appreciate another set of eyes on these two errors I?m getting while testing ingest of MARC XML (resource records) from our library catalog into 3.2: 1. lang_materials/0/language_and_script/language : Property is required but was missing I?ve tried a few different things here and can?t pinpoint the problem. There isn?t a MARC field that maps to this ASpace field in the most recent import map (unless I?m misreading it, which is entirely possible ? it?s been a long day), but native ASpace resource records exported as MARCXML will import back in with language and script data, so I can tell there is a MARC field that maps to this. Based on MARC XML from resource records created in ASpace, I tried adding language info in 040$a or 041$b, but neither resulted in a successful import, just the same error message. Is there a MARC field I?m missing in the import map and/or the ASpace-generated MARC? The Help Center does say this field can be imported as MARC XML, but doesn?t say how. 2. notes/0/subnotes/0/content : Property is required but was missing I see that other folks have encountered this error with an index number to point them to the note that?s missing content, but lacking one here I?m not sure where to start. And looking at my sample files with my fallible human eyes I don?t see any empty fields. Has anyone run into this before? Is this just pointing at my missing language field? If there?s a resource or guide I?ve missed ? especially for interpreting import job errors -- please point me to it! MARC XML is attached (with the added 040 and 041 fields) in case it?s helpful. Thank you! __________________________________ Sarah Newhouse (she, her, hers) Digital Preservation Archivist Othmer Library of Chemical History t. +1.215.873.8249 Science History Institute Chemistry ? Engineering ? Life Sciences 315 Chestnut Street ? Philadelphia, PA 19106 ? U.S.A. Learn about the scientific discoveries that changed our world at sciencehistory.org/learn -------------- next part -------------- An HTML attachment was scrubbed... URL: From snewhouse at sciencehistory.org Tue Jan 24 15:34:41 2023 From: snewhouse at sciencehistory.org (Newhouse, Sarah) Date: Tue, 24 Jan 2023 20:34:41 +0000 Subject: [Archivesspace_Users_Group] MARC XML import In-Reply-To: <5906A179-08D2-4E4A-A966-B8FCF1DD4ECD@princeton.edu> References: <5906A179-08D2-4E4A-A966-B8FCF1DD4ECD@princeton.edu> <0bc9ca69-de9a-4006-bb1e-b786269e2036.5bc9e4a4-201a-4113-8500-06887445b2b4.7c4783d1-bc23-41cf-bb80-e53b0fbfee54@emailsignatures365.codetwo.com> <0bc9ca69-de9a-4006-bb1e-b786269e2036.1fe061b8-c04b-4022-ab6c-ea3cb89cd359.0cba9d83-b7cb-430c-a417-f3af18a6cef6@emailsignatures365.codetwo.com> Message-ID: Regine, Thank you so much for suggesting this! It was indeed a whitespace issue. Somewhere along the line, one of my XML editors stripped out the whitespaces and I never noticed because 008 is not a field I was paying any close attention to. I suspect this was from bouncing between versions of Oxygen when working from different computers, but can?t be sure. __________________________________ Sarah Newhouse (she, her, hers) Digital Preservation Archivist Othmer Library of Chemical History t. +1.215.873.8249 Science History Institute Chemistry ? Engineering ? Life Sciences 315 Chestnut Street ? Philadelphia, PA 19106 ? U.S.A. Learn about the scientific discoveries that changed our world at sciencehistory.org/learn From: archivesspace_users_group-bounces at lyralists.lyrasis.org On Behalf Of Regine I. Heberlein Sent: Tuesday, January 24, 2023 1:14 PM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] MARC XML import Hi Sarah, Following up on Brian?s suggestion that your 008 looks short?depending on how your MARC-XML is being created, you may want to make sure that the blanks aren?t being collapsed by some sort of whitespace normalization setting. I know I?ve run into this with oXygen before. Hope this helps, Regine Regine Heberlein (she/her) Library IT Data Analyst 609-258-6156 heberlei at princeton.edu **My working day may not be your working day. Please do not feel obliged to reply to this email outside of your normal working hours.** From: > on behalf of "Newhouse, Sarah" > Reply-To: Archivesspace Users Group > Date: Tuesday, January 24, 2023 at 12:32 PM To: Archivesspace Users Group > Subject: Re: [Archivesspace_Users_Group] MARC XML import Aha! Thank you. I?ll investigate 008 with our cataloging librarians. __________________________________ Sarah Newhouse (she, her, hers) Digital Preservation Archivist Othmer Library of Chemical History t. +1.215.873.8249 Science History Institute Chemistry ? Engineering ? Life Sciences 315 Chestnut Street ? Philadelphia, PA 19106 ? U.S.A. Learn about the scientific discoveries that changed our world at sciencehistory.org/learn From: archivesspace_users_group-bounces at lyralists.lyrasis.org > On Behalf Of Brian Harrington Sent: Tuesday, January 24, 2023 11:13 AM To: Archivesspace Users Group > Subject: Re: [Archivesspace_Users_Group] MARC XML import Hi Sarah, 1. The MARCXML importer gets the language from 008/35-37. I?m not sure what happened to it, but the 008 in your record looks short. Looking at the code, if the language positions are blank the language is set to undefined. But that would only work if there are actual blanks in those positions. 2. Underneath the error messages, you can see a representation of the JSON record that ASpace is trying to load. If you look at ?notes?=>, the first one (position 0) has an empty "subnotes"=>[{"jsonmodel_type"=>"note_text"}]. You can see this is an odd note with a label of ?Publication date?. In this case, ASpace is trying to create a date note from the 260$c, which is blank in your case, so the it creates a bad note. However, it?s only trying to create that note because it can?t get the date from the 008. ASpace only recognizes 008 dates if 008/06 is an i, k, or s, and you have a q there. It probably should recognize the q, but the quick fix would be to change that to an i or an s. I hope this helps. Brian -- Brian Harrington (he/him) Data Migration Specialist LYRASIS brian.harrington at lyrasis.org From: archivesspace_users_group-bounces at lyralists.lyrasis.org > on behalf of Newhouse, Sarah > Date: Monday, January 23, 2023 at 5:05 PM To: Archivesspace Users Group > Subject: [Archivesspace_Users_Group] MARC XML import Hi all, I would really appreciate another set of eyes on these two errors I?m getting while testing ingest of MARC XML (resource records) from our library catalog into 3.2: 1. lang_materials/0/language_and_script/language : Property is required but was missing I?ve tried a few different things here and can?t pinpoint the problem. There isn?t a MARC field that maps to this ASpace field in the most recent import map (unless I?m misreading it, which is entirely possible ? it?s been a long day), but native ASpace resource records exported as MARCXML will import back in with language and script data, so I can tell there is a MARC field that maps to this. Based on MARC XML from resource records created in ASpace, I tried adding language info in 040$a or 041$b, but neither resulted in a successful import, just the same error message. Is there a MARC field I?m missing in the import map and/or the ASpace-generated MARC? The Help Center does say this field can be imported as MARC XML, but doesn?t say how. 2. notes/0/subnotes/0/content : Property is required but was missing I see that other folks have encountered this error with an index number to point them to the note that?s missing content, but lacking one here I?m not sure where to start. And looking at my sample files with my fallible human eyes I don?t see any empty fields. Has anyone run into this before? Is this just pointing at my missing language field? If there?s a resource or guide I?ve missed ? especially for interpreting import job errors -- please point me to it! MARC XML is attached (with the added 040 and 041 fields) in case it?s helpful. Thank you! __________________________________ Sarah Newhouse (she, her, hers) Digital Preservation Archivist Othmer Library of Chemical History t. +1.215.873.8249 Science History Institute Chemistry ? Engineering ? Life Sciences 315 Chestnut Street ? Philadelphia, PA 19106 ? U.S.A. Learn about the scientific discoveries that changed our world at sciencehistory.org/learn -------------- next part -------------- An HTML attachment was scrubbed... URL: From gboggio at mitre.org Thu Jan 26 14:06:53 2023 From: gboggio at mitre.org (Jerry Boggio) Date: Thu, 26 Jan 2023 19:06:53 +0000 Subject: [Archivesspace_Users_Group] ArchivesSpace v3.3.1 with External Solr v 9.1.1 Message-ID: Hello Archivesspace Users Group; We are planning to upgrade to version 3.3.1 or ArchivesSpace and were considering upgrading our local installation of External Solr from version 8.11.2 to 9.1.1. Has anyone had experience with running ASpace 3.3.1 with the latest version of External Solr? Is Solr 9.1.1 compatible with ASpace 3.3.1? Look forward to hearing your opinions. Thank you! Gerard (Jerry) Boggio | MITRE Corporation | R124 - Collaboration & Info Management | 781-271-2719 -------------- next part -------------- An HTML attachment was scrubbed... URL: From Joshua.D.Shaw at dartmouth.edu Thu Jan 26 15:38:27 2023 From: Joshua.D.Shaw at dartmouth.edu (Joshua D. Shaw) Date: Thu, 26 Jan 2023 20:38:27 +0000 Subject: [Archivesspace_Users_Group] External Solr - Memory Allocation? In-Reply-To: References: Message-ID: Following up on this. And looking for some advice! Even with the Solr timeout set to 40 minutes, I'm seeing some random Solr Timeout errors, though these do NOT cause the indexer to restart. In the latest test run I see one Solr Timeout for delete and one for commit - both following the PUI indexer run for AOs for the first and largest repo (~630k AOs). The indexer throws the delete timeout error, waits for a loooong time with seemingly no activity, throws the commit timeout error, waits again, and then picks back up as if nothing had gone wrong and continues with the initial index run. All of the index data looks correct (ie correct number of objects in both the staff and PUI). My theory is that the Solr update phase really is taking a super loooong time, but that the data has all been sent to Solr so the timeouts are really just ArchivesSpace waiting for Solr in between indexing one object type and the next and no index data is lost. There are no corresponding log entries in the Solr logs that I can find. I'm running solr 8.11.6 with 4GB and AS 3.3.1 with 4GB. Both bare metal on my laptop, so no container issues that might be at play. Solr memory use peaks at around 3.5GB. I've kept the stock thread and records per thread settings and just upped the timeout (to 2400). I guess the next step is to set the timeout even higher - maybe an hour (3600)? I don't see a reason to run a lower thread or record count, but can certainly try that as well, though I'm not looking forward to the time it will take (the current run takes 21 hours as it is - up from about 15 for 3.1.1) Any advice appreciated! Thanks! Joshua ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Joshua D. Shaw Sent: Tuesday, January 24, 2023 6:56 AM To: Archivesspace Users Group Subject: [Archivesspace_Users_Group] External Solr - Memory Allocation? Hey all We're about to jump to v3.3.1 and I'm wondering if anyone has any suggestions for memory allocation for Solr? Currently we're running 6GB for the entire suite in v3.1.1 and are looking to keep the same overall memory footprint. Wondering if something like a 75/25 split (ie 4GB for AS and 2GB for Solr) would be a reasonable allocation? Or are people finding that Solr is more demanding? Thanks! Joshua ___________________ Joshua Shaw (he, him) Library Web & Application Developer Digital Library Technologies Group Dartmouth College 603.646.0405 -------------- next part -------------- An HTML attachment was scrubbed... URL: From blake.carver at lyrasis.org Thu Jan 26 16:07:03 2023 From: blake.carver at lyrasis.org (Blake Carver) Date: Thu, 26 Jan 2023 21:07:03 +0000 Subject: [Archivesspace_Users_Group] ArchivesSpace v3.3.1 with External Solr v 9.1.1 In-Reply-To: References: Message-ID: It'll be best to stick to 8.x for now. Last I checked ArchivesSpace didn't like 9. ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Jerry Boggio Sent: Thursday, January 26, 2023 2:06 PM To: archivesspace_users_group at lyralists.lyrasis.org Subject: [Archivesspace_Users_Group] ArchivesSpace v3.3.1 with External Solr v 9.1.1 Hello Archivesspace Users Group; We are planning to upgrade to version 3.3.1 or ArchivesSpace and were considering upgrading our local installation of External Solr from version 8.11.2 to 9.1.1. Has anyone had experience with running ASpace 3.3.1 with the latest version of External Solr? Is Solr 9.1.1 compatible with ASpace 3.3.1? Look forward to hearing your opinions. Thank you! Gerard (Jerry) Boggio | MITRE Corporation | R124 - Collaboration & Info Management | 781-271-2719 -------------- next part -------------- An HTML attachment was scrubbed... URL: From blake.carver at lyrasis.org Thu Jan 26 16:12:52 2023 From: blake.carver at lyrasis.org (Blake Carver) Date: Thu, 26 Jan 2023 21:12:52 +0000 Subject: [Archivesspace_Users_Group] External Solr - Memory Allocation? In-Reply-To: References: Message-ID: That's... interesting. That RAM allocation seems fine. That Solr timeout is way higher than I would think is needed. Maybe set the loglevel to debug and see if it spits out something more useful? Maybe you'll be able to see what it's up to during that looooong time. I like your theory on that. ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Joshua D. Shaw Sent: Thursday, January 26, 2023 3:38 PM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] External Solr - Memory Allocation? Following up on this. And looking for some advice! Even with the Solr timeout set to 40 minutes, I'm seeing some random Solr Timeout errors, though these do NOT cause the indexer to restart. In the latest test run I see one Solr Timeout for delete and one for commit - both following the PUI indexer run for AOs for the first and largest repo (~630k AOs). The indexer throws the delete timeout error, waits for a loooong time with seemingly no activity, throws the commit timeout error, waits again, and then picks back up as if nothing had gone wrong and continues with the initial index run. All of the index data looks correct (ie correct number of objects in both the staff and PUI). My theory is that the Solr update phase really is taking a super loooong time, but that the data has all been sent to Solr so the timeouts are really just ArchivesSpace waiting for Solr in between indexing one object type and the next and no index data is lost. There are no corresponding log entries in the Solr logs that I can find. I'm running solr 8.11.6 with 4GB and AS 3.3.1 with 4GB. Both bare metal on my laptop, so no container issues that might be at play. Solr memory use peaks at around 3.5GB. I've kept the stock thread and records per thread settings and just upped the timeout (to 2400). I guess the next step is to set the timeout even higher - maybe an hour (3600)? I don't see a reason to run a lower thread or record count, but can certainly try that as well, though I'm not looking forward to the time it will take (the current run takes 21 hours as it is - up from about 15 for 3.1.1) Any advice appreciated! Thanks! Joshua ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Joshua D. Shaw Sent: Tuesday, January 24, 2023 6:56 AM To: Archivesspace Users Group Subject: [Archivesspace_Users_Group] External Solr - Memory Allocation? Hey all We're about to jump to v3.3.1 and I'm wondering if anyone has any suggestions for memory allocation for Solr? Currently we're running 6GB for the entire suite in v3.1.1 and are looking to keep the same overall memory footprint. Wondering if something like a 75/25 split (ie 4GB for AS and 2GB for Solr) would be a reasonable allocation? Or are people finding that Solr is more demanding? Thanks! Joshua ___________________ Joshua Shaw (he, him) Library Web & Application Developer Digital Library Technologies Group Dartmouth College 603.646.0405 -------------- next part -------------- An HTML attachment was scrubbed... URL: From shand at ap.org Thu Jan 26 17:59:34 2023 From: shand at ap.org (Hand, Sarit) Date: Thu, 26 Jan 2023 22:59:34 +0000 Subject: [Archivesspace_Users_Group] Archival object batch import, excel sheet, URI ID In-Reply-To: <727BED1AE3AF4641B0C9B8E31AE9C81F27033D8E@yunwu2.brooklynmuseum.org> References: <727BED1AE3AF4641B0C9B8E31AE9C81F27033D8E@yunwu2.brooklynmuseum.org> Message-ID: Hello, Did you resolve this issue? I am having the same issue. The resource already has archival objects, series and sub-series. I am now adding file level records to the different sub-series. Since there are not that many series, I might delete them all and do this one batch, but it would be nice if it was clear how to add child AOs to pre-existing AOs. My first attempt, I put the resource URI in the column and under AP ref ID I added the # correlating to the AO I wanted the records to be added to as a child record. I put "2" in the level; I got this error. [cid:image007.png at 01D931AE.24BF07F0] [cid:image001.png at 01D931AD.BDF3E040] I tried putting the AO URI in the ref-id column, nothing in the Ref ID column. It throws an error. [cid:image008.png at 01D931AE.24BF07F0] [cid:image002.png at 01D931AD.BDF3E040] I visited the same page in the Help manual and added a comment requesting clarification (see highlighted text). [cid:image009.png at 01D931AE.A9E57CC0] Thanks Cheers [cid:image003.png at 01D931AD.476D29E0] Sarit Hand Digital Archivist AP Corporate Archives 200 Liberty Street New York, NY 10281 T 212.621.7035 shand at ap.org ap.org [cid:image004.png at 01D931AD.476D29E0] From: archivesspace_users_group-bounces at lyralists.lyrasis.org On Behalf Of Molly Seegers Sent: Wednesday, August 4, 2021 10:38 AM To: archivesspace_users_group at lyralists.lyrasis.org Cc: Aida Garrido ; Aida Garrido Subject: [Archivesspace_Users_Group] Archival object batch import, excel sheet, URI ID [EXTERNAL] Posting this question on behalf of the Archives' Summer Intern, Aida Garrido. Hello, I'm testing the following feature with one archival object: https://archivesspace.atlassian.net/wiki/spaces/ArchivesSpaceUserManual/pages/1173913646/Import+Archival+Objects+from+Excel+or+CSV+File+from+v2.8.1?focusedCommentId=2870837249#comment-2870837249 I want to add an item to a series, i.e. an archival object child to another archival object. I get the following error: Row 6: ERROR Row 6 will not be processed due to errors: Input resource's URI [] does not match row's URI [5244] [cid:image005.png at 01D931AD.476D29E0] I am ingesting at the level of a series, whose internal resource record number from URL is 5244. [cid:image006.png at 01D931AD.476D29E0] That is the exact number I wrote for the resource URI column. For hierarchical relationship I wrote down 2 and for description level I wrote item. >From the error I get, it looks as though the import is not picking up where I'm ingesting from - the brackets are being left blank. What might I be doing wrong? I have also attached the excel sheet I'm importing. Thank you. Molly Seegers Museum Archivist Brooklyn Museum 200 Eastern Parkway, Brooklyn, NY 11238-6052 P 718-501-6119 Pronouns: she/her brooklynmuseum.org Exhibitions | Membership | Shop | Follow @brooklynmuseum The information contained in this communication is intended for the use of the designated recipients named above. If the reader of this communication is not the intended recipient, you are hereby notified that you have received this communication in error, and that any review, dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify The Associated Press immediately by telephone at +1-212-621-1500 and delete this email. Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image003.png Type: image/png Size: 1094 bytes Desc: image003.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image004.png Type: image/png Size: 5548 bytes Desc: image004.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image005.png Type: image/png Size: 39189 bytes Desc: image005.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image006.png Type: image/png Size: 48251 bytes Desc: image006.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 7635 bytes Desc: image001.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.png Type: image/png Size: 6701 bytes Desc: image002.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image007.png Type: image/png Size: 4453 bytes Desc: image007.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image008.png Type: image/png Size: 4688 bytes Desc: image008.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image009.png Type: image/png Size: 16539 bytes Desc: image009.png URL: From Joshua.D.Shaw at dartmouth.edu Thu Jan 26 18:02:10 2023 From: Joshua.D.Shaw at dartmouth.edu (Joshua D. Shaw) Date: Thu, 26 Jan 2023 23:02:10 +0000 Subject: [Archivesspace_Users_Group] External Solr - Memory Allocation? In-Reply-To: References: Message-ID: Thanks, Blake! I'm running default config values for the AS log levels so they are all set to 'debug'. I took a closer look, and the timeout message happens exactly after the timeout amount I set (as you'd expect). Interestingly, Solr is in the middle of deleting documents when it goes silent I, [2023-01-26T09:18:40.357101 #78764] INFO -- : Thread-3384: Deleted 100 documents: # .... 40 minutes pass with all the other AS log chatter ... E, [2023-01-26T09:58:40.400971 #78764] ERROR -- : Thread-3384: SolrIndexerError when deleting records: Timeout error with POST {....} I, [2023-01-26T09:58:40.410522 #78764] INFO -- : Thread-3384: Deleted 100 documents: # This continuing delete phase goes on for a bit until it stops logging batch deletes. I, [2023-01-26T09:59:11.734200 #78764] INFO -- : Thread-3384: Deleted 9 documents: # .... 40 minutes pass with all the other AS log chatter ... And then the commit error pops up E, [2023-01-26T10:39:11.746166 #78764] ERROR -- : Thread-3384: SolrIndexerError when committing: Timeout error with POST {"commit":{"softCommit":false}}. Then after some more time I, [2023-01-26T11:06:35.678926 #78764] INFO -- : Thread-3384: Deleted 186992 documents: # .... This all seems to indicate to me that the commit phase is taking an inordinate amount of time (almost 2 hours - maybe that's what I need to set the timeout to?). After that, the indexer starts the 2nd repo I, [2023-01-26T11:06:35.765797 #78764] INFO -- : Thread-3384: PUI Indexer [2023-01-26 11:06:35 -0500] Indexed 2 additional PUI records in repository Sherman .... The indexer waits for a looong time with no timeout and no messaging - even though this is a tiny repo - and then starts the 3rd repo I, [2023-01-26T11:31:32.795602 #78764] INFO -- : Thread-3384: PUI Indexer [2023-01-26 11:31:32 -0500] Indexed 188 additional PUI records in repository Rauner-XO And then the indexer starts the 4th repo soon after and seems to go on to complete normally I, [2023-01-26T11:31:33.369369 #78764] INFO -- : Thread-3384: PUI Indexer [2023-01-26 11:31:33 -0500] ~~~ Indexed 25 of 74785 archival_object records in repository thedartmouth The Solr logs indicate that Solr is working this entire time doing adds and deletes. For example in one of the quiet phases: 2023-01-26 10:23:35.928 INFO (qtp2101153819-523) [ x:archivesspace] o.a.s.u.p.LogUpdateProcessorFactory [archivesspace] webapp=/solr path=/update params={}{add=... 2023-01-26 10:23:38.195 INFO (qtp2101153819-468) [ x:archivesspace] o.a.s.u.p.LogUpdateProcessorFactory [archivesspace] webapp=/solr path=/update params={}{deleteByQuery=... So I'm falling back to this just being super? slow for some reason. I do have some custom indexer addons, but I'm not sure why the PUI indexer would become so much slower (21 hours) when the Staff indexer completes in a normal amount of time (a little under 6 hours). For previous versions this hasn't been quite that different (6hrs vs about 13hrs). We do have some collections that are quite large (10s of thousands of AOs), so maybe that's part of the issue. I haven't checked to see if the PUI indexer is gathering that much more data (and traversing the tree more times - maybe?) than it was in 3.1.1, but that's on my 'to check' list. Joshua ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Blake Carver Sent: Thursday, January 26, 2023 4:12 PM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] External Solr - Memory Allocation? That's... interesting. That RAM allocation seems fine. That Solr timeout is way higher than I would think is needed. Maybe set the loglevel to debug and see if it spits out something more useful? Maybe you'll be able to see what it's up to during that looooong time. I like your theory on that. ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Joshua D. Shaw Sent: Thursday, January 26, 2023 3:38 PM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] External Solr - Memory Allocation? Following up on this. And looking for some advice! Even with the Solr timeout set to 40 minutes, I'm seeing some random Solr Timeout errors, though these do NOT cause the indexer to restart. In the latest test run I see one Solr Timeout for delete and one for commit - both following the PUI indexer run for AOs for the first and largest repo (~630k AOs). The indexer throws the delete timeout error, waits for a loooong time with seemingly no activity, throws the commit timeout error, waits again, and then picks back up as if nothing had gone wrong and continues with the initial index run. All of the index data looks correct (ie correct number of objects in both the staff and PUI). My theory is that the Solr update phase really is taking a super loooong time, but that the data has all been sent to Solr so the timeouts are really just ArchivesSpace waiting for Solr in between indexing one object type and the next and no index data is lost. There are no corresponding log entries in the Solr logs that I can find. I'm running solr 8.11.6 with 4GB and AS 3.3.1 with 4GB. Both bare metal on my laptop, so no container issues that might be at play. Solr memory use peaks at around 3.5GB. I've kept the stock thread and records per thread settings and just upped the timeout (to 2400). I guess the next step is to set the timeout even higher - maybe an hour (3600)? I don't see a reason to run a lower thread or record count, but can certainly try that as well, though I'm not looking forward to the time it will take (the current run takes 21 hours as it is - up from about 15 for 3.1.1) Any advice appreciated! Thanks! Joshua ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Joshua D. Shaw Sent: Tuesday, January 24, 2023 6:56 AM To: Archivesspace Users Group Subject: [Archivesspace_Users_Group] External Solr - Memory Allocation? Hey all We're about to jump to v3.3.1 and I'm wondering if anyone has any suggestions for memory allocation for Solr? Currently we're running 6GB for the entire suite in v3.1.1 and are looking to keep the same overall memory footprint. Wondering if something like a 75/25 split (ie 4GB for AS and 2GB for Solr) would be a reasonable allocation? Or are people finding that Solr is more demanding? Thanks! Joshua ___________________ Joshua Shaw (he, him) Library Web & Application Developer Digital Library Technologies Group Dartmouth College 603.646.0405 -------------- next part -------------- An HTML attachment was scrubbed... URL: From blake.carver at lyrasis.org Fri Jan 27 09:01:10 2023 From: blake.carver at lyrasis.org (Blake Carver) Date: Fri, 27 Jan 2023 14:01:10 +0000 Subject: [Archivesspace_Users_Group] External Solr - Memory Allocation? In-Reply-To: References: Message-ID: > I'm running default config values for the AS log levels so they are all set to 'debug'. I'm seeing "INFO" and not "DEBUG" there. > Deleted 186992 documents How much is in the deleted_records table? Try truncating that. ArchivesSpace is going and deleting anything in that table. > So I'm falling back to this just being super? slow for some reason. Could be some complex records, could be there's way too much in the deleted table. > but I'm not sure why the PUI indexer would become so much slower (21 hours) Yep, sounds about right. The PUI is slow. > We do have some collections that are quite large (10s of thousands of AOs), so maybe that's part of the issue. No doubt that's slowing it down too. ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Joshua D. Shaw Sent: Thursday, January 26, 2023 6:02 PM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] External Solr - Memory Allocation? Thanks, Blake! I'm running default config values for the AS log levels so they are all set to 'debug'. I took a closer look, and the timeout message happens exactly after the timeout amount I set (as you'd expect). Interestingly, Solr is in the middle of deleting documents when it goes silent I, [2023-01-26T09:18:40.357101 #78764] INFO -- : Thread-3384: Deleted 100 documents: # .... 40 minutes pass with all the other AS log chatter ... E, [2023-01-26T09:58:40.400971 #78764] ERROR -- : Thread-3384: SolrIndexerError when deleting records: Timeout error with POST {....} I, [2023-01-26T09:58:40.410522 #78764] INFO -- : Thread-3384: Deleted 100 documents: # This continuing delete phase goes on for a bit until it stops logging batch deletes. I, [2023-01-26T09:59:11.734200 #78764] INFO -- : Thread-3384: Deleted 9 documents: # .... 40 minutes pass with all the other AS log chatter ... And then the commit error pops up E, [2023-01-26T10:39:11.746166 #78764] ERROR -- : Thread-3384: SolrIndexerError when committing: Timeout error with POST {"commit":{"softCommit":false}}. Then after some more time I, [2023-01-26T11:06:35.678926 #78764] INFO -- : Thread-3384: Deleted 186992 documents: # .... This all seems to indicate to me that the commit phase is taking an inordinate amount of time (almost 2 hours - maybe that's what I need to set the timeout to?). After that, the indexer starts the 2nd repo I, [2023-01-26T11:06:35.765797 #78764] INFO -- : Thread-3384: PUI Indexer [2023-01-26 11:06:35 -0500] Indexed 2 additional PUI records in repository Sherman .... The indexer waits for a looong time with no timeout and no messaging - even though this is a tiny repo - and then starts the 3rd repo I, [2023-01-26T11:31:32.795602 #78764] INFO -- : Thread-3384: PUI Indexer [2023-01-26 11:31:32 -0500] Indexed 188 additional PUI records in repository Rauner-XO And then the indexer starts the 4th repo soon after and seems to go on to complete normally I, [2023-01-26T11:31:33.369369 #78764] INFO -- : Thread-3384: PUI Indexer [2023-01-26 11:31:33 -0500] ~~~ Indexed 25 of 74785 archival_object records in repository thedartmouth The Solr logs indicate that Solr is working this entire time doing adds and deletes. For example in one of the quiet phases: 2023-01-26 10:23:35.928 INFO (qtp2101153819-523) [ x:archivesspace] o.a.s.u.p.LogUpdateProcessorFactory [archivesspace] webapp=/solr path=/update params={}{add=... 2023-01-26 10:23:38.195 INFO (qtp2101153819-468) [ x:archivesspace] o.a.s.u.p.LogUpdateProcessorFactory [archivesspace] webapp=/solr path=/update params={}{deleteByQuery=... So I'm falling back to this just being super? slow for some reason. I do have some custom indexer addons, but I'm not sure why the PUI indexer would become so much slower (21 hours) when the Staff indexer completes in a normal amount of time (a little under 6 hours). For previous versions this hasn't been quite that different (6hrs vs about 13hrs). We do have some collections that are quite large (10s of thousands of AOs), so maybe that's part of the issue. I haven't checked to see if the PUI indexer is gathering that much more data (and traversing the tree more times - maybe?) than it was in 3.1.1, but that's on my 'to check' list. Joshua ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Blake Carver Sent: Thursday, January 26, 2023 4:12 PM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] External Solr - Memory Allocation? That's... interesting. That RAM allocation seems fine. That Solr timeout is way higher than I would think is needed. Maybe set the loglevel to debug and see if it spits out something more useful? Maybe you'll be able to see what it's up to during that looooong time. I like your theory on that. ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Joshua D. Shaw Sent: Thursday, January 26, 2023 3:38 PM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] External Solr - Memory Allocation? Following up on this. And looking for some advice! Even with the Solr timeout set to 40 minutes, I'm seeing some random Solr Timeout errors, though these do NOT cause the indexer to restart. In the latest test run I see one Solr Timeout for delete and one for commit - both following the PUI indexer run for AOs for the first and largest repo (~630k AOs). The indexer throws the delete timeout error, waits for a loooong time with seemingly no activity, throws the commit timeout error, waits again, and then picks back up as if nothing had gone wrong and continues with the initial index run. All of the index data looks correct (ie correct number of objects in both the staff and PUI). My theory is that the Solr update phase really is taking a super loooong time, but that the data has all been sent to Solr so the timeouts are really just ArchivesSpace waiting for Solr in between indexing one object type and the next and no index data is lost. There are no corresponding log entries in the Solr logs that I can find. I'm running solr 8.11.6 with 4GB and AS 3.3.1 with 4GB. Both bare metal on my laptop, so no container issues that might be at play. Solr memory use peaks at around 3.5GB. I've kept the stock thread and records per thread settings and just upped the timeout (to 2400). I guess the next step is to set the timeout even higher - maybe an hour (3600)? I don't see a reason to run a lower thread or record count, but can certainly try that as well, though I'm not looking forward to the time it will take (the current run takes 21 hours as it is - up from about 15 for 3.1.1) Any advice appreciated! Thanks! Joshua ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Joshua D. Shaw Sent: Tuesday, January 24, 2023 6:56 AM To: Archivesspace Users Group Subject: [Archivesspace_Users_Group] External Solr - Memory Allocation? Hey all We're about to jump to v3.3.1 and I'm wondering if anyone has any suggestions for memory allocation for Solr? Currently we're running 6GB for the entire suite in v3.1.1 and are looking to keep the same overall memory footprint. Wondering if something like a 75/25 split (ie 4GB for AS and 2GB for Solr) would be a reasonable allocation? Or are people finding that Solr is more demanding? Thanks! Joshua ___________________ Joshua Shaw (he, him) Library Web & Application Developer Digital Library Technologies Group Dartmouth College 603.646.0405 -------------- next part -------------- An HTML attachment was scrubbed... URL: From ph448 at cam.ac.uk Sat Jan 28 03:51:19 2023 From: ph448 at cam.ac.uk (Peter Heiner) Date: Sat, 28 Jan 2023 08:51:19 +0000 Subject: [Archivesspace_Users_Group] External Solr - Memory Allocation? In-Reply-To: References: Message-ID: <20230128085118.tddyutnoq2y4wlhw@sparkly> Joshua D. Shaw wrote on 2023-01-26 23:02:10: > Thanks, Blake! > > I'm running default config values for the AS log levels so they are all set to 'debug'. I took a closer look, and the timeout message happens exactly after the timeout amount I set (as you'd expect). Interestingly, Solr is in the middle of deleting documents when it goes silent > > I, [2023-01-26T09:18:40.357101 #78764] INFO -- : Thread-3384: Deleted 100 documents: # > > .... 40 minutes pass with all the other AS log chatter ... > > E, [2023-01-26T09:58:40.400971 #78764] ERROR -- : Thread-3384: SolrIndexerError when deleting records: Timeout error with POST {....} > I, [2023-01-26T09:58:40.410522 #78764] INFO -- : Thread-3384: Deleted 100 documents: # > > This continuing delete phase goes on for a bit until it stops logging batch deletes. > > I, [2023-01-26T09:59:11.734200 #78764] INFO -- : Thread-3384: Deleted 9 documents: # > > .... 40 minutes pass with all the other AS log chatter ... And then the commit error pops up > > E, [2023-01-26T10:39:11.746166 #78764] ERROR -- : Thread-3384: SolrIndexerError when committing: > Timeout error with POST {"commit":{"softCommit":false}}. > > Then after some more time > > I, [2023-01-26T11:06:35.678926 #78764] INFO -- : Thread-3384: Deleted 186992 documents: # > > .... This all seems to indicate to me that the commit phase is taking an inordinate amount of time (almost 2 hours - maybe that's what I need to set the timeout to?). After that, the indexer starts the 2nd repo We're experiencing the exact same issue at least in the largest of our 30-odd repositories: I, [2023-01-28T07:24:48.015356 #2036632] INFO -- : Thread-2006: Staff Indexer [2023-01-28 07:24:48 +0000] ~~~ Indexed 536300 of 587664 archival_object records in repository CUL E, [2023-01-28T07:25:47.217953 #2036632] ERROR -- : Thread-2016: SolrIndexerError when committing: Timeout error with POST {"commit":{"softCommit":false}}. We've had this problem from the start but were unable to dig deeper because our in-house monitoring wasn't granular or even capable enough. Our crude solution was using an external Solr since 2.5 and an external indexer since around 2.7, and periodic restarts out of hours, but we've started getting problems despite that. Our Solr is allocated 6GB of memory and timeout is set to 1200 seconds, but the problem is that searches in AS fail during the wait and that makes AS unusable, so we're reluctant to increase that. Why does one even have to run a periodic indexer? Aren't there guarantees in the system that updates are seen through to the index in realtime, do bulk updates not trigger a refresh of updated records selectively? Reading the code seems to suggest that updates are queued until processed, does AS need a more durable queue? Thanks, p From brian.hoffman at lyrasis.org Sat Jan 28 11:11:15 2023 From: brian.hoffman at lyrasis.org (Brian Hoffman) Date: Sat, 28 Jan 2023 16:11:15 +0000 Subject: [Archivesspace_Users_Group] External Solr - Memory Allocation? In-Reply-To: <20230128085118.tddyutnoq2y4wlhw@sparkly> References: <20230128085118.tddyutnoq2y4wlhw@sparkly> Message-ID: << Why does one even have to run a periodic indexer? Aren't there guarantees in the system that updates are seen through to the index in realtime, do bulk updates not trigger a refresh of updated records selectively? Reading the code seems to suggest that updates are queued until processed, does AS need a more durable queue? >> In theory, if your index is ?up to date? (according to the indexer_state directory) the periodic indexer should have no work to do. I think this is part of a class of problems that arise when for some reason the periodic indexer cannot get through its workload and therefore tries and tries again. That is what happens, for example, when a MySQL database contains a record with a bad timestamp due to DST. If someone could file a JIRA issue with as much info as possible for recreating the problem (and maybe someone who could be contacted to supply a database copy) then it could probably be prioritized and addressed. From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Peter Heiner Date: Saturday, January 28, 2023 at 3:51 AM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] External Solr - Memory Allocation? Joshua D. Shaw wrote on 2023-01-26 23:02:10: > Thanks, Blake! > > I'm running default config values for the AS log levels so they are all set to 'debug'. I took a closer look, and the timeout message happens exactly after the timeout amount I set (as you'd expect). Interestingly, Solr is in the middle of deleting documents when it goes silent > > I, [2023-01-26T09:18:40.357101 #78764] INFO -- : Thread-3384: Deleted 100 documents: # > > .... 40 minutes pass with all the other AS log chatter ... > > E, [2023-01-26T09:58:40.400971 #78764] ERROR -- : Thread-3384: SolrIndexerError when deleting records: Timeout error with POST {....} > I, [2023-01-26T09:58:40.410522 #78764] INFO -- : Thread-3384: Deleted 100 documents: # > > This continuing delete phase goes on for a bit until it stops logging batch deletes. > > I, [2023-01-26T09:59:11.734200 #78764] INFO -- : Thread-3384: Deleted 9 documents: # > > .... 40 minutes pass with all the other AS log chatter ... And then the commit error pops up > > E, [2023-01-26T10:39:11.746166 #78764] ERROR -- : Thread-3384: SolrIndexerError when committing: > Timeout error with POST {"commit":{"softCommit":false}}. > > Then after some more time > > I, [2023-01-26T11:06:35.678926 #78764] INFO -- : Thread-3384: Deleted 186992 documents: # > > .... This all seems to indicate to me that the commit phase is taking an inordinate amount of time (almost 2 hours - maybe that's what I need to set the timeout to?). After that, the indexer starts the 2nd repo We're experiencing the exact same issue at least in the largest of our 30-odd repositories: I, [2023-01-28T07:24:48.015356 #2036632] INFO -- : Thread-2006: Staff Indexer [2023-01-28 07:24:48 +0000] ~~~ Indexed 536300 of 587664 archival_object records in repository CUL E, [2023-01-28T07:25:47.217953 #2036632] ERROR -- : Thread-2016: SolrIndexerError when committing: Timeout error with POST {"commit":{"softCommit":false}}. We've had this problem from the start but were unable to dig deeper because our in-house monitoring wasn't granular or even capable enough. Our crude solution was using an external Solr since 2.5 and an external indexer since around 2.7, and periodic restarts out of hours, but we've started getting problems despite that. Our Solr is allocated 6GB of memory and timeout is set to 1200 seconds, but the problem is that searches in AS fail during the wait and that makes AS unusable, so we're reluctant to increase that. Why does one even have to run a periodic indexer? Aren't there guarantees in the system that updates are seen through to the index in realtime, do bulk updates not trigger a refresh of updated records selectively? Reading the code seems to suggest that updates are queued until processed, does AS need a more durable queue? Thanks, p _______________________________________________ Archivesspace_Users_Group mailing list Archivesspace_Users_Group at lyralists.lyrasis.org http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew.morrison at bodleian.ox.ac.uk Mon Jan 30 04:46:06 2023 From: andrew.morrison at bodleian.ox.ac.uk (Andrew Morrison) Date: Mon, 30 Jan 2023 09:46:06 +0000 Subject: [Archivesspace_Users_Group] External Solr - Memory Allocation? In-Reply-To: References: Message-ID: <9c415928-1de3-19fc-c32a-b73e2a99c8ee@bodleian.ox.ac.uk> Truncating the "deleted_records" table will prevent the OAI-PMH service from being able to send out deletion notifications. Maybe it is worth trying on a testing system, but probably not a good idea on a production system. Also note that is this... > Deleted 186992 documents ... is not logging the deletion of deleted records. It is logging the deletion of Solr documents for unpublished records. But few, if any, exist to be deleted. ArchivesSpace sent 186992 IDs to Solr to delete, just in case any of them were unpublished immediately before this index run. Solr returned a 200 OK response, even if none were found, so ArchivesSpace reports them as all deleted. The other logged deletions, before that, are likewise not "real" 99.9% of the time. These batches... > Deleted 100 documents ...are when ArchivesSpace tells Solr to delete the tree nodes for all archival objects without children, just in case any of them had children before this index run. Only the PUIIndexer does this, which is part of why it is slower (it is also allocated fewer threads in default config.) Again, 99.99% of the time there's nothing for Solr to delete, but it has to search its indexes for them anyway. And these... > Deleted 25 documents ...are the deletion of URIs in the deleted_records table. But most of those were deleted long ago by previous index runs. Again, it is just in case any new ones were recently deleted (and even those were probably deleted by the RealtimeIndexer.) This "belt and braces" approach prevents a few stray records remaining in the PUI when they've been deleted or unpublished, but it seems to be the cause of the longest wait times for commits when re-indexing large repositories. Maybe something has changed in newer versions of Solr to make this process slower, possibly specifically for deletions? Andrew. On 27/01/2023 14:01, Blake Carver wrote: > > I'm running default config values for the AS log levels so they are > all set to 'debug'. > > I'm seeing "INFO" and not "DEBUG" there. > > > Deleted 186992 documents > > How much is in the deleted_records table? Try truncating that. > ArchivesSpace is going and deleting anything in that table. > > > So I'm falling back to this just being super? slow for some reason. > > Could be some complex records, could be there's way too much in the > deleted table. > > > but I'm not sure why the PUI indexer would become so much slower (21 > hours) > > Yep, sounds about right. The PUI is slow. > > > We do have some collections that are quite large (10s of thousands > of AOs), so maybe that's part of the issue. > > No doubt that's slowing it down too. > > > ------------------------------------------------------------------------ > *From:* archivesspace_users_group-bounces at lyralists.lyrasis.org > on behalf of > Joshua D. Shaw > *Sent:* Thursday, January 26, 2023 6:02 PM > *To:* Archivesspace Users Group > > *Subject:* Re: [Archivesspace_Users_Group] External Solr - Memory > Allocation? > Thanks, Blake! > > I'm running default config values for the AS log levels so they are > all set to 'debug'. I took a closer look, and the timeout message > happens exactly after the timeout amount I set (as you'd expect). > Interestingly, Solr is in the middle of deleting documents when it > goes silent > > I, [2023-01-26T09:18:40.357101 #78764] ?INFO -- : Thread-3384: Deleted > 100 documents: # > > .... 40 minutes pass with all the other AS log chatter ... > > E, [2023-01-26T09:58:40.400971 #78764] ERROR -- : Thread-3384: > SolrIndexerError when deleting records: Timeout error with ?POST {....} > I, [2023-01-26T09:58:40.410522 #78764] ?INFO -- : Thread-3384: Deleted > 100 documents: # > > This continuing delete phase goes on for a bit until it stops logging > batch deletes. > > I, [2023-01-26T09:59:11.734200 #78764] ?INFO -- : Thread-3384: Deleted > 9 documents: # > > .... 40 minutes pass with all the other AS log chatter ... And then > the commit error pops up > > E, [2023-01-26T10:39:11.746166 #78764] ERROR -- : Thread-3384: > SolrIndexerError when committing: > Timeout error with ?POST {"commit":{"softCommit":false}}. > > Then after some more time > > I, [2023-01-26T11:06:35.678926 #78764] ?INFO -- : Thread-3384: Deleted > 186992 documents: # > > .... This all seems to indicate to me that the commit phase is taking > an inordinate amount of time (almost 2 hours - maybe that's what I > need to set the timeout to?). After that, the indexer starts the 2nd repo > > I, [2023-01-26T11:06:35.765797 #78764] ?INFO -- : Thread-3384: PUI > Indexer [2023-01-26 11:06:35 -0500] Indexed 2 additional PUI records > in repository Sherman > > .... The indexer waits for a looong time with no timeout and no > messaging - even though this is a tiny repo - and then starts the 3rd repo > > I, [2023-01-26T11:31:32.795602 #78764] ?INFO -- : Thread-3384: PUI > Indexer [2023-01-26 11:31:32 -0500] Indexed 188 additional PUI records > in repository Rauner-XO > > And then the indexer starts the 4th repo soon after and seems to go on > to complete normally > > I, [2023-01-26T11:31:33.369369 #78764] ?INFO -- : Thread-3384: PUI > Indexer [2023-01-26 11:31:33 -0500] ~~~ Indexed 25 of 74785 > archival_object records in repository thedartmouth > > The Solr logs indicate?that Solr is working this entire time doing > adds and deletes. For example in one of the quiet phases: > > 2023-01-26 10:23:35.928 INFO ?(qtp2101153819-523) [ ? x:archivesspace] > o.a.s.u.p.LogUpdateProcessorFactory [archivesspace] ?webapp=/solr > path=/update params={}{add=... > 2023-01-26 10:23:38.195 INFO ?(qtp2101153819-468) [ ? x:archivesspace] > o.a.s.u.p.LogUpdateProcessorFactory [archivesspace] ?webapp=/solr > path=/update params={}{deleteByQuery=... > > So I'm falling back to this just being *super*? slow for some reason. > I do have some custom indexer addons, but I'm not sure why the PUI > indexer would become *so* much slower (21 hours) when the Staff > indexer completes in a normal amount of time (a little under 6 hours). > For previous versions this hasn't?been quite that different (6hrs vs > about 13hrs). We do have some collections that are quite large (10s of > thousands of AOs), so maybe that's part of the issue. > > I haven't checked to see if the PUI indexer is gathering that much > more data (and traversing the tree more times - maybe?) than it was in > 3.1.1, but that's on my 'to check' list. > > Joshua > ------------------------------------------------------------------------ > *From:* archivesspace_users_group-bounces at lyralists.lyrasis.org > on behalf of > Blake Carver > *Sent:* Thursday, January 26, 2023 4:12 PM > *To:* Archivesspace Users Group > > *Subject:* Re: [Archivesspace_Users_Group] External Solr - Memory > Allocation? > That's... interesting. > > That RAM allocation seems fine. That Solr timeout is way higher?than I > would think is needed. > > Maybe set the loglevel to debug and see if it spits out something more > useful? Maybe you'll be able to see what it's up to during that > looooong time. I like your theory on that. > > > ------------------------------------------------------------------------ > *From:* archivesspace_users_group-bounces at lyralists.lyrasis.org > on behalf of > Joshua D. Shaw > *Sent:* Thursday, January 26, 2023 3:38 PM > *To:* Archivesspace Users Group > > *Subject:* Re: [Archivesspace_Users_Group] External Solr - Memory > Allocation? > Following up on this. And looking for some advice! > > Even with the Solr timeout set to 40 minutes, I'm seeing some random > Solr Timeout errors, though these do *NOT* cause the indexer to > restart. In the latest test run I see one Solr Timeout for delete and > one for commit - both following the PUI indexer run for AOs for the > first and largest repo (~630k AOs). > > The indexer throws the delete timeout error, waits for a loooong time > with seemingly no activity, throws the commit timeout error, waits > again, and then picks back up as if nothing had gone wrong and > continues with the initial index run. All of the index data looks > correct (ie correct number of objects in both the staff and PUI). > > My theory?is that the Solr update phase really is taking a super > loooong time, but that the data has all been sent to Solr so the > timeouts are really just ArchivesSpace waiting for Solr in between > indexing one object type and the next and no index data is lost. > > There are no corresponding log entries in the Solr logs that I can find. > > I'm running solr 8.11.6 with 4GB and AS 3.3.1 with 4GB. Both bare > metal on my laptop, so no container issues that might be at play. Solr > memory use peaks at around 3.5GB. > > I've kept the stock thread and records per thread settings and just > upped the timeout (to 2400). I guess the next step is to set the > timeout even higher - maybe an hour (3600)? I don't see a reason to > run a lower thread or record count, but can certainly try that as > well, though I'm not looking forward to the time it will take (the > current run takes 21 hours as it is - up from about 15 for 3.1.1) > > Any advice appreciated! > > Thanks! > Joshua > > ------------------------------------------------------------------------ > *From:* archivesspace_users_group-bounces at lyralists.lyrasis.org > on behalf of > Joshua D. Shaw > *Sent:* Tuesday, January 24, 2023 6:56 AM > *To:* Archivesspace Users Group > > *Subject:* [Archivesspace_Users_Group] External Solr - Memory Allocation? > Hey all > > We're about to jump to v3.3.1 and I'm wondering if anyone has any > suggestions for memory allocation for Solr? > > Currently we're running 6GB for the entire suite in v3.1.1 and are > looking to keep the same overall memory footprint. Wondering if > something like a 75/25 split (ie 4GB for AS and 2GB for Solr) would be > a reasonable allocation? Or are people finding that Solr is more > demanding? > > Thanks! > Joshua > > ___________________ > Joshua Shaw (he, him) > Library Web & Application Developer > Digital Library Technologies Group > Dartmouth College > 603.646.0405 > > _______________________________________________ > Archivesspace_Users_Group mailing list > Archivesspace_Users_Group at lyralists.lyrasis.org > http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group -------------- next part -------------- An HTML attachment was scrubbed... URL: From Joshua.D.Shaw at dartmouth.edu Mon Jan 30 14:13:20 2023 From: Joshua.D.Shaw at dartmouth.edu (Joshua D. Shaw) Date: Mon, 30 Jan 2023 19:13:20 +0000 Subject: [Archivesspace_Users_Group] External Solr - Memory Allocation? In-Reply-To: <9c415928-1de3-19fc-c32a-b73e2a99c8ee@bodleian.ox.ac.uk> References: <9c415928-1de3-19fc-c32a-b73e2a99c8ee@bodleian.ox.ac.uk> Message-ID: Thanks, Andrew & Blake, et al! One of my thoughts was to run the index against an earlier 8.x version of Solr to see if there are any significant changes in the indexer time. Failing that, I think my only option is to increase the timeout to about 2hrs. @Brian - I don't know if this is related to the DST issue since the indexer does not? restart the index round for me - like it typically does in those DST circumstances. To elaborate on a couple of specific thoughts Blake brought up. 1. We only have about 31.5K records in the deleted table, but as Andrew pointed out, I'm pretty sure those aren't really relevant here. 2. I may be reading the configs wrong, but it looks like 'debug' is the default level (https://github.com/archivesspace/archivesspace/blob/v3.3.1/common/config/config-defaults.rb#L53) 3. We do have some very 'wide' collections with many series and many files under each series. Though these collections tend not to be very 'deep' - ie only 3-4 levels. I'd be interested to hear if other institutions with similar collections also experience the same timeout issues. Thanks, Peter, for chiming in. Will update if/when I find out more. Joshua ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Andrew Morrison Sent: Monday, January 30, 2023 4:46 AM To: archivesspace_users_group at lyralists.lyrasis.org Subject: Re: [Archivesspace_Users_Group] External Solr - Memory Allocation? Truncating the "deleted_records" table will prevent the OAI-PMH service from being able to send out deletion notifications. Maybe it is worth trying on a testing system, but probably not a good idea on a production system. Also note that is this... > Deleted 186992 documents ... is not logging the deletion of deleted records. It is logging the deletion of Solr documents for unpublished records. But few, if any, exist to be deleted. ArchivesSpace sent 186992 IDs to Solr to delete, just in case any of them were unpublished immediately before this index run. Solr returned a 200 OK response, even if none were found, so ArchivesSpace reports them as all deleted. The other logged deletions, before that, are likewise not "real" 99.9% of the time. These batches... > Deleted 100 documents ...are when ArchivesSpace tells Solr to delete the tree nodes for all archival objects without children, just in case any of them had children before this index run. Only the PUIIndexer does this, which is part of why it is slower (it is also allocated fewer threads in default config.) Again, 99.99% of the time there's nothing for Solr to delete, but it has to search its indexes for them anyway. And these... > Deleted 25 documents ...are the deletion of URIs in the deleted_records table. But most of those were deleted long ago by previous index runs. Again, it is just in case any new ones were recently deleted (and even those were probably deleted by the RealtimeIndexer.) This "belt and braces" approach prevents a few stray records remaining in the PUI when they've been deleted or unpublished, but it seems to be the cause of the longest wait times for commits when re-indexing large repositories. Maybe something has changed in newer versions of Solr to make this process slower, possibly specifically for deletions? Andrew. On 27/01/2023 14:01, Blake Carver wrote: > I'm running default config values for the AS log levels so they are all set to 'debug'. I'm seeing "INFO" and not "DEBUG" there. > Deleted 186992 documents How much is in the deleted_records table? Try truncating that. ArchivesSpace is going and deleting anything in that table. > So I'm falling back to this just being super? slow for some reason. Could be some complex records, could be there's way too much in the deleted table. > but I'm not sure why the PUI indexer would become so much slower (21 hours) Yep, sounds about right. The PUI is slow. > We do have some collections that are quite large (10s of thousands of AOs), so maybe that's part of the issue. No doubt that's slowing it down too. ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Joshua D. Shaw Sent: Thursday, January 26, 2023 6:02 PM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] External Solr - Memory Allocation? Thanks, Blake! I'm running default config values for the AS log levels so they are all set to 'debug'. I took a closer look, and the timeout message happens exactly after the timeout amount I set (as you'd expect). Interestingly, Solr is in the middle of deleting documents when it goes silent I, [2023-01-26T09:18:40.357101 #78764] INFO -- : Thread-3384: Deleted 100 documents: # .... 40 minutes pass with all the other AS log chatter ... E, [2023-01-26T09:58:40.400971 #78764] ERROR -- : Thread-3384: SolrIndexerError when deleting records: Timeout error with POST {....} I, [2023-01-26T09:58:40.410522 #78764] INFO -- : Thread-3384: Deleted 100 documents: # This continuing delete phase goes on for a bit until it stops logging batch deletes. I, [2023-01-26T09:59:11.734200 #78764] INFO -- : Thread-3384: Deleted 9 documents: # .... 40 minutes pass with all the other AS log chatter ... And then the commit error pops up E, [2023-01-26T10:39:11.746166 #78764] ERROR -- : Thread-3384: SolrIndexerError when committing: Timeout error with POST {"commit":{"softCommit":false}}. Then after some more time I, [2023-01-26T11:06:35.678926 #78764] INFO -- : Thread-3384: Deleted 186992 documents: # .... This all seems to indicate to me that the commit phase is taking an inordinate amount of time (almost 2 hours - maybe that's what I need to set the timeout to?). After that, the indexer starts the 2nd repo I, [2023-01-26T11:06:35.765797 #78764] INFO -- : Thread-3384: PUI Indexer [2023-01-26 11:06:35 -0500] Indexed 2 additional PUI records in repository Sherman .... The indexer waits for a looong time with no timeout and no messaging - even though this is a tiny repo - and then starts the 3rd repo I, [2023-01-26T11:31:32.795602 #78764] INFO -- : Thread-3384: PUI Indexer [2023-01-26 11:31:32 -0500] Indexed 188 additional PUI records in repository Rauner-XO And then the indexer starts the 4th repo soon after and seems to go on to complete normally I, [2023-01-26T11:31:33.369369 #78764] INFO -- : Thread-3384: PUI Indexer [2023-01-26 11:31:33 -0500] ~~~ Indexed 25 of 74785 archival_object records in repository thedartmouth The Solr logs indicate that Solr is working this entire time doing adds and deletes. For example in one of the quiet phases: 2023-01-26 10:23:35.928 INFO (qtp2101153819-523) [ x:archivesspace] o.a.s.u.p.LogUpdateProcessorFactory [archivesspace] webapp=/solr path=/update params={}{add=... 2023-01-26 10:23:38.195 INFO (qtp2101153819-468) [ x:archivesspace] o.a.s.u.p.LogUpdateProcessorFactory [archivesspace] webapp=/solr path=/update params={}{deleteByQuery=... So I'm falling back to this just being super? slow for some reason. I do have some custom indexer addons, but I'm not sure why the PUI indexer would become so much slower (21 hours) when the Staff indexer completes in a normal amount of time (a little under 6 hours). For previous versions this hasn't been quite that different (6hrs vs about 13hrs). We do have some collections that are quite large (10s of thousands of AOs), so maybe that's part of the issue. I haven't checked to see if the PUI indexer is gathering that much more data (and traversing the tree more times - maybe?) than it was in 3.1.1, but that's on my 'to check' list. Joshua ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Blake Carver Sent: Thursday, January 26, 2023 4:12 PM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] External Solr - Memory Allocation? That's... interesting. That RAM allocation seems fine. That Solr timeout is way higher than I would think is needed. Maybe set the loglevel to debug and see if it spits out something more useful? Maybe you'll be able to see what it's up to during that looooong time. I like your theory on that. ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Joshua D. Shaw Sent: Thursday, January 26, 2023 3:38 PM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] External Solr - Memory Allocation? Following up on this. And looking for some advice! Even with the Solr timeout set to 40 minutes, I'm seeing some random Solr Timeout errors, though these do NOT cause the indexer to restart. In the latest test run I see one Solr Timeout for delete and one for commit - both following the PUI indexer run for AOs for the first and largest repo (~630k AOs). The indexer throws the delete timeout error, waits for a loooong time with seemingly no activity, throws the commit timeout error, waits again, and then picks back up as if nothing had gone wrong and continues with the initial index run. All of the index data looks correct (ie correct number of objects in both the staff and PUI). My theory is that the Solr update phase really is taking a super loooong time, but that the data has all been sent to Solr so the timeouts are really just ArchivesSpace waiting for Solr in between indexing one object type and the next and no index data is lost. There are no corresponding log entries in the Solr logs that I can find. I'm running solr 8.11.6 with 4GB and AS 3.3.1 with 4GB. Both bare metal on my laptop, so no container issues that might be at play. Solr memory use peaks at around 3.5GB. I've kept the stock thread and records per thread settings and just upped the timeout (to 2400). I guess the next step is to set the timeout even higher - maybe an hour (3600)? I don't see a reason to run a lower thread or record count, but can certainly try that as well, though I'm not looking forward to the time it will take (the current run takes 21 hours as it is - up from about 15 for 3.1.1) Any advice appreciated! Thanks! Joshua ________________________________ From: archivesspace_users_group-bounces at lyralists.lyrasis.org on behalf of Joshua D. Shaw Sent: Tuesday, January 24, 2023 6:56 AM To: Archivesspace Users Group Subject: [Archivesspace_Users_Group] External Solr - Memory Allocation? Hey all We're about to jump to v3.3.1 and I'm wondering if anyone has any suggestions for memory allocation for Solr? Currently we're running 6GB for the entire suite in v3.1.1 and are looking to keep the same overall memory footprint. Wondering if something like a 75/25 split (ie 4GB for AS and 2GB for Solr) would be a reasonable allocation? Or are people finding that Solr is more demanding? Thanks! Joshua ___________________ Joshua Shaw (he, him) Library Web & Application Developer Digital Library Technologies Group Dartmouth College 603.646.0405 _______________________________________________ Archivesspace_Users_Group mailing list Archivesspace_Users_Group at lyralists.lyrasis.org http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group -------------- next part -------------- An HTML attachment was scrubbed... URL: From Jessica.Crouch at lyrasis.org Tue Jan 31 14:40:53 2023 From: Jessica.Crouch at lyrasis.org (Jessica Crouch) Date: Tue, 31 Jan 2023 19:40:53 +0000 Subject: [Archivesspace_Users_Group] ArchivesSpace Update - January 2023 In-Reply-To: References: Message-ID: [cid:image001.jpg at 01D8A330.1E6801C0] ArchivesSpace Update ? January 2023 Upcoming Schedule at a Glance: February 21: ArchivesSpace Overview and Basic Record Creation Training March 29: Using Agents, Subjects and Classifications in ArchivesSpace Training April 4-5: Save the date for the ArchivesSpace Virtual Member Forum April 19: Using the ArchivesSpace Public User Interface Training Development We continue to gather up development and testing work for the next release, including work on some outstanding feature requests related to digital objects. More details will be available soon. New development is always up on our test server at https://test.archivesspace.org/staff/. Registration open for ArchivesSpace Overview and Basic Record Creation training (intended for audiences outside the contiguous U.S.) While our community-wide trainings provide virtual learning opportunities on a variety of topics, we recognize attending the live version of these trainings is difficult for users outside the time zones represented in North America. To better facilitate access to training for some other parts of our community, we will be offering a free training on February 21, 2023, from 1:00pm-4pm Australian Eastern Daylight Time (AEDT). This training will provide an overview of the ArchivesSpace application and basic record creation. During this training, trainers from the ArchivesSpace Trainers Corps will cover some of the most frequently used areas of the ArchivesSpace application and will provide adequate time for Q&A. A full agenda will be provided to attendees in advance of the training. We hope the feedback we receive from this training will inform future ArchivesSpace training opportunities in a variety of time zones. Date: February 21, 2023 Time: 1:00pm-4:00pm Australian Eastern Daylight Time (find your local time*) Trainers: Corey Schmidt (University of Georgia) and Jessica Crouch (ArchivesSpace) Registration: https://lyrasis.zoom.us/webinar/register/WN_uwnVJgYbRaaHbxiBWzVOSA *Please note this training occurs in the evening, Monday, February 20th for some time zones. Anyone using ArchivesSpace may register for this training. This training will be recorded. The recording will be made available via the ArchivesSpace website and YouTube channel. You do not need to register for the training to have access to the recordings. We understand this training is outside regular business hours for users in the United States. Registration is limited. We ask that users within the contiguous U.S. consider refraining from registering for this training in order to allow others the opportunity to attend. While this training does not assume any prior experience with ArchivesSpace, attendees are expected to have an understanding of archival workflows and terminology. Due to the large number of attendees, trainers will not be able to troubleshoot individual implementations of ArchivesSpace or provide additional archival instruction beyond the use of ArchivesSpace. For more information about ArchivesSpace, visit our Getting Started webpage or email ArchivesSpaceHome at lyrasis.org Save The Date: ArchivesSpace Virtual Member Forum Mark your calendars for this year's Virtual Member Forum! Taking place April 4-5, 2023, our Virtual Member Forum will be a two-day event spanning a variety of time zones and ArchivesSpace experience levels. A schedule and information about how to register for the event will be released closer to the forum. We are now accepting session proposals via our online form at https://forms.gle/jpSyx7VWZQ91HfTP8. We will be reviewing proposals on a rolling basis, so we encourage you to get your proposals in early. Submissions will be closed on March 24, 2023, with final notification by March 27, 2023. The ArchivesSpace program team is particularly interested in presentations or facilitated discussions related to the following topics: * Demonstrations of workflows using different modules or features of the application * Managing digital objects in ArchivesSpace * Developing local training and documentation * Workflows and tips for editing existing records in ArchivesSpace or other data cleanup projects * Examples of anti-racism, anti-colonialism and redescription work being executed in ArchivesSpace * Demonstrations of plugins or tools you?ve developed to make your work in ArchivesSpace easier The forum will include a mix of opportunities to share and learn from each other about many different aspects of ArchivesSpace and all presenter submissions are welcome. We anticipate recording many parts of the forum, but for it to be a success we will also need as many live participants as possible. We encourage you to dip in and out of the live program as much as you can. You will no doubt ?meet? a different set of colleagues each time. This year's Virtual Member Forum is the successor to the Online Forums we held in previous years. To better support our member community and recognize their direct contributions to the development and sustainability of the ArchivesSpace application, registration for this year's event is open to users from ArchivesSpace member organizations only. Going forward, we plan to make the Virtual Member Forum an online complement to our in-person Annual Member Forum, one of the many benefits of ArchivesSpace membership. This change will give even more members than before the opportunity to participate in a forum, no matter where they live or their capacity for attending events in person in a given year. If you are interested in learning more about ArchivesSpace membership for your organization, feel free to email us at ArchivesSpaceHome at lyrasis.org. Thank you for considering submitting a proposal. We?re looking forward to a great event, with your help! Local Contexts Initiative and Integration Plugin webinar recording available Thank you to everyone who joined ArchivesSpace community members and Local Contexts project members for a webinar on the Local Contexts initiative and the new ArchivesSpace plugin developed by ArchivesSpace community member Joshua Shaw that integrates the Traditional Knowledge and Biocultural Labels in ArchivesSpace. The recording for this webinar is now available at https://archivesspace.org/archives/7683. Registration now open for Spring 2023 ArchivesSpace community-wide trainings Registration is now open for our Spring community-wide trainings. To learn more about upcoming community-wide trainings or view information about past trainings, visit https://archivesspace.org/using-archivesspace/trainings. The first training offered this year will be on using Agents, Subjects and Classifications in ArchivesSpace. This training will be offered on March 29, 2023, from 1pm-4pm ET/10am-1pm PT. If you are interested in attending this training, you can register at https://www.lyrasis.org/ASpaceEventMarch. In this training, users will learn how to create, edit, and manage agent and subject records, link agent and subject records to material description records, and create and manage classifications in ArchivesSpace. The second training offered this year will be on using the ArchivesSpace Public User Interface (PUI). This training will be offered on April 19, 2023, from 1pm-4pm ET/10am-1pm PT. If you are interested in attending this training, you can register at https://www.lyrasis.org/ASpaceEventApril. In this training, users will learn how to navigate the ArchivesSpace Public User Interface, use subject and agent records to link related material, how to use classification to group related material and how to make customizations to the Public User Interface to improve the user experience for staff and researchers. Both trainings assume a basic understanding of archival workflows and metadata formats as well as experience navigating the ArchivesSpace staff user interface. If you plan to attend our community-wide trainings without the required background, we encourage you to review the recorded ArchivesSpace basics training available on our Getting Started webpage. These trainings are open to anyone using or interested in ArchivesSpace but ArchivesSpace members are eligible for a discount. If you qualify for the member discount, you will need to reach out to ArchivesSpaceHome at lyrasis.org to receive a member promo code before registering. Membership Update As of January 31, we have 468 General members, 15 Educational Program members, and 3 Registered Service Providers. Thank you to our current ArchivesSpace members for your contribution to the development and sustainability of the ArchivesSpace application. ArchivesSpace has a membership model to support the continued development of the application and user community and ArchivesSpace members are eligible for a variety of benefits. If you are interested in ArchivesSpace membership for your organization, email us at ArchivesSpaceHome at lyrasis.org. ________________________________ ArchivesSpace monthly updates provide news about ArchivesSpace community and program activities and are sent to our member listservs, the ArchivesSpace Google Group, and SAA?s Collection Management Section listserv, as well as being posted on the ArchivesSpace website. Please feel free to share this update with people you know who have an interest in ArchivesSpace but may not be on one of these lists. Jessica Dowd Crouch Community Engagement Coordinator for ArchivesSpace jessica.crouch at lyrasis.org [page1image482511520] -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 22525 bytes Desc: image001.jpg URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.jpg Type: image/jpeg Size: 29139 bytes Desc: image002.jpg URL: