[Archivesspace_Users_Group] Staging upgrade using Vagrant/Virtualbox, data in MySQL tables, Solr core present, reindexed, yet no data displayed in ASpace application
Mark Cyzyk
mcyzyk at gmail.com
Thu Jul 13 10:49:27 EDT 2023
In the Solr Web Gui --> Archivesspace Core --> Dataimport, there is this
error message:
The solrconfig.xml file for this index does not have an operational
DataImportHandler defined!
My solrconfig.xml, straight from 3.2.0 codebase:
|<?xml version="1.0" encoding="utf-8"?> <config> <lib
dir="${solr.install.dir}/dist/" regex="solr-analysis-extras.*\.jar" />
<lib dir="${solr.install.dir}/contrib/analysis-extras/lucene-libs"
regex="lucene-analyzers-icu-.*\.jar" /> <lib
dir="${solr.install.dir}/contrib/analysis-extras/lib/"
regex="icu4j-*.*\.jar" /> <schemaFactory
class="ClassicIndexSchemaFactory"/>
<luceneMatchVersion>8.10</luceneMatchVersion>
<dataDir>${solr.data.dir:}</dataDir> <directoryFactory
name="DirectoryFactory"
class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}" />
<updateHandler class="solr.DirectUpdateHandler2">
<maxPendingDeletes>1000</maxPendingDeletes> <autoCommit>
<maxDocs>1000</maxDocs> <maxTime>60000</maxTime>
<openSearcher>false</openSearcher> </autoCommit> </updateHandler>
<query> <maxBooleanClauses>1024</maxBooleanClauses> <filterCache
class="solr.FastLRUCache" size="512" initialSize="512" autowarmCount="0"
/> <queryResultCache class="solr.LRUCache" size="512" initialSize="512"
autowarmCount="0" /> <documentCache class="solr.LRUCache" size="512"
initialSize="512" autowarmCount="0" />
<enableLazyFieldLoading>true</enableLazyFieldLoading>
<queryResultWindowSize>20</queryResultWindowSize>
<queryResultMaxDocsCached>200</queryResultMaxDocsCached>
<useColdSearcher>false</useColdSearcher>
<maxWarmingSearchers>2</maxWarmingSearchers> </query> <requestDispatcher
handleSelect="false"> <requestParsers enableRemoteStreaming="true"
multipartUploadLimitInKB="2048000" /> <httpCaching never304="true" />
</requestDispatcher> <requestHandler name="/select"
class="solr.SearchHandler"> <lst name="defaults"> <str
name="defType">edismax</str> <str name="echoParams">explicit</str> <int
name="rows">10</int> <str name="df">fullrecord</str> <str
name="pf">four_part_id^50</str> <str name="qf">title^25 four_part_id^50
fullrecord</str> <str name="bq">primary_type:resource^100</str> <str
name="bq">primary_type:accession^100</str> <str
name="bq">primary_type:subject^50</str> <str
name="bq">primary_type:agent_person^50</str> <str
name="bq">primary_type:agent_corporate_entity^30</str> <str
name="bq">primary_type:agent_family^30</str> </lst> </requestHandler>
<requestHandler name="/update"
class="solr.UpdateRequestHandler"></requestHandler> <requestHandler
name="/analysis/document" class="solr.DocumentAnalysisRequestHandler"
startup="lazy" /> <requestHandler name="/analysis/field" startup="lazy"
class="solr.FieldAnalysisRequestHandler" /> <requestHandler
name="/replication" class="solr.ReplicationHandler" startup="lazy" />
<queryResponseWriter name="json" class="solr.JSONResponseWriter"> <str
name="content-type">text/plain; charset=UTF-8</str>
</queryResponseWriter> <admin> <defaultQuery>*:*</defaultQuery> </admin>
<requestHandler name="/admin/luke"
class="org.apache.solr.handler.admin.LukeRequestHandler" /> </config> |
<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
Mark Cyzyk, M.A., M.L.S.
Library Applications Group
The Sheridan Libraries
The Johns Hopkins University
mcyzyk at jhu.edu
Verba volant, scripta manent.
On 7/13/23 10:40, Mark Cyzyk wrote:
>> Hi Mark,
>>
>> If you go to this URL
>>
>> http://localhost:8983/solr/#/archivesspace/files?file=schema.xml
>>
>> does the schema file shown there match the same one you copied over?
>>
>> https://github.com/archivesspace/archivesspace/blob/v3.2.0/solr/schema.xml
>>
>> Jesse
>
>
> Yes, they match:
>
> <schema name="archivesspace" version="1.5">
>
>
> Thinking...
>
> Mark
>
> ||
>
> <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
> Mark Cyzyk, M.A., M.L.S.
> Library Applications Group
> The Sheridan Libraries
> The Johns Hopkins University
> mcyzyk at jhu.edu
>
> Verba volant, scripta manent.
> On 7/12/23 16:20, Jesse Martinez wrote:
>> Hi Mark,
>>
>> If you go to this URL
>>
>> http://localhost:8983/solr/#/archivesspace/files?file=schema.xml
>>
>> does the schema file shown there match the same one you copied over?
>>
>> https://github.com/archivesspace/archivesspace/blob/v3.2.0/solr/schema.xml
>>
>> Jesse
>>
>>
>> On Wed, Jul 12, 2023 at 3:44 PM Mark Cyzyk <mcyzyk at gmail.com> wrote:
>>
>> Another clue, from the Log in the Solr Web GUI:
>>
>>> 7/12/2023, 3:19:27 PM WARN false
>>> CoreContainer Not all security plugins configured!
>>> authentication=disabled authorization=disabled. Solr is only as
>>> secure as you make it. Consider configuring
>>> authentication/authorization before exposing Solr to users
>>> internal or external. See https://s.apache.org/solrsecurity for
>>> more info
>>>
>>
>> Maybe ASpace simply does not have permission to add to Solr?
>> (And yet, this is just a warning. And yet, for the sake of
>> replicability, this very same Solr setup worked in Vagrant during
>> our last ASpace upgrade.)
>>
>> Here, setting up Solr in my Vagrant file (which I recently ran
>> line-by-line, manually, without error):
>>
>> solr_provision = <<SCRIPT
>> cd /opt
>> sudo wget
>> https://archive.apache.org/dist/lucene/solr/8.10.0/solr-8.10.0.tgz
>> sudo tar zxf solr-8.10.0.tgz
>> sudo bash /opt/solr-8.10.0/bin/install_solr_service.sh
>> /opt/solr-8.10.0.tgz
>> sudo mkdir -p /opt/solr/server/solr/configsets/archivesspace/conf
>> sudo cp -r /vagrant/archivesspace_CURRENT/solr/*
>> /opt/solr/server/solr/configsets/archivesspace/conf/
>> sudo chown -R solr:solr /opt/solr
>> sudo su - solr -c "/opt/solr/bin/solr create -c archivesspace
>> -p 8983 -d archivesspace"
>> sudo systemctl restart solr
>> SCRIPT
>>
>>
>> Reading about Solr and Permissions....
>>
>> And sorry for all the emails in this thread. And yet, once this
>> is resolved it may have value for others in future?
>>
>> Mark
>>
>> <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
>> Mark Cyzyk, M.A., M.L.S.
>> Library Applications Group
>> The Sheridan Libraries
>> The Johns Hopkins University
>> mcyzyk at jhu.edu
>>
>>
>> On 7/12/23 11:46 AM, Joshua D. Shaw wrote:
>>> Hey Mark
>>>
>>> After the updates you've made to the config, try shutting down
>>> AS and clearing the indexer state directories and restarting.
>>> That should tell AS to try a new index.
>>>
>>> Also, I think that +0000 is just a timestamp offset. Its not
>>> counting the objects indexed. That log line looks something
>>> like: PUI Indexer ... indexed x of y resources
>>>
>>> Joshua
>>>
>>> ------------------------------------------------------------------------
>>> *From:* archivesspace_users_group-bounces at lyralists.lyrasis.org
>>> <archivesspace_users_group-bounces at lyralists.lyrasis.org>
>>> <mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>
>>> on behalf of Mark Cyzyk <mcyzyk at gmail.com> <mailto:mcyzyk at gmail.com>
>>> *Sent:* Wednesday, July 12, 2023 11:32 AM
>>> *To:* Brian Hoffman <brian.hoffman at lyrasis.org>
>>> <mailto:brian.hoffman at lyrasis.org>; mcyzyk at jhu.edu
>>> <mcyzyk at jhu.edu> <mailto:mcyzyk at jhu.edu>; Archivesspace Users
>>> Group <archivesspace_users_group at lyralists.lyrasis.org>
>>> <mailto:archivesspace_users_group at lyralists.lyrasis.org>;
>>> jesse.martinez at bc.edu <jesse.martinez at bc.edu>
>>> <mailto:jesse.martinez at bc.edu>
>>> *Subject:* Re: [Archivesspace_Users_Group] Staging upgrade using
>>> Vagrant/Virtualbox, data in MySQL tables, Solr core present,
>>> reindexed, yet no data displayed in ASpace application
>>>
>>> Brian,
>>>
>>> I am trying to duplicate our Production ASpace 3.2.0 in a local
>>> Vagrant box so I can stage an upgrade to the latest codebase.
>>>
>>>> Try uncommenting that line and changing it to:
>>>>
>>>> AppConfig[:solr_url] =
>>>> "http://localhost:8983/solr/archivesspace"
>>>> <http://localhost:8983/solr/archivesspace>
>>>
>>> Done. And yet:
>>>
>>> ************************************************************
>>> Welcome to ArchivesSpace!
>>> You can now point your browser to http://localhost:8080
>>> ************************************************************
>>> I, [2023-07-12T15:28:45.335247 #51936] INFO -- :
>>> Thread-3306: PUI Indexer [2023-07-12 15:28:45 +0000] Running
>>> index round
>>> I, [2023-07-12T15:28:47.533454 #51936] INFO -- :
>>> Thread-3306: PUI Indexer [2023-07-12 15:28:47 +0000] Index
>>> round complete
>>> I, [2023-07-12T15:28:50.634218 #51936] INFO -- :
>>> Thread-2956: Staff Indexer [2023-07-12 15:28:50 +0000]
>>> Running index round
>>> I, [2023-07-12T15:28:52.715507 #51936] INFO -- :
>>> Thread-2956: Staff Indexer [2023-07-12 15:28:52 +0000] Index
>>> round complete
>>>
>>>
>>> Still +0000!
>>>
>>> Mark
>>>
>>> <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
>>> Mark Cyzyk, M.A., M.L.S.
>>> Library Applications Group
>>> The Sheridan Libraries
>>> The Johns Hopkins University
>>> mcyzyk at jhu.edu
>>>
>>> Verba volant, scripta manent.
>>> On 7/12/23 11:10, Brian Hoffman wrote:
>>>>
>>>> Hi Mark,
>>>>
>>>> What version of ASpace are you using?
>>>>
>>>> Try uncommenting that line and changing it to:
>>>>
>>>> AppConfig[:solr_url] =
>>>> "http://localhost:8983/solr/archivesspace"
>>>> <http://localhost:8983/solr/archivesspace>
>>>>
>>>> *From: *Mark Cyzyk <mcyzyk at gmail.com> <mailto:mcyzyk at gmail.com>
>>>> *Date: *Wednesday, July 12, 2023 at 10:59 AM
>>>> *To: *Brian Hoffman <brian.hoffman at lyrasis.org>
>>>> <mailto:brian.hoffman at lyrasis.org>, mcyzyk at jhu.edu
>>>> <mcyzyk at jhu.edu> <mailto:mcyzyk at jhu.edu>, Archivesspace Users
>>>> Group <archivesspace_users_group at lyralists.lyrasis.org>
>>>> <mailto:archivesspace_users_group at lyralists.lyrasis.org>,
>>>> jesse.martinez at bc.edu <jesse.martinez at bc.edu>
>>>> <mailto:jesse.martinez at bc.edu>
>>>> *Subject: *Re: [Archivesspace_Users_Group] Staging upgrade
>>>> using Vagrant/Virtualbox, data in MySQL tables, Solr core
>>>> present, reindexed, yet no data displayed in ASpace application
>>>>
>>>>
>>>> Brian,
>>>>
>>>> That value, commented out because it's the default, in my
>>>> config is:
>>>>
>>>> ## The ArchivesSpace Solr index listens on port 8090 by
>>>> default. You can
>>>> ## set it to something else below.
>>>> #AppConfig[:solr_url] = "http://localhost:8090"
>>>> <http://localhost:8090>
>>>>
>>>>
>>>> Mark
>>>>
>>>>
>>>> <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
>>>> Mark Cyzyk, M.A., M.L.S.
>>>> Library Applications Group
>>>> The Sheridan Libraries
>>>> The Johns Hopkins University
>>>> mcyzyk at jhu.edu
>>>>
>>>> Verba volant, scripta manent.
>>>>
>>>> On 7/12/23 10:34, Brian Hoffman wrote:
>>>>
>>>> Hi Mark,
>>>>
>>>> I am referring to the value for :solr_url in the file
>>>> ‘config/config.rb’
>>>>
>>>> Brian
>>>>
>>>> *From: *Mark Cyzyk <mcyzyk at gmail.com> <mailto:mcyzyk at gmail.com>
>>>> *Date: *Wednesday, July 12, 2023 at 10:32 AM
>>>> *To: *Brian Hoffman <brian.hoffman at lyrasis.org>
>>>> <mailto:brian.hoffman at lyrasis.org>, mcyzyk at jhu.edu
>>>> <mcyzyk at jhu.edu> <mailto:mcyzyk at jhu.edu>, Archivesspace
>>>> Users Group
>>>> <archivesspace_users_group at lyralists.lyrasis.org>
>>>> <mailto:archivesspace_users_group at lyralists.lyrasis.org>,
>>>> jesse.martinez at bc.edu <jesse.martinez at bc.edu>
>>>> <mailto:jesse.martinez at bc.edu>
>>>> *Subject: *Re: [Archivesspace_Users_Group] Staging upgrade
>>>> using Vagrant/Virtualbox, data in MySQL tables, Solr core
>>>> present, reindexed, yet no data displayed in ASpace application
>>>>
>>>> Thanks, Brian,
>>>>
>>>> Here is what I see on screen:
>>>>
>>>>
>>>>
>>>>
>>>> <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
>>>>
>>>> Mark Cyzyk, M.A., M.L.S.
>>>>
>>>> Library Applications Group
>>>>
>>>> The Sheridan Libraries
>>>>
>>>> The Johns Hopkins University
>>>>
>>>> mcyzyk at jhu.edu
>>>>
>>>>
>>>>
>>>> Verba volant, scripta manent.
>>>>
>>>> On 7/12/23 10:24, Brian Hoffman wrote:
>>>>
>>>> Hi Mark,
>>>>
>>>> I believe you have incorrectly set the solr url to a
>>>> screen on the control panel.
>>>>
>>>> Try replacing:
>>>>
>>>> http://localhost:8983/solr/#/~cores/archivesspace
>>>> <http://localhost:8983/solr/#/~cores/archivesspace>
>>>>
>>>> with
>>>>
>>>> http://localhost:8983/solr/archivesspace
>>>> <http://localhost:8983/solr/#/~cores/archivesspace>
>>>>
>>>> *From:
>>>> *archivesspace_users_group-bounces at lyralists.lyrasis.org
>>>> <archivesspace_users_group-bounces at lyralists.lyrasis.org>
>>>> <mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>
>>>> on behalf of Mark Cyzyk <mcyzyk at gmail.com>
>>>> <mailto:mcyzyk at gmail.com>
>>>> *Date: *Wednesday, July 12, 2023 at 10:19 AM
>>>> *To: *jesse.martinez at bc.edu <jesse.martinez at bc.edu>
>>>> <mailto:jesse.martinez at bc.edu>, mcyzyk at jhu.edu
>>>> <mcyzyk at jhu.edu> <mailto:mcyzyk at jhu.edu>
>>>> *Cc: *Archivesspace Users Group
>>>> <archivesspace_users_group at lyralists.lyrasis.org>
>>>> <mailto:archivesspace_users_group at lyralists.lyrasis.org>
>>>> *Subject: *Re: [Archivesspace_Users_Group] Staging
>>>> upgrade using Vagrant/Virtualbox, data in MySQL tables,
>>>> Solr core present, reindexed, yet no data displayed in
>>>> ASpace application
>>>>
>>>> Darn:
>>>>
>>>>
>>>>
>>>>
>>>> |{|
>>>>
>>>> | "|responseHeader|":|{
>>>>
>>>> "status":0,
>>>>
>>>> "QTime":187,
>>>>
>>>> "params":{
>>>>
>>>> "q":"*:*",
>>>>
>>>> "indent":"true",
>>>>
>>>> "q.op":"OR",
>>>>
>>>> "_":"1689171084647"}}|,|
>>>>
>>>> | "|response|":|{"numFound":0,"start":0,"numFoundExact":true,"docs":[]
>>>>
>>>> }}
>>>>
>>>>
>>>> I will look more carefully at my Solr Aspace config
>>>> (and yet the steps in my Vagrant file for setting this
>>>> up are the same as last summer when I did this!).
>>>>
>>>> Mark
>>>>
>>>>
>>>>
>>>> <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
>>>>
>>>> Mark Cyzyk, M.A., M.L.S.
>>>>
>>>> Library Applications Group
>>>>
>>>> The Sheridan Libraries
>>>>
>>>> The Johns Hopkins University
>>>>
>>>> mcyzyk at jhu.edu
>>>>
>>>>
>>>>
>>>> Verba volant, scripta manent.
>>>>
>>>> On 7/12/23 10:09, Jesse Martinez wrote:
>>>>
>>>> Thanks for sharing that, Mark!
>>>>
>>>> I have another quick follow up question.
>>>>
>>>> What do you see when you run a solr search like
>>>> this one:
>>>>
>>>> http://localhost:8983/solr/#/archivesspace/query?q=*:*&q.op=OR&indent=true
>>>> <http://localhost:8983/solr/#/archivesspace/query?q=*:*&q.op=OR&indent=true>
>>>>
>>>> You should see all your indexed records. But if you
>>>> don't see any results then it may indicate that
>>>> your archivesspace configuration directory is not
>>>> correct.
>>>>
>>>> Jesse
>>>>
>>>> On Wed, Jul 12, 2023 at 10:05 AM Mark Cyzyk
>>>> <mcyzyk at gmail.com> wrote:
>>>>
>>>>
>>>> Thanks, Jesse,
>>>>
>>>> Here is the output of those commands:
>>>>
>>>> ps aux | grep solr | grep -v grep
>>>> solr 898 0.0 0.1 19096 9812
>>>> ? Ss Jul11 0:00
>>>> /lib/systemd/systemd --user
>>>> solr 910 0.0 0.0 169492 3368
>>>> ? S Jul11 0:00 (sd-pam)
>>>> solr 1223 0.5 12.5 4018692 742508
>>>> ? Sl Jul11 7:41 java -server
>>>> -Xms512m -Xmx512m -XX:+UseG1GC
>>>> -XX:+PerfDisableSharedMem
>>>> -XX:+ParallelRefProcEnabled
>>>> -XX:MaxGCPauseMillis=250 -XX:+UseLargePages
>>>> -XX:+AlwaysPreTouch
>>>> -XX:+ExplicitGCInvokesConcurrent
>>>> -verbose:gc -XX:+PrintHeapAtGC
>>>> -XX:+PrintGCDetails -XX:+PrintGCDateStamps
>>>> -XX:+PrintGCTimeStamps
>>>> -XX:+PrintTenuringDistribution
>>>> -XX:+PrintGCApplicationStoppedTime
>>>> -Xloggc:/var/solr/logs/solr_gc.log
>>>> -XX:+UseGCLogFileRotation
>>>> -XX:NumberOfGCLogFiles=9
>>>> -XX:GCLogFileSize=20M
>>>> -Dsolr.jetty.inetaccess.includes=
>>>> -Dsolr.jetty.inetaccess.excludes=
>>>> -Dsolr.log.dir=/var/solr/logs
>>>> -Djetty.port=8983 -DSTOP.PORT=7983
>>>> -DSTOP.KEY=solrrocks -Duser.timezone=UTC
>>>> -XX:-OmitStackTraceInFastThrow
>>>> -XX:OnOutOfMemoryError=/opt/solr/bin/oom_solr.sh
>>>> 8983 /var/solr/logs
>>>> -Djetty.home=/opt/solr/server
>>>> -Dsolr.solr.home=/var/solr/data
>>>> -Dsolr.data.home=
>>>> -Dsolr.install.dir=/opt/solr
>>>> -Dsolr.default.confdir=/opt/solr/server/solr/configsets/_default/conf
>>>> -Dlog4j.configurationFile=/var/solr/log4j2.xml
>>>> -Xss256k -Dsolr.log.muteconsole -jar
>>>> start.jar --module=http --module=gzip
>>>>
>>>> Puzzling!
>>>>
>>>> Mark
>>>>
>>>>
>>>>
>>>> <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
>>>>
>>>> Mark Cyzyk, M.A., M.L.S.
>>>>
>>>> Library Applications Group
>>>>
>>>> The Sheridan Libraries
>>>>
>>>> The Johns Hopkins University
>>>>
>>>> mcyzyk at jhu.edu
>>>>
>>>>
>>>>
>>>> Verba volant, scripta manent.
>>>>
>>>> On 7/12/23 09:52, Jesse Martinez wrote:
>>>>
>>>> ps aux | grep solr | grep -v grep
>>>>
>>>>
>>>> --
>>>>
>>>> Jesse Martinez
>>>>
>>>> Senior Library Applications Developer
>>>>
>>>> O'Neill Library, Boston College
>>>>
>>>> jesse.martinez at bc.edu
>>>>
>>>> 617-552-2509
>>>>
>>>> he/him/his
>>>>
>>>
>>
>> _______________________________________________
>> Archivesspace_Users_Group mailing list
>> Archivesspace_Users_Group at lyralists.lyrasis.org
>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>>
>>
>>
>> --
>> Jesse Martinez
>> Senior Library Applications Developer
>> O'Neill Library, Boston College
>> jesse.martinez at bc.edu
>> 617-552-2509
>> he/him/his
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20230713/b455bb9a/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: snapshot1.png
Type: image/png
Size: 98031 bytes
Desc: not available
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20230713/b455bb9a/attachment.png>
More information about the Archivesspace_Users_Group
mailing list