[Archivesspace_Users_Group] Phantom records post-migration to version 3.2.0

Andrew Morrison andrew.morrison at bodleian.ox.ac.uk
Fri Aug 12 03:39:04 EDT 2022 

That plug-in does the equivalent of the "Full reindex" process described 
here:

https://archivesspace.github.io/tech-docs/administration/indexes.html

So, it is not something you want running upon startup of a production 
system, as records will be unavailable until they've been re-indexed, 
which might take a long time, depending on the size of your collections. 
It was probably written to help automate the management of Lyrasis's own 
sandbox.archivesspace.org and test.archivesspace.org servers, which only 
contain a few test records.

A one-off "full reindex" is probably required to fix your phantom 
records. But you need to watch the application log to monitor that the 
indexing process starts, and runs all the way to the end, without 
errors. Personally, I'd trigger it manually, but if your prefer to use 
the plug-in, then you need to change 4567 in the curl command to the 
port on which your backend is listening. Usually that is 8089, but 
double-check the AppConfig[:backend_url] setting in config.rb. If you 
don't know how to obtain the session token in $SESSION, see the 
instructions here:

https://archivesspace.github.io/tech-docs/api/

If you want to figure out the cause of the phantom records, you should 
identify some by comparing what's in Solr against what's in MySQL, 
before running the re-index. Then you could check when they were deleted 
in the "deleted_records" MySQL table.

Andrew.



On 11/08/2022 15:45, Jerry Boggio wrote:
>
> Hi Don and other ArchivesSpace Users;
>
> Referring to the page Don suggested:
>
> *ArchivesSpace Reindexer plugin*
>
> This plugin can be used in two ways:
>
> 1.On system startup to initiate a reindex
>
> 2.Via the api to trigger a reindex
>
> TODO: consider running as a job and making it available that way too.
>
> *On startup*
>
> ·Set AppConfig[:reindex_on_startup] = true in config.rb
>
> ·Restart ArchivesSpace
>
> *Via the api*
>
> curl -H "X-ArchivesSpace-Session: $SESSION" -X POST 
> http://localhost:4567/plugins/reindex
>
> We have set “AppConfig[:reindex_on_startup] = true”inconfig.rb before 
> the last restart and still have “phantom” records.
>
> Looking at the API call, which port should be used here? The port for 
> the External Solr Index, the one specified, or something else?
>
> We would also like to know the cause of the “phantom” records. Does it 
> have something to do with the setup-database.sh script?
>
> By the way, when running the check_index.sh script in Linux it 
> generates an error. Should there be a new script for External Solr?
>
> /apps/archivesspace/scripts>ll
>
> total 52
>
> -rw-r--r-- 1 archspc users 317 Apr 16  2021 backup.bat
>
> -rwxr-xr-x 1 archspc users 365 Apr 16  2021 backup.sh
>
> -rwxr-xr-x 1 archspc users 271 Apr 16  2021 checkindex.bat
>
> *-rwxr-xr-x 1 archspc users 360 Apr 16  2021 checkindex.sh***
>
> -rw-r--r-- 1 archspc users 290 Apr 16  2021 ead_export.bat
>
> -rwxr-xr-x 1 archspc users 350 Apr 16  2021 ead_export.sh
>
> -rwxr-xr-x 1 archspc users 217 Apr 16  2021 find-base.sh
>
> -rw-r--r-- 1 archspc users 496 Apr 16  2021 initialize-plugin.bat
>
> -rwxr-xr-x 1 archspc users 804 Dec 22  2021 initialize-plugin.sh
>
> -rw-r--r-- 1 archspc users 295 Apr 16  2021 password-reset.bat
>
> -rwxr-xr-x 1 archspc users 353 Apr 16  2021 password-reset.sh
>
> drwxr-xr-x 2 archspc users  46 Feb  4  2022 rb
>
> -rwxr-xr-x 1 archspc users 304 Apr 16  2021 setup-database.bat
>
> -rwxr-xr-x 1 archspc users 322 Apr 16  2021 setup-database.sh
>
> /apps/archivesspace/scripts>./checkindex.sh
>
> *RuntimeError: Solr war file not found***
>
> find_solr_war at ../scripts/rb/checkindex.rb:29
>
> check at ../scripts/rb/checkindex.rb:10
>
> <main> at ../scripts/rb/checkindex.rb:86
>
> /apps/archivesspace/scripts>
>
> I know there are a lot of questions here and we would appreciate your 
> help in getting answers to all of them.
>
> Thank you again!
>
> *Gerard (Jerry) Boggio* | *MITRE Corporation* | R124 - Collaboration & 
> Info Management| 781-271-2719
>
> Greg,
> One easy way to rebuild the index is to install the Lyrasis reindexer
> plugin: https://github.com/lyrasis/aspace-reindexer. Once installed, you
> can either have it rebuild the index via the startup config or by
> the endpoint that the plugin adds to the API. We recently migrated from
> 3.0.2 to 3.2.0 and everything went smoothly for us. Once the new 
> version of
> Aspace was running we hit the endpoint and the database rebuilt.
> Don
> Donald R. Mennerich, digital archivist
> New York University Libraries
> don.mennerich at nyu.edu 
> <http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group> 
> (212) 992-6264
>
> *From:* Jerry Boggio <gboggio at mitre.org>
> *Sent:* Tuesday, August 9, 2022 2:22 PM
> *To:* archivesspace_users_group at lyralists.lyrasis.org
> *Cc:* Erin Faulder <EFAULDER at mitre.org>
> *Subject:* Phantom records post-migration to version 3.2.0
>
> Hello ASpace Users;
>
> We just upgraded from ArchivesSpace version 2.7.1 to version 3.2.0 and 
> now have “phantom” records in Resources and Subjects. By phantom I 
> mean records show up in ASpace, but are not in the database and when 
> trying to view the record returns:
>
> As part of this upgrade on our Test machine  we:
>
>   * Started using External Solr as opposed to Internal
>   * Purged the contents of the following directories, but did not
>     delete the directories:
>       o /apps/archivesspace/data/indexer_pui_state
>       o /apps/archivesspace/data/indexer_state
>       o /apps/archivesspace/data/tmp
>   * Kept, but moved /apps/archivesspace/data/solr_backups  to
>      /apps/archivesspace/data/old_solr_backups; and created a new
>     /apps/archivesspace/data/solr_backups  directory
>   * Refreshed our Test MySQL database from our v2.7.1 Prod version and
>     ran the setup-database.sh script to convert to v3.2.0. We are
>     running MySQL version 8.
>
> This left the following directory structure under the data directory:
>
> /apps/archivesspace/data>ll
>
> total 36
>
> drwxr-xr-x  5 archspc users  147 Oct 16  2019 archivesspace_demo_db
>
> drwxr-xr-x  9 archspc users 4096 Oct 18  2019 demo_db_backups
>
> -rw-r--r--  1 archspc users   32 Oct  8  2019 
> frontend_cookie_secret_cookie_secret.dat
>
> -rw-r--r--  1 archspc users   32 May 14  2020 frontend_cookie_secret.dat
>
> drwxr-xr-x  2 archspc users 4096 Aug  9 13:59 indexer_pui_state
>
> drwxr-xr-x  2 archspc users 4096 Aug  9 13:59 indexer_state
>
> drwxr-xr-x 13 archspc users 4096 Aug  9 00:00 old_solr_backups
>
> -rw-r--r--  1 archspc users   32 Oct  8  2019 
> public_cookie_secret_cookie_secret.dat
>
> -rw-r--r--  1 archspc users   32 May 14  2020 public_cookie_secret.dat
>
> drwxr-xr-x  3 archspc users   22 Oct 28  2019 shared
>
> drwxr-xr-x  2 archspc users    6 Aug  9 10:33 solr_backups
>
> drwxr-xr-x 13 archspc users 4096 Aug  9 13:59 tmp
>
> /apps/archivesspace/data>
>
> External Solr was installed on the same Linux server as ArchivesSpace. 
> We installed solr-8.10.0, but see that a newer version solr-8.11.2 is 
> available. Should we be using the newer version? How can we clear the 
> External Solr index in order to rebuild it?
>
> What needs to be done to eliminate these phantom records? We are 
> assuming it is something left over from the prior version.
>
> Please advise or let us know if you need more information.
>
> Thank you.
>
> *Gerard (Jerry) Boggio* | *MITRE Corporation* | R124 - Collaboration & 
> Info Management| 781-271-2719
>
>
> _______________________________________________
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group at lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20220812/2b1357ac/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 39627 bytes
Desc: not available
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20220812/2b1357ac/attachment.png>

More information about the Archivesspace_Users_Group mailing list