[Archivesspace_Users_Group] Solr indexing performance

[Mark Cyzyk]

Dear ASpace User Group,

I've got ASpace 3.2.0 running against external Solr in our Development VM, but Solr is taking a loooong time building the initial index.

Like, it's been running now for a full week!

I have tweaked the config.rb settings and restarted the ASpace service, but nothing seems to speed it up.

VM:
2 cpus
8 GB RAM
looking at used resources, seems like there is a lot still FREE

My settings in config.rb:

## By setting the next two options, you can control how many CPU cores are used,
## and the amount of memory that will be consumed by the indexing process (more
## cores and/or more records per thread means more memory used).
AppConfig[:indexer_records_per_thread] = 250              <-- I bumped this up from 25
AppConfig[:indexer_thread_count] = 2
AppConfig[:indexer_solr_timeout_seconds] = 999999
#
## PUI Indexer Settings
AppConfig[:pui_indexer_enabled] = true
AppConfig[:pui_indexing_frequency_seconds] = 15         <-- I decreased to this, down from 30
AppConfig[:pui_indexer_records_per_thread] = 250        <-- I bumped this up from 25
AppConfig[:pui_indexer_thread_count] = 2

Does anyone know how to speed up Solr indexing?  I can't seem to find the bottleneck here.

Advice appreciated,

Mark

--

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
Mark Cyzyk, M.A., M.L.S.
Library Applications Group
The Sheridan Libraries
The Johns Hopkins University
mcyzyk at jhu.edu

Verba volant, scripta manent.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20220704/02bc3124/attachment.html>