[Archivesspace_Users_Group] indexing problem?

Mon Aug 10 09:00:09 EDT 2015

Thank you Chris, we'll check this and confirm the other questions.

Melissa

From: archivesspace_users_group-bounces at lyralists.lyrasis.org [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] On Behalf Of Chris Fitzpatrick
Sent: Monday, August 10, 2015 8:58 AM
To: archivesspace_users_group at lyralists.lyrasis.org
Subject: Re: [Archivesspace_Users_Group] indexing problem?

Hi Melissa,

It might be a little tricky for you all, since your Aspace has a lot of customizations and plugins installed that modify a lot of the core ( like top_containers and things having to do with barcodes).

But, to answer the question of configuration, yes the defaults are enabled by default. Your config.rb file can be empty, and the values found in config-defaults.r<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_archivesspace_archivesspace_blob_master_common_config_config-2Ddefaults.rb&d=AwMF-g&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=hZ2NgbpBpYrVGDxkdkF51g3GQgQXAv3xnKDZbIHkbFs&m=TrFnsrdmFufBFCzQolHKFtUM9Dcgf9FDiY5KwXE4-n4&s=7ILoNJDQhtEZNtV2f-TAFi1EYe5f4-3QxidaL125OTs&e=>b would be used. ASpace ships with a config.rb file that has the config-defaults.rb files commented out just to show what those defaults are.

I've seen the EOF error happen on some random cases where the indexer has nothing to index. What's happening is it's reading a indexer state file that is empty, which it is not expecting. I've found this to be usually be benign. But you can remove this pretty easily...

To check this, see if there are any empty files in the indexer_state directory:

$ find data/indexer_state/ -type f -empty

You can delete those:

$ find data/indexer_state/ -type f -empty -exec rm {} \;

When the indexer runs next, it will recreate that any files it needs to track the indexer state. If this keeps recurring, then we'll have to figure out why.

Also, EOF can in some situations be sent for a HTTP request.

 First question is if you have the backend on a SSL connection, possibly with a self-signed cert?

Also, do you have a proxy in front of the backend?

You can try and install this to you plugins/local/indexer directory, restart aspace, and see where the EOF error is happening.

b,chris

Chris Fitzpatrick | Developer, ArchivesSpace
Skype: chrisfitzpat  | Phone: 918.236.6048
http://archivesspace.org/

________________________________
From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> <archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>> on behalf of Wisner, Melissa <melissa.wisner at yale.edu<mailto:melissa.wisner at yale.edu>>
Sent: Friday, August 7, 2015 4:48 PM
To: archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>
Subject: [Archivesspace_Users_Group] indexing problem?

Hello,

Our AS users have reported they were unable to search/retrieve data they just added to ArchivesSpace. We assume this is a problem with the index, if after even several hours a record cannot be retrieved by a barcode.

Scenario 1: over 2 -3 days a newly added record could not be retrieved by barcode. When staff would try and create a new record with the same barcode, AS would tell them barcode already exists.  After the 3rd day it could be retrieved through a barcode search.

Scenario 2: same report came in Thursday morning. Newly added record could not be retrieved by barcode. Today it still can't. But I can retrieve it by the top container number highlighted below in yellow.

This behavior is very odd. The reported issue from Thursday morning is shown in this snippet from the log files. There appears to be an error message at the end of this regarding End of File/EOF. I do not know if that is the problem, or what that error means.

Additionally, the config file for our production AS does not include certain entries that are shown here as defaults: https://github.com/archivesspace/archivesspace/blob/master/common/config/config-defaults.rb<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_archivesspace_archivesspace_blob_master_common_config_config-2Ddefaults.rb&d=AwMF-g&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=hZ2NgbpBpYrVGDxkdkF51g3GQgQXAv3xnKDZbIHkbFs&m=TrFnsrdmFufBFCzQolHKFtUM9Dcgf9FDiY5KwXE4-n4&s=7ILoNJDQhtEZNtV2f-TAFi1EYe5f4-3QxidaL125OTs&e=>

Specifically- AppConfig[:solr_indexing_frequency_seconds] = 30 and  AppConfig[:indexer_solr_timeout_seconds] = 300

If these are not defined, does the default still apply? A default of 30 seconds seems reasonable, and if it is a default even if not defined, why would we see such delays with our index?

This is a major problem, as staff cannot do their work when they can't find the records they've been working on.

Started GET "/resolve/edit?uri=%2Frepositories%2F12%2Ftop_containers%2F224618" for 130.132.143.66 at 2015-08-06 11:18:07 -0400
Processing by ResolverController#resolve_edit as HTML
  Parameters: {"uri"=>"/repositories/12/top_containers/224618"}
Redirected to http://aspace.library.yale.edu/plugins/top_containers/224618/edit
Completed 302 Found in 176.0ms
Started GET "/plugins/top_containers/224618/edit" for 130.132.143.66 at 2015-08-06 11:18:07 -0400
Processing by TopContainersController#edit as HTML
  Parameters: {"id"=>"224618"}
Started POST "/update_monitor/poll" for 130.132.143.66 at 2015-08-06 11:18:09 -0400
Processing by UpdateMonitorController#poll as JSON
  Parameters: {"lock_version"=>"0", "uri"=>"/repositories/12/archival_objects/2104298"}
Completed 200 OK in 204.0ms (Views: 2.0ms)
  Rendered shared/_breadcrumb.html.erb (26.0ms)
  Rendered shared/_sidebar.html.erb (3.0ms)
  Rendered /archivesspace/plugins/container_management/frontend/views/top_containers/_sidebar.html.erb (6.0ms)
  Rendered /archivesspace/plugins/container_management/frontend/views/top_containers/_toolbar.html.erb (29.0ms)
  Rendered shared/_flash_messages.html.erb (0.0ms)
  Rendered shared/_form_messages.html.erb (4.0ms)
  Rendered /archivesspace/plugins/container_management/frontend/views/container_profiles/_linker.html.erb (11.0ms)
  Rendered container_locations/_template.html.erb (0.0ms)
  Rendered shared/_subrecord_form.html.erb (15.0ms)
  Rendered /archivesspace/plugins/container_management/frontend/views/top_containers/_form.html.erb (57.0ms)
  Rendered shared/_sidebar.html.erb (5.0ms)
  Rendered /archivesspace/plugins/container_management/frontend/views/top_containers/_sidebar.html.erb (13.0ms)
  Rendered /archivesspace/plugins/container_management/frontend/views/top_containers/_toolbar.html.erb (42.0ms)
  Rendered shared/_flash_messages.html.erb (0.0ms)
  Rendered shared/_form_messages.html.erb (7.0ms)
  Rendered /archivesspace/plugins/container_management/frontend/views/container_profiles/_linker.html.erb (23.0ms)
  Rendered container_locations/_template.html.erb (1.0ms)
  Rendered shared/_subrecord_form.html.erb (16.0ms)
  Rendered /archivesspace/plugins/container_management/frontend/views/top_containers/_form.html.erb (101.0ms)
  Rendered /archivesspace/plugins/container_management/frontend/views/container_profiles/_linker.html.erb (25.0ms)
  Rendered container_locations/_template.html.erb (1.0ms)
  Rendered shared/_subrecord_form.html.erb (14.0ms)
  Rendered locations/_linker.html.erb (12.0ms)
  Rendered /archivesspace/plugins/container_management/frontend/views/top_containers/edit.html.erb within layouts/application (523.0ms)
  Rendered /archivesspace/plugins/container_management/frontend/views/layout_head.html.erb (7.0ms)
  Rendered /archivesspace/plugins/oclc/frontend/views/layout_head.html.erb (1.0ms)
  Rendered /archivesspace/plugins/yale_accessions/frontend/views/layout_head.html.erb (4.0ms)
  Rendered shared/_browser_support.html.erb (0.0ms)
  Rendered shared/_header_user.html.erb (71.0ms)
  Rendered shared/_header_global.html.erb (87.0ms)
  Rendered site/_branding.html.erb (2.0ms)
  Rendered shared/_advanced_search.html.erb (404.0ms)
  Rendered shared/_header_repository.html.erb (497.0ms)
  Rendered site/_footer.html.erb (0.0ms)
  Rendered shared/_templates.html.erb (17.0ms)
Completed 200 OK in 3774.0ms (Views: 1299.0ms)
Aug 06, 2015 11:18:14 AM org.apache.solr.core.SolrCore execute
INFO: [collection1] webapp= path=/select params={facet=true&pf=four_part_id^4&sort=&start=0&q=ms+*&qf=four_part_id^3+title^2+finding_aid_filing_title^2+fullrecord&facet.field=primary_type&facet.field=creators&facet.field=subjects&wt=json&fq=repository:"/repositories/12"+OR+repository:global&fq=types:("resource")&fq=-exclude_by_default:true&rows=10&defType=edismax} hits=3381 status=0 QTime=41654
Completed 200 OK in 44280.0ms (Views: 370.0ms)
2015-08-06 11:18:17 -0400: Running index round
#<EOFError: End of file reached>

Melissa Wisner

Yale Library IT
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20150810/fbe7efb8/attachment.html>