[Archivesspace_Users_Group] # of Collection Mgmt records not matching # of Accession records

[McPhee, Laurel]

Steve, thank you so much for this tip.

With this information, we were able to discover we hadn’t really been purging our deleted records and running a true re-index, when we thought we were. We had only been deleting the one index-related directory, not both. It does seem to take longer to re-index when deleting both directories – so much so that I was getting really nervous that we’d messed something up.  But we’ll all good now.

Thank you so much for the solution!

Laurel


From: archivesspace_users_group-bounces at lyralists.lyrasis.org <archivesspace_users_group-bounces at lyralists.lyrasis.org> On Behalf Of Majewski, Steven Dennis (sdm7g)
Sent: Monday, January 14, 2019 10:46 AM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org>
Subject: Re: [Archivesspace_Users_Group] # of Collection Mgmt records not matching # of Accession records


I noticed, while researching this issue, that there is no  .dat file in my archivesspace/data/inderer_state directly named container_management.
Perhaps that info is kept in corresponding resource files, however, I see that there is a primary_type in Solr records for container_management.
Which makes me suspicious that there is something missing in the codebase to keep these records in sync.  ( I’m on 2.5.x, so I don’t know what would be different in 2.3.x )  It also makes me question if you did a complete reindex ( i.e. delete ALL of data/indexer_state/* and data/solr_index/ and restart ) or partial, as that might make a difference here. Usually, deleting the indexer_state files in enough, but I’m suspicious here.

I suggest you open the Solr web console to ArchivesSpace — usually at port 8090, unless you have changed the default, and enter:
primary_type: collection_management in the Query console.
Look first at response.numFound, which from your description, I expect doesn’t match the number returned from SQL query.
Then you might drill down to inspect if there are duplicate records with the same parent_id, or if there are id # returned in the solr query that have been deleted from SQL. What else you might look at depends on what clues you initially see.


BTW: I found it very simple to add collection_management_subreports to accession_report and resource_list_report in 2.5.x,
However, I think the reports subsystem was very different in 2.3.x. And that would still only give you the records that are in SQL, not the ones that aren’t, which seems to be the problem.

— Steve Majewski




On Jan 14, 2019, at 10:51 AM, McPhee, Laurel <lmcphee at ucsd.edu<mailto:lmcphee at ucsd.edu>> wrote:

Hi Blake,

The MySQL tables show:

mysql> select count(*) from accession;
+----------+
| count(*) |
+----------+
|     2702 |
+----------+
1 row in set (0.01 sec)

mysql> select count(*) from collection_management;
+----------+
| count(*) |
+----------+
|     2695 |
+----------+
1 row in set (0.00 sec)

Without knowing much about what that means, my hunch is that our “real” number of records is correct, or just off by a mere handful. But for some reason we’re seeing lots of junk appear in the staff interface filters and view. Any insights? (and sorry for confusion on my originally reported numbers, I meant to type we had 2,700 accessions, not 2,600. So the number in the tables is exactly what we expect to see there).

Laurel

From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> <archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>> On Behalf Of Blake Carver
Sent: Friday, January 11, 2019 8:21 AM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>>
Subject: Re: [Archivesspace_Users_Group] # of Collection Mgmt records not matching # of Accession records

If you look in the actual MySQL tables, what's the counts?
________________________________
From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> <archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>> on behalf of McPhee, Laurel <lmcphee at ucsd.edu<mailto:lmcphee at ucsd.edu>>
Sent: Friday, January 11, 2019 10:59:11 AM
To: Archivesspace Users Group
Subject: [Archivesspace_Users_Group] # of Collection Mgmt records not matching # of Accession records

Hi all, and happy Friday.

We’re attempting to troubleshoot a strange problem. I’d appreciate knowing if anyone else has this issue, or has any suggestions. Our database analyst has started looking into it, but it’s a puzzler.

We have 2,600 accession records. As far as I know, the Collection Mgmt subtable has a 1-to-1 relationship with the Accession table. However, if I filter a search on Collection Mgmt (Browse-->Collection Mgmt) over 3,600 records display. A careful search through the results shows that often (not always!) when we update the processing status on an accession, using the little pull-down menu, a “ghost” collection management record lingers on, and will appear in search results sets within Collection Management. So one accession record, although it looks perfectly correct and updated in the accession record staff view, will have two collection management records, one showing “processed” and one showing “unprocessed” if you search for that same record in the Collection Mgmt browse feature.

We thought a purge of our deleted records and a re-index might fix this. But, the plot thickens: at the end of the day yesterday, when we purged our deleted records and reindexed in our test instance, the Collection Mgmt table now shows over 4,000 results! It’s like it made the problem WORSE. Instead of duplicate ghosts, we now have triplicate ghosts. Does anyone else have this problem with a numbers mis-match between Coll Mgmt and Accessions?

I can’t replicate it in the Sandbox because the problem doesn’t seem to happen with “new” accession records. If I make a new test accession record, the problem won’t occur. But if I change the processing status of an older, legacy record (one that imported over from AT a few years ago), it sometimes occurs. We’re running v2.3.2.

Laurel McPhee
Supervisory Archivist, Special Collections & Archives Program
UC San Diego Library | • 858-534-5619 | • lmcphee at ucsd.edu<mailto:lmcphee at ucsd.edu>

_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20190114/0b333bfd/attachment.html>