[Archivesspace_Users_Group] Size of Database

[Custer, Mark]

I just asked for an update about the size of our Solr index that's actually in production, and right now it's around 12 GB (quite a bit less than the 18 GB I mentioned previously, which is what I have on my desktop).

I don't know if there's a good way to estimate the size of the Solr index in relation to the MySQL database (especially given that some folks, like us, might have plugins that increase the size of both :)), but based on what Kevin and I reported, it seems like your could ballpark the size of the Solr index by assuming that it'll be around 3-4 times the size of the MySQL database.

That said, the number of compressed backups that you keep will also affect the amount of disk space that you need.  I assume that we're moving backup copies of our MySQL database somewhere else for safe keeping, however, and I think that we've updated our ASpace configuration file to only make Solr backups once per day, and to only keep one of those around at a time.

Mark



From: archivesspace_users_group-bounces at lyralists.lyrasis.org [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] On Behalf Of Kerr, Susan
Sent: Wednesday, June 24, 2015 9:30 AM
To: Archivesspace Users Group
Subject: Re: [Archivesspace_Users_Group] Size of Database

Great information.  I'll look forward to the technical documentation and timing will be good for our needs.  Thanks!  Susan

From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] On Behalf Of Custer, Mark
Sent: Tuesday, June 23, 2015 8:34 PM
To: Archivesspace Users Group
Subject: Re: [Archivesspace_Users_Group] Size of Database

Susan,

I don't know if there are guidelines out there yet about all of the technical resources that institutions will need for ArchivesSpace, but we're trying to figure that out right now, as well, and then have our IT folks document that information (not so much to plan for the size of the database, which isn't terribly large, but to make sure that we're allotting enough RAM, etc., and that we have everything optimized).  All of which is to say that hopefully in a week or so we will have more information to share about what works best at our scale!

As for what exactly our scale is, right now our ArchivesSpace database is about 3.5 GB.  We combined a few AT databases into one, though, so I wouldn't assume that most institutions would have more than 1 GB of data in their database, if that (but I could be way off base with that assumption).  In any event, here are a few stats from our ArchivesSpace database to give you a slightly better sense about its contents:

Accessions:         21,106
Resources:          5,279
Archival Components (subordinate levels of desc., not counting the resource records):   2,097,805 (and at some point, we might not want so many records in the same table! :))

But that's just the size of the MySQL database.  There's also the size of the SOLR index to consider, which can be substantially larger. I've got one on my desktop right now, for instance, where the SOLR directories take up about 18 GB -- the MySQL database is the same size, in that case.

As I mentioned, though, hopefully we'll have more information to share in the near future regarding the exact details of our setup (including details about the webserver, etc.).  In the meantime, I'd love to hear more about what you mean by significant data, as well as learn more about the different ASpace database sizes already out there in production.

All my best,

Mark


P.S.  one nice thing about ArchivesSpace is that it's pretty easy to just set up everything as a test on a local machine.  After you do that (or, better yet, get someone in IT to do that so that they can see for themselves), you can then import as much data into your test application that you want, and that should give you a much better estimate regarding your database size and SOLR index once it's up on a server, as well as the types of resources that you might need.



From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] On Behalf Of Kerr, Susan
Sent: Monday, June 22, 2015 3:17 PM
To: archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>
Subject: [Archivesspace_Users_Group] Size of Database

I have been asked how much space the ArchivesSpace database will require.  What has been the experience of those with significant data.

Thanks,
Susan
------------------------------------
Susan Kerr
Systems Librarian
704-894-2424, 704-894-2625 (fax)
sukerr at davidson.edu<mailto:sukerr at davidson.edu>

DAVIDSON COLLEGE LIBRARY
209 RIDGE RD
BOX 7200
DAVIDSON, NC 28035-7200




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20150624/5ad49d4d/attachment.html>