[Archivesspace_Users_Group] Anyone making use of Harvard's aspace-import-excel plugin?

[Majewski, Steven Dennis (sdm7g)]

We have been using the plugin routinely. 
We did recently  have some problems when we tried it on a very large spreadsheet ( > 2000 rows )

It seemed to me that when the web app appeared to have finished  importing the spreadsheet, it was really still processing in the background for a long time. ( hours! ) That made it appear that it didn’t completely work. We then tried to delete and redo again, which caused even more problems. Or It might be that the archival_objects are created but not yet indexed. (*) 
So I would suggest first doing import on a test server, and to wait a long time for the job to complete. 
When we redid it again, on production, waiting overnight for completion, we got everything but a couple of rows. 
We didn’t spend any more time trying to locate the exact error on those rows — we just added the missing values manually. 
So I’m not 100% sure of my diagnosis and description of the problem. 
But even with the manual fixup, it still saved a lot of time on the other 2000 archival objects! 

But if you’re seeing issues with 100+ lines, perhaps we need to implement some post import QA check to verify that everything imported correctly. 

(*) In one case, we tried to delete archival_objects after what appeared to be an unsuccessful import, however after some were deleted, more continued to appear, and the 2nd attempt to import hit errors trying to recreate still existing records. 


— Steve Majewski


> On Jan 8, 2019, at 6:12 PM, Emily Pyers <epyers at slv.vic.gov.au> wrote:
> 
> Link to it in GitHub here: https://github.com/harvard-library/aspace-import-excel <https://github.com/harvard-library/aspace-import-excel>
>  
> We’re really excited about it (the 80% rule!) and have implemented it on our test system (v2.4.0).  We’ve been testing it and it’s mostly working fine, until we attempt to load spreadsheets that have more than around 100 lines. Attempts to load spreadsheets over this size are resulting in job timeouts and overall system slowdowns, and we’ve tested pretty thoroughly – it doesn’t seem to be an issue with the data.  Our tech team have reviewed the system logs, but are unable to pinpoint the source of the problem.
>  
> I was hoping someone out there might be able to confirm that a) they have successfully used the plugin to load spreadsheets greater than 100 lines (since this seems to be a point of scepticism for some of our internal stakeholders), and that b) if you might be able to point us towards any potential areas of investigation?
>  
> I would be so appreciative for any help on this!
>  
> Cheers,
>  
> Emily 
> 
> Emily Pyers | Metadata & Archival Systems Specialist | Collection Development & Description 
> In the office Monday, Wednesday - Friday 9.30am-3pm
> State Library Victoria | 328 Swanston Street | Melbourne VIC 3000 
> T +61 3 8664 7368 | epyers at slv.vic.gov.au <mailto:epyers at slv.vic.gov.au> 
> slv.vic.gov.au <http://slv.vic.gov.au/>
> 
>  <https://www.slv.vic.gov.au/email_campaign>	 
> 
>  <http://www.facebook.com/pages/Melbourne-Australia/State-Library-of-Victoria/32256104331>	 <http://twitter.com/Library_Vic>	 <http://www.youtube.com/user/statelibraryvictoria>	 <http://instagram.com/library_vic>	
> This message and any attachment is intended only for the use of the Addressee and may contain information that is PRIVILEGED and CONFIDENTIAL. If you are not the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please delete all copies of the message and its attachments and notify the sender immediately. Thank you. _______________________________________________
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group at lyralists.lyrasis.org <mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group <http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20190108/1c8f0273/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3598 bytes
Desc: not available
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20190108/1c8f0273/attachment.bin>