[Archivesspace_Users_Group] Hooks, Triggers, Callback ( was : Re: Code4Lib 2016 events )

Fri Jan 8 07:05:13 EST 2016

Hi,

So, I threw this together :

https://github.com/archivesspace/aspace-queue

This hooks in Resque ( a messaging library that uses Redis ) to the indexer. You can see how to do this in just a few lines here:

https://github.com/archivesspace/aspace-queue/blob/master/indexer/indexer.rb

You can watch the queue using the resque-web application that comes with Reque ( look in the documentation ).

Or you can write a resque listener to pull messages from redis to do jobs.
I think this is similar to how the Sofia / SocalSphere hydra app does it?

Is this look like the right direction? It does add another datastore (Redis), but this is an in-memory store and is pretty easy to manage for ephemeral data.

b,chris.

Chris Fitzpatrick | Developer, ArchivesSpace
Skype: chrisfitzpat  | Phone: 918.236.6048
http://archivesspace.org/

________________________________
From: archivesspace_users_group-bounces at lyralists.lyrasis.org <archivesspace_users_group-bounces at lyralists.lyrasis.org> on behalf of Galligan, Patrick <PGalligan at rockarch.org>
Sent: Friday, January 8, 2016 4:11 AM
To: 'Archivesspace Users Group'; 'Archivesspace Users Group'
Subject: Re: [Archivesspace_Users_Group] Code4Lib 2016 events

There is definitely an integrations advisory group. I'm a member and it's headed by Max Eckard.

We're putting together a google form so people can provide more information about their integrations. Until then, we will grab info from the list.

-Patrick Galligan

Sent with Good (www.good.com)

-----Original Message-----
From: Jason Loeffler [j at minorscience.com<mailto:j at minorscience.com>]
Sent: Thursday, January 07, 2016 09:34 PM Eastern Standard Time
To: Archivesspace Users Group
Subject: Re: [Archivesspace_Users_Group] Code4Lib 2016 events

I'm working with a few member organizations who'd like to integrate with their respective content management systems in order to present their collections to non-specialist, general interest audiences through their branded website, as well offer the means to further enrich resources outside of the DACS paradigm.

That said, easy enough to rough out with your favorite HTTP client/framework (Guzzle, for example) and the ASpace API, especially by passing values into search parameters. (Haven't toyed with querying the index directly.)

I don't recall, is there an "integration" advisory committee?

Jason Loeffler
Principal
Minor Science | Application Development & Metadata Strategy
Brooklyn, New York

On Thu, Jan 7, 2016 at 7:54 PM, Arnold, Hillel <harnold at rockarch.org<mailto:harnold at rockarch.org>> wrote:
I wasn’t thinking of EAD export, actually.
My thinking was more that having a messaging system in ArchivesSpace which allows people to tie systems together based on events opens the door to exporting data in other models and serializations. Which I think would be a good thing.

Hillel

From: <archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>> on behalf of Brian Hoffman <brianjhoffman at gmail.com<mailto:brianjhoffman at gmail.com>>
Reply-To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>>
Date: Thursday, January 7, 2016 at 5:22 PM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>>
Subject: Re: [Archivesspace_Users_Group] Code4Lib 2016 events

“Real-time updates” and EAD will never go nicely together. People who aren’t serializing their data to XML as an end-in-itself should consider taking EAD out of the equation and moving the data directly from point A to point B. People who do believe having archival data serialized as XML is an end-in-itself should in the long run consider an alternative to EAD that is more accommodating to granular updates.

On Jan 7, 2016, at 5:10 PM, Chris Fitzpatrick <Chris.Fitzpatrick at lyrasis.org<mailto:Chris.Fitzpatrick at lyrasis.org>> wrote:

Hi Steve,

Yeah, so that might be slightly tricky...if a bunch of components are updated, you don't want to trigger multiple EAD exports for each component update, but just a one time EAD export for all the updates. EAD export can really require a lot of SQL queries, so that could really kill your database server.

But, it that the primary use case for most folks? You want to trigger an EAD export when an associated record ( not just a resource, but a child archival object or instance ) is updated?

If so, I think your wanting to hook into the backend rather than the indexer is actually a better idea...

b,chris.

Chris Fitzpatrick | Developer, ArchivesSpace
Skype: chrisfitzpat  | Phone: 918.236.6048<tel:918.236.6048>
http://archivesspace.org/

________________________________
From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> <archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>> on behalf of Majewski, Steven Dennis (sdm7g) <sdm7g at eservices.virginia.edu<mailto:sdm7g at eservices.virginia.edu>>
Sent: Thursday, January 7, 2016 10:45 PM
To: Archivesspace Users Group
Subject: Re: [Archivesspace_Users_Group] Code4Lib 2016 events

We would have a similar use case. We have not managed to import all of our legacy EAD into ArchivesSpace, so we
will want to export EAD from ArchivesSpace to XTF whenever published resources are updated. We were considering
whether this action should be made more explicit by triggering it on an associated Event instead of just an update,
so that you could save partial updates without triggering this export chain.

I was thinking of using Sinatra’s after filters in a backend plugin to trigger a check.
I hadn’t though of hooking into the indexer.

— Steve Majewski

On Jan 7, 2016, at 4:05 PM, Arnold, Hillel <harnold at rockarch.org<mailto:harnold at rockarch.org>> wrote:

Yeah, the use cases I’m thinking of largely have to do with pushing to a discovery layer other than the public interface (or triggering an application/service that pulls data out of AS into a discovery system. There’s also the possibility of pushing to a repository solution too, I suppose. I’m sure there would be other use cases – those are just the ones that I’ve thought of thus far…

Hillel

From: <archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>> on behalf of Chris Fitzpatrick <Chris.Fitzpatrick at lyrasis.org<mailto:Chris.Fitzpatrick at lyrasis.org>>
Reply-To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>>
Date: Thursday, January 7, 2016 at 3:59 PM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>>
Subject: Re: [Archivesspace_Users_Group] Code4Lib 2016 events

Hi,

Great. Maybe we can plan some kind of meetup to do some coding work as well...

I am seeing a convergence of your request and the others for the OAI-PHM responder. I like your idea, since it I think most of it is actually pretty much already built in the indexer, since it polls the Aspace backend for created/updated/deleted records and triggers some actions to be passed off to another API ( i.e Solr's )

I think it would be pretty easy to essentially do something else, like "when a record is created, POST the JSON to this URI". Or  maybe something slightly more sophisticated, like push it messaging queue like Redis or RabbitMQ, that a second service (like a microservice ) could listen to and consume messages.

What would a use case be? Like pushing to Blacklight or Primo or Ebsco?

best,chris.

Chris Fitzpatrick | Developer, ArchivesSpace
Skype: chrisfitzpat  | Phone: 918.236.6048<tel:918.236.6048>
http://archivesspace.org/

________________________________
From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> <archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>> on behalf of Arnold, Hillel <harnold at rockarch.org<mailto:harnold at rockarch.org>>
Sent: Thursday, January 7, 2016 7:47 PM
To: Archivesspace Users Group
Subject: Re: [Archivesspace_Users_Group] Code4Lib 2016 events

Hey Chris,
I’ll be at Code4Lib too, although I’m already committed to other sessions for the preconference.

One idea we’ve been batting around here is some sort of messaging system (inspired by Github’s webhooks [1]) which would allow authorized users to activate customizable HTTP callbacks for events, for example creation, modification or deletion of a resource or component. It seems like people have taken a number of approaches to figuring out when data changes in AS, but as far as I know all of these involve some sort of regularly scheduled cron job that looks at the Solr index or makes a ton of API calls. It would be amazing if the system could send notifications out – I think it would allow for much more efficient and flexible integration with other systems.

Hillel

-----------
Hillel Arnold
Assistant Director, Head of Digital Programs
Rockefeller Archive Center
914.366.6382<tel:914.366.6382>

From: <archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>> on behalf of Chris Fitzpatrick <Chris.Fitzpatrick at lyrasis.org<mailto:Chris.Fitzpatrick at lyrasis.org>>
Reply-To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>>
Date: Thursday, January 7, 2016 at 7:29 AM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>>
Subject: [Archivesspace_Users_Group] Code4Lib 2016 events

Hi,

There was a question shortly before the holidays about some Aspace-related activities at Code4Lib this year in March.

I am planning on attending, and will be at the Code4Arc pre-conf <http://2016.code4lib.org/workshops/Code4Arc/> , which was a great session we had last year at C4L.
I am thinking of also signing up to lead a breakout session one of the days, and might put together something for a lighting talk.

Is anyone else planning on going? Are there some possible ideas that anyone has or would like to see if attending? We might be able to expand on the breakout, if there's critical mass to possibly meet more somewhere.

Best, Chris.

Chris Fitzpatrick | Developer, ArchivesSpace
Skype: chrisfitzpat  | Phone: 918.236.6048<tel:918.236.6048>
http://archivesspace.org/
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20160108/ce6353d9/attachment.html>