[Archivesspace_Users_Group] Help using AS Restful API to query

Fri Nov 8 11:53:37 EST 2019

And Steve, now that I’m looking at your question, I *think* that everything already mentioned should point you in the right direction for doing that search with the API.

It sounds like your use case might be aimed at performing very granular updates, but in case it’s helpful, here’s the approach that Hudson Molonglo (thanks James, et al.!) provided for us so that we can figure out which finding aids to export, adding a new endpoint named “/resource-update-feed” via a plugin. See https://github.com/hudmol/archivesspace_export_service/tree/master/backend .  In this case, we wanted to know not just when a resource or archival record had been edited in a finding aid, but when any of the records that link to it were (e.g. the archival object wasn’t edited, but an associated digital object was).  We use that endpoint as part of a larger service, but I’ve also found it useful in other contexts.

Additionally, it was important to figure out which records had been unpublished or suppressed since a particular time, which is why I like this approach, since the idea is that you send a request to that new endpoint, and it will return to you a list of IDs that need to be added (either because they’re brand new or because they’ve been edited), as well as a list of IDs that should be removed.  Again, this use case was just for Resource records, but I imagine that a similar approach would be useful for other record types.

Mark

From: archivesspace_users_group-bounces at lyralists.lyrasis.org [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] On Behalf Of Seth Shaw
Sent: Friday, 08 November, 2019 11:26 AM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org>
Subject: Re: [Archivesspace_Users_Group] Help using AS Restful API to query

Thanks, James and Mark. I appreciate the additional pointers. (And sorry for high-jacking your thread, Steve!)

On Fri, Nov 8, 2019 at 8:20 AM Custer, Mark <mark.custer at yale.edu<mailto:mark.custer at yale.edu>> wrote:
Seth,

Here’s another example:

search?type[]=archival_object&page=1&aq=
{
    "query": {
        "op": "AND",
        "subqueries": [
            {
                "field": "keyword",
                "value": "39002102378974",
                "jsonmodel_type": "field_query",
                "negated": false,
                "literal": false
            },
            {
                "field": "types",
                "value": "pui",
                "jsonmodel_type": "field_query",
                "negated": true
            }
        ],
        "jsonmodel_type": "boolean_query"
    },
    "jsonmodel_type": "advanced_query"
}

In that case, for example, we might have one archival object where that barcode has been applied.  In the Solr index, there will be two documents, if and only if that record has been published.  But, we can exclude the PUI document in the result set with that second part of the subquery.

Mark

From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>] On Behalf Of Seth Shaw
Sent: Friday, 08 November, 2019 11:13 AM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>>
Subject: Re: [Archivesspace_Users_Group] Help using AS Restful API to query

That might be it. We don't use the PUI for patron access; but we do still have it left on as the staff occasionally look at it. Certainly something to investigate.

Regardless, either the API should use one index consistently, have a documented filter (I don't see one there) and/or give some other obvious indication as to where a result came from.

On Fri, Nov 8, 2019 at 8:02 AM James Bullen <james at hudmol.com<mailto:james at hudmol.com>> wrote:

Hi Seth,

I’m not seeing that. Could it be you’re seeing pui docs as well - the instance I’m testing on has the pui turned off.

Cheers,
James

On Nov 8, 2019, at 10:51 AM, Seth Shaw <seth.shaw at unlv.edu<mailto:seth.shaw at unlv.edu>> wrote:

James, I was hoping using the filter as you described would remove the duplicate results issue I was having by using the advanced query compound search; but I'm seeing the same thing as before.

Running the search via the API: 'archivestest:8089/search?type[]=archival_object&page=1&filter={"query":{"comparator":"greater_than","field":"system_mtime","value":"2019-10-02","jsonmodel_type":"date_field_query"}}' returns, in part, `{"page_size":10,"first_page":1,"last_page":874,"this_page":1,"offset_first":1,"offset_last":10,"total_hits":8732,` ...

Whereas the SQL query `SELECT count(*) FROM archival_object WHERE system_mtime > '2019-10-02';` is returning "4369" (half the results of the REST query).

Have you run into this issue before?

On Fri, Nov 8, 2019 at 7:34 AM James Bullen <james at hudmol.com<mailto:james at hudmol.com>> wrote:

Something like this works for me:

/search type[]=resource type[]=archival_object page=1 filter={"query":{"comparator":"greater_than","field":"system_mtime","value":"2019-10-02","jsonmodel_type":"date_field_query"}}

On Nov 8, 2019, at 10:24 AM, Seth Shaw <seth.shaw at unlv.edu<mailto:seth.shaw at unlv.edu>> wrote:

To do this you need to use the advanced query parameter which, unfortunately, is not well documented. There are a few email threads that describe using the advanced search though:

http://lyralists.lyrasis.org/mailman/htdig/archivesspace_users_group/2015-June/001734.html<https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Flyralists.lyrasis.org%2Fmailman%2Fhtdig%2Farchivesspace_users_group%2F2015-June%2F001734.html&data=02%7C01%7Cmark.custer%40yale.edu%7C14a98c82e2d841b696f508d764686ccb%7Cdd8cbebb21394df8b4114e3e87abeb5c%7C0%7C1%7C637088271973732409&sdata=4HOPKRSdqZZdb%2FZ15TJsDOQ%2B%2FxQrFf8Msy9nE77wGQc%3D&reserved=0>

I've formulated queries like you describe before (I'll have to dig through my notes to see if I can find it again); but the result set was consistently giving me back duplicate results for some unknown reason; so I stopped using it.

Ideally, we would be able to add a 'modified_since' parameter to each entity's endpoint to get the most recent set. The capability exists in the code, but hasn't been exposed to the REST end-point. I've submitted a ticket that will hopefully lead to this being resolved: https://archivesspace.atlassian.net/browse/ANW-962?filter=-2<https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Farchivesspace.atlassian.net%2Fbrowse%2FANW-962%3Ffilter%3D-2&data=02%7C01%7Cmark.custer%40yale.edu%7C14a98c82e2d841b696f508d764686ccb%7Cdd8cbebb21394df8b4114e3e87abeb5c%7C0%7C1%7C637088271973742403&sdata=Mju0CwGm0Fqsvy5bi6G45brUiBOylJiZPYHgXoTNv1U%3D&reserved=0>.

On Fri, Nov 8, 2019 at 7:07 AM Steve Mattison <smattiso at nd.edu<mailto:smattiso at nd.edu>> wrote:
Community,

I'm new to using ArchivesSpace, and new to using the AS API.  I need to use the API to search within a given repository to find records that have been modified after a particular time (e.g. modified within the last 48 hours).  (We then want to export metadata related to those resources or archival_objects for a project we're working on.)

I have found the documentation for search-this-repository<https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Farchivesspace.github.io%2Farchivesspace%2Fapi%2F%3Fshell%23search-this-repository&data=02%7C01%7Cmark.custer%40yale.edu%7C14a98c82e2d841b696f508d764686ccb%7Cdd8cbebb21394df8b4114e3e87abeb5c%7C0%7C1%7C637088271973752395&sdata=7WZLNWnShXavBBsLzHPBDZzLutLClo%2BwNeevuUvd9eI%3D&reserved=0>, but don't know how to formulate a query to find all resources and/or archival_objects where the system_mtime is after a particular time.  I would be fine with performing two searches, one for resources and one for archival_objects, if that is required.

Any help with the syntax for the query would be much appreciated.

Thanks for your help.

--
Steve Mattison
Lead Software Engineer, Digital Library Technologies
Hesburgh Libraries

University of Notre Dame
271 Hesburgh Library
Notre Dame, IN 46556-5629
o: 574-631-8559
e: steve.mattison at nd.edu<mailto:steve.mattison at nd.edu>

[https://docs.google.com/uc?export=download&id=1eLUXKKJMijoIFzQtpJutmM1uLHON9ZDt&revid=0B-gl36FdlSuTV2ZPM1NBek5HamNiOWN1UllhT3d2VWtCU3hjPQ]
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group<https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Flyralists.lyrasis.org%2Fmailman%2Flistinfo%2Farchivesspace_users_group&data=02%7C01%7Cmark.custer%40yale.edu%7C14a98c82e2d841b696f508d764686ccb%7Cdd8cbebb21394df8b4114e3e87abeb5c%7C0%7C1%7C637088271973752395&sdata=fevX5AF8SWAFXqivwvqFMgz%2BliJVQuLyeQqyOBJDpe4%3D&reserved=0>
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group<https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Flyralists.lyrasis.org%2Fmailman%2Flistinfo%2Farchivesspace_users_group&data=02%7C01%7Cmark.custer%40yale.edu%7C14a98c82e2d841b696f508d764686ccb%7Cdd8cbebb21394df8b4114e3e87abeb5c%7C0%7C1%7C637088271973762387&sdata=wiG8sxfVwA8MbIriUUkPeTD4NSOEzCM8uQYyZoV8Ml0%3D&reserved=0>

!DSPAM:5dc588a9250937848916637!

_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group<https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Flyralists.lyrasis.org%2Fmailman%2Flistinfo%2Farchivesspace_users_group&data=02%7C01%7Cmark.custer%40yale.edu%7C14a98c82e2d841b696f508d764686ccb%7Cdd8cbebb21394df8b4114e3e87abeb5c%7C0%7C1%7C637088271973762387&sdata=wiG8sxfVwA8MbIriUUkPeTD4NSOEzCM8uQYyZoV8Ml0%3D&reserved=0>
!DSPAM:5dc58f16257571993821028! _______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group<https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Flyralists.lyrasis.org%2Fmailman%2Flistinfo%2Farchivesspace_users_group&data=02%7C01%7Cmark.custer%40yale.edu%7C14a98c82e2d841b696f508d764686ccb%7Cdd8cbebb21394df8b4114e3e87abeb5c%7C0%7C1%7C637088271973772382&sdata=Edp5U64nNxhEKyrvFCUAsKiiRlAPenwlM2dc6QJr6FM%3D&reserved=0>

!DSPAM:5dc58f16257571993821028!

_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group<https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Flyralists.lyrasis.org%2Fmailman%2Flistinfo%2Farchivesspace_users_group&data=02%7C01%7Cmark.custer%40yale.edu%7C14a98c82e2d841b696f508d764686ccb%7Cdd8cbebb21394df8b4114e3e87abeb5c%7C0%7C1%7C637088271973772382&sdata=Edp5U64nNxhEKyrvFCUAsKiiRlAPenwlM2dc6QJr6FM%3D&reserved=0>
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group<https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Flyralists.lyrasis.org%2Fmailman%2Flistinfo%2Farchivesspace_users_group&data=02%7C01%7Cmark.custer%40yale.edu%7C14a98c82e2d841b696f508d764686ccb%7Cdd8cbebb21394df8b4114e3e87abeb5c%7C0%7C1%7C637088271973782388&sdata=BEN4USjdkzt3LSt3fhp3jvCS2DfYstVnYZWv6r4Hdpg%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20191108/71aaca78/attachment.html>