[Archivesspace_Users_Group] Reporting update request

Custer, Mark mark.custer at yale.edu
Thu Jan 21 14:03:41 EST 2016

Hi Chris,

Thanks for the examples, which are always appreciated!

As to why we haven't made this particular example into a report, that reason is threefold:

1.       We just (finally!) upgraded to version 1.4.2, so Jasper reports are now (almost) an option for us.

2.       We don't have any reports working in our server environment just yet (and I'm outside of that work, so I'm not entirely sure what the holdup is there).  I do have the Jasper reports, both packaged and custom, working fine on a local machine, though, so that roadblock should be addressed.... eventually.

3.       The other reason is because I don't know of a documented way yet to pass parameters to newly-created reports.  The only parameters currently passed in the reports module are contained in the last 3 reports (last in the list of reports, since I believe that these 3 reports were actually the first available in a previous release), which are run completely within ASpace, using the API to get data out, and not with Jasper, right?  I also haven't heard of anyone who has those last 3 API reports set up correctly in version 1.4.2 (but I could get the Jasper reports to run fine, on a fresh, local install at least).  There was another thread on the listserv about those 3 reports, but I don't know if a resolution was provided.  And without being able to pass those date parameters for this specific report, we need to do run them outside of the ASpace application for the time being.  I also got an error while trying to run one of those 3 API reports on test.aspace, http://test.archivesspace.org/jobs/30, and ditto (although a different error), when trying to run things on the sandbox, http://sandbox.archivesspace.org/reports

I really appreciate all of the other tips, and at some point I'll go through those examples and experiment with a few other options since I really would like to do be able to do this reporting with the API in addition to Jasper.

All that said, the feedback I still keep hearing from our users, at least (and at least one other member of the Report subteam) is that they want to be able to take a query from their advanced search, and then export that data.  I still see a need for more detailed, PDF-style reports for those annual reporting needs (and Jasper seems a good fit to me for that, since it's open-source), but there's also the more immediate need of the daily user - and often those users just want to get some data out of the application, quickly, for whatever reason.


From: archivesspace_users_group-bounces at lyralists.lyrasis.org [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] On Behalf Of Chris Fitzpatrick
Sent: Thursday, January 21, 2016 12:02 PM
To: Archivesspace Users Group
Subject: Re: [Archivesspace_Users_Group] Reporting update request

Hi Mark,

The resources we had on the project team for reports has transitioned off the project with the expectation that the community would be creating their own reports. Jasper was seen as a primary way for doing this, but there seems to be some mixed reactions to that. So, you have the SQL query and you're comfortable using Jasper...why haven't you made this into a report? What is the road block?

In regards to getting data out from the API, can do it with the aspace API (which would give you JSON) but it's also actually pretty easy to do with the Solr API.

The query :


Essential is just passed to Solr as:


All you have to do is change the wt ( response format ) to csv and the rows to something like 100000.

( you can drop the facet stuff, since you're not needing to display that ).

Solr query syntax is pretty well documented. <https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.apache.org_solr_SolrQuerySyntax&d=AwMF-g&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=s7ciGQfUJeaV_ryx908hbeXDoU9aqDwDN0Z0VbfsJ3Y&m=hFfmAru0om87a7nxCojHz0OfAx01FLvwPsPBmA-ufDI&s=S3XIHi0sbPjFlieRfz5Q7Ym0oME64mWfdp3UbXObfHc&e=>

And there's the Solr control panel that is available at port :8090 .

A pro tip is to have apace running and watch the log as you submit some queries. You'll see the request go to the frontend and then see it pass to the backend api, then passed to Solr.

For Solr search, the log will look like :

INFO: [collection1] webapp= path=/select params={ ....

Take whats in params ( it'll start with a ?q= ) and just past that into http://your.aspace.org:8090/collection1/select?q<https://urldefense.proofpoint.com/v2/url?u=http-3A__your.aspace.org-3A8090_collection1_select-3Fq&d=AwMF-g&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=s7ciGQfUJeaV_ryx908hbeXDoU9aqDwDN0Z0VbfsJ3Y&m=hFfmAru0om87a7nxCojHz0OfAx01FLvwPsPBmA-ufDI&s=oFw_iB0KZIWxOjRQ8dEr58D1YjEQ36YNe_w1oAArdKM&e=>......

Make sense?


Chris Fitzpatrick | Developer, ArchivesSpace
Skype: chrisfitzpat  | Phone: 918.236.6048

From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> <archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>> on behalf of Custer, Mark <mark.custer at yale.edu<mailto:mark.custer at yale.edu>>
Sent: Thursday, January 21, 2016 3:06 PM
To: Archivesspace Users Group
Subject: Re: [Archivesspace_Users_Group] Reporting update request


Speaking as a former member of the UAC Reports subteam, we found that the most (only?) useful report for former AT users was the print-screen report.  Given that, and in addition to feedback from staff here, it sounds to me like the biggest help of all for reporting would be the following:

*         Staff could specify what columns they would like to display in a the search result screen (and multiple dates could display as a list in a single column, whereas description from the notes table couldn't display, just like they didn't in the AT)

*         Staff could take any search result that they produced, like this one, http://sandbox.archivesspace.org/advanced_search?advanced=true&dop1=greater_than&dop2=lesser_than&f0=keyword&f1=create_time&f2=create_time&f3=suppressed&filter_term%5B%5D=%7B%22primary_type%22%3A%22accession%22%7D&op1=AND&op2=AND&op3=AND&t0=text&t1=date&t2=date&t3=boolean&v0=pape%2A&v1=2015-06-30&v2=2016-07-01&v3=false<https://urldefense.proofpoint.com/v2/url?u=http-3A__sandbox.archivesspace.org_advanced-5Fsearch-3Fadvanced-3Dtrue-26dop1-3Dgreater-5Fthan-26dop2-3Dlesser-5Fthan-26f0-3Dkeyword-26f1-3Dcreate-5Ftime-26f2-3Dcreate-5Ftime-26f3-3Dsuppressed-26filter-5Fterm-255B-255D-3D-257B-2522primary-5Ftype-2522-253A-2522accession-2522-257D-26op1-3DAND-26op2-3DAND-26op3-3DAND-26t0-3Dtext-26t1-3Ddate-26t2-3Ddate-26t3-3Dboolean-26v0-3Dpape-252A-26v1-3D2015-2D06-2D30-26v2-3D2016-2D07-2D01-26v3-3Dfalse&d=AwMF-g&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=s7ciGQfUJeaV_ryx908hbeXDoU9aqDwDN0Z0VbfsJ3Y&m=hFfmAru0om87a7nxCojHz0OfAx01FLvwPsPBmA-ufDI&s=STagCl1jnT8OQFR5iqydW9RbBmbfdyI17QWX3dyCEEM&e=>, and then export those results as a CSV file.  And let's pretend, in this case, that the user has also added columns for the Identifier and the Accession Date.  Also, every page of results would need to be exported.  In this example, there are just 3 results, but even if there were 3,333 results, then all of those results should be exported into a single CSV file.

>From my understanding, that's the biggest user request:  as a staff user, I want to select what fields display for my search results (title isn't enough), perform an advanced search, and then export my results.

Right now, for our statistical reports, we just run these "reports" to get our data out with a read-only MySQL user with a few SQL scripts, like this one:

select value as 'Accession type'

, COUNT(*) as 'Total accessions measured in linear feet'

, ROUND(SUM(extent.number), 2) as 'Linear feet'

from accession

left join extent on accession.id = extent.accession_id

left join enumeration_value on acquisition_type_id = enumeration_value.id


(extent_type_id IN (select id

from enumeration_value

where LOWER(value) like '%linear%'))

and repo_id = 11 #hardcoded value for now

and accession.accession_date >= '20151001' #change dates as needed

and accession.accession_date <= '20151231'

group by acquisition_type_id;


P.S.  All that said, I really like Jasper, actually!  But until it's easy to pass in different parameters in the staff interface when running a report (date ranges, search terms, etc.), I don't think it'll be as useful to most ASpace users as being able to export all of their search results in some fashion.

From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] On Behalf Of Chris Fitzpatrick
Sent: Thursday, January 21, 2016 7:51 AM
To: Archivesspace Users Group
Subject: Re: [Archivesspace_Users_Group] Reporting update request

I can help in regards to getting data out via the API. What are some things you're wanting to get out?


Chris Fitzpatrick | Developer, ArchivesSpace
Skype: chrisfitzpat  | Phone: 918.236.6048


From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> <archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>> on behalf of Carll, Johanna <jcarll at radcliffe.harvard.edu<mailto:jcarll at radcliffe.harvard.edu>>
Sent: Wednesday, January 20, 2016 5:46 PM
To: Archivesspace Users Group
Subject: [Archivesspace_Users_Group] Reporting update request

Hi all

As we begin the second half of the fiscal year and plan for how we will produce our end of year statistical reports, it would be useful to have an update on AS reports.  Specifically, is there a timeline for when we can expect a release that includes improved functionality of the existing reports (date limiting, improved csv exports, etc.)?  Also, have there been any further developments on the approach proposed in the below report from the Reports sub-group in the UAC minutes from November 5th?

currently testing reports to assist features prioritization sub-team; team wants to approach reports in a different way-get data out to use in own way; Brad Westbrook will talk to programmers about getting data out via API; sub-team feels that Jasper is not user friendly-stored reports are difficult to edit or customize unless you are a programmer and difficult to write canned reports that can be used by multiple repositories



Johanna Carll
Archivist and Metadata Specialist
Schlesinger Library
Radcliffe Institute for Advanced Study
Harvard University
10 Garden Street
Cambridge, MA 02138
jcarll at radcliffe.harvard.edu<mailto:jcarll at radcliffe.harvard.edu>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20160121/9e8fe800/attachment.html>

More information about the Archivesspace_Users_Group mailing list