<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
You (or your friendly neighborhood sysadmin) can change up the Apache logs to add/remove quite a few things. There's all sorts of good stuff you can get in there:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<a href="http://httpd.apache.org/docs/current/mod/mod_log_config.html">http://httpd.apache.org/docs/current/mod/mod_log_config.html</a><br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> archivesspace_users_group-bounces@lyralists.lyrasis.org <archivesspace_users_group-bounces@lyralists.lyrasis.org> on behalf of Steele, Henry
<Henry.Steele@tufts.edu><br>
<b>Sent:</b> Wednesday, March 25, 2020 9:06 AM<br>
<b>To:</b> Archivesspace Users Group <archivesspace_users_group@lyralists.lyrasis.org><br>
<b>Subject:</b> Re: [Archivesspace_Users_Group] how to find if a certain page was accessed in the PUI</font>
<div> </div>
</div>
<style>
<!--
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
p.x_MsoNormal, li.x_MsoNormal, div.x_MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif}
a:link, span.x_MsoHyperlink
{color:#0563C1;
text-decoration:underline}
a:visited, span.x_MsoHyperlinkFollowed
{color:#954F72;
text-decoration:underline}
p
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif}
p.x_msonormal0, li.x_msonormal0, div.x_msonormal0
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif}
p.x_xmsonormal, li.x_xmsonormal, div.x_xmsonormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif}
p.x_xmsochpdefault, li.x_xmsochpdefault, div.x_xmsochpdefault
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Calibri",sans-serif}
span.x_xmsohyperlink
{color:#0563C1;
text-decoration:underline}
span.x_xmsohyperlinkfollowed
{color:#954F72;
text-decoration:underline}
span.x_xemailstyle17
{font-family:"Calibri",sans-serif;
color:#1F4E79}
span.x_EmailStyle24
{font-family:"Calibri",sans-serif;
color:#1F497D}
.x_MsoChpDefault
{font-size:10.0pt}
@page WordSection1
{margin:1.0in 1.0in 1.0in 1.0in}
div.x_WordSection1
{}
-->
</style>
<div lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="x_WordSection1">
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:#1F497D">Thank you, Blake. This is all really helpful. I will see if I can use some of these strategies to look through our logs.</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:#1F497D"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:#1F497D">It may not be there. Our pui log level is “fatal” and I’m not sure the Apache (/var/log/httpd) contain this kind of information. But it’s certainly good
to know how to look</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:#1F497D"> </span></p>
<div>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:#1F497D">Henry Steele</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:#1F497D">Systems Librarian</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:#1F497D">Tufts University Library Technology Services</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:#1F497D">(617)627-5239</span></p>
</div>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:#1F497D"> </span></p>
<div>
<div style="border:none; border-top:solid #E1E1E1 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="x_MsoNormal"><b><span style="font-size:11.0pt; font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt; font-family:"Calibri",sans-serif"> archivesspace_users_group-bounces@lyralists.lyrasis.org <archivesspace_users_group-bounces@lyralists.lyrasis.org>
<b>On Behalf Of </b>Blake Carver<br>
<b>Sent:</b> Wednesday, March 25, 2020 9:01 AM<br>
<b>To:</b> Archivesspace Users Group <archivesspace_users_group@lyralists.lyrasis.org><br>
<b>Subject:</b> Re: [Archivesspace_Users_Group] how to find if a certain page was accessed in the PUI</span></p>
</div>
</div>
<p class="x_MsoNormal"> </p>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black">I know quite a few people use Google Analytics, which is not something I find useful at all, but it's used quite often. Check matamo for an open source analytics product. There
are many others. I know matamo gives you the ability to customize things, and I bet it could be quite useful, though I've not touched it in many years. </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black">I think your best bet is to get to know your Apache logs. You should be able to get something useful out of there, but you'll need to learn what your logging there, and maybe
change it up. Read up on Apache's "LogFormat" , it's pretty flexible and you can customize that on your server. You can also customize where log files end up for which domain name, so that might help as well.
<span style="background:white"> If you're running the PUI and STAFF sides on different URLs, or prefixes, that will help set them apart for logging. </span>This is all one of those "It Depends" kinds of things. Using grep/awk/sed etc... will let you pull out
different things from the logs. Try tailing the log as you look at different things on the site and see how those get logged, then work up some simple greps to pull out just what you need every day. This is a simple one I use to see the busiest sites on a
server: </span></p>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
</div>
</div>
<div>
<p class="x_MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Calibri",sans-serif; color:black">cat /var/log/apache2/other_vhosts_access.log.1 | awk {'print $1'} | sort |uniq -c |sort -nr | head -20</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black">(If you're looking at that and thinking "You don't need cat in there, dummy" I know I know, old habits die hard)</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black">You could do the same kind of grep work on the archivesspace.out log file and get something out of it. You might need to experiment with loglevel on that to see what you can
get. DEBUG is probably way too much. </span></p>
</div>
<div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black">Here's some real nginx logs... these are based on real logs with some details changed to protect the innocent.</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black">Here's one you might see quite often, if someone is logged into the staff side you'll see this POST to check their session:</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black">4.4.4.4 example.edu - [25/Mar/2020:12:32:22 +0000] "POST /update_monitor/poll HTTP/1.1" 200 4751 "<a href="https://example.edu/resources/134/edit">https://example.edu/resources/134/edit</a>"
"lock_version=12&uri=%2Frepositories%2F5%2Fresources%2F134" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:71.0) Gecko/20100101 Firefox/71.0" "-"</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black">Here's another one, someone is looked at a resource on the staff side:</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black">4.4.4.4 example.edu - [25/Mar/2020:12:31:03 +0000] "GET /resources/2774?inline=true&undefined_id=%2Frepositories%2F3%2Fresources%2F2774 HTTP/1.1" 200 9839 "<a href="https://example.edu/resources/2774">https://example.edu/resources/2774</a>"
"-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36" "-"</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black">And here's a bot crawling the public side.</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black">216.244.66.240 example.edu - [25/Mar/2020:12:29:58 +0000] "GET /repositories/2/archival_objects/97930 HTTP/1.1" 200 21473 "-" "-" "Mozilla/5.0 (compatible; DotBot/1.1;
<a href="http://www.opensiteexplorer.org/dotbot">http://www.opensiteexplorer.org/dotbot</a>,
<a href="mailto:help@moz.com">help@moz.com</a>)" "-"</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black">Depending on how you configure your Apache/nginx/whatever logs, those log lines will look different and you can log a bunch of different things.
</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black">On the ArchivesSpace side (archivesspace/logs/archivesspace.out) the logs can look different depending on your log level. Here's one set to debug showing the indexer doing some
work:</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black">INFO: [collection1] webapp= path=/update params={} {add=[/repositories/2/archival_objects/33921#pui, /repositories/2/archival_objects/33922#pui, /repositories/2/archival_objects/33923#pui,
/repositories/2/archival_objects/33924#pui, /repositories/2/archival_objects/33925#pui, /repositories/2/archival_objects/33926#pui, /repositories/2/archival_objects/33927#pui, /repositories/2/archival_objects/33928#pui, /repositories/2/archival_objects/33929#pui,
/repositories/2/archival_objects/33930#pui, ... (25 adds)]} 0 6</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black">Here's one line from me viewing a resource on the staff side, as you can see it'll be a bit more challenging to get useful stuff out of this log, but it's in there:</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black">[2020-03-25T08:45:29-04:00] INFO: [collection1] webapp= path=/select params={facet.field=assessment_record_types&facet.field=assessment_surveyors&facet.field=assessment_review_required&facet.field=assessment_reviewers&facet.field=assessment_completed&facet.field=assessment_inactive&facet.field=assessment_survey_year&facet.field=assessment_sensitive_material&csv.escape=\&start=0&q.op=AND&fq=repository:"/repositories/3"+OR+repository:global&fq=types:("assessment")&fq=(-types:("pui_only")+AND+(assessment_record_uris:("\/repositories\/3\/resources\/406")))&fq=-exclude_by_default:true&sort=&rows=30&bq=primary_type:resource^100&q=*:*&facet.limit=20&defType=edismax&qf=four_part_id^3+title^2+finding_aid_filing_title^2+fullrecord&pf=four_part_id^4&csv.header=true&csv.encapsulator="&facet.mincount=0&wt=json&facet=true}
hits=0 status=0 QTime=61 </span></p>
</div>
<p class="x_MsoNormal"><span style="font-family:"Calibri",sans-serif; color:black"> </span></p>
</div>
</div>
<div class="x_MsoNormal" align="center" style="text-align:center">
<hr size="2" width="98%" align="center">
</div>
<div id="x_divRplyFwdMsg">
<p class="x_MsoNormal"><b><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:black">From:</span></b><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:black">
<a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org">archivesspace_users_group-bounces@lyralists.lyrasis.org</a> <<a href="mailto:archivesspace_users_group-bounces@lyralists.lyrasis.org">archivesspace_users_group-bounces@lyralists.lyrasis.org</a>>
on behalf of Steele, Henry <<a href="mailto:Henry.Steele@tufts.edu">Henry.Steele@tufts.edu</a>><br>
<b>Sent:</b> Wednesday, March 25, 2020 7:54 AM<br>
<b>To:</b> Archivesspace Users Group <<a href="mailto:archivesspace_users_group@lyralists.lyrasis.org">archivesspace_users_group@lyralists.lyrasis.org</a>><br>
<b>Subject:</b> [Archivesspace_Users_Group] how to find if a certain page was accessed in the PUI</span>
</p>
<div>
<p class="x_MsoNormal"> </p>
</div>
</div>
<div>
<div>
<p class="x_xmsonormal"><span style="color:#1F4E79">Good morning,</span></p>
<p class="x_xmsonormal"><span style="color:#1F4E79"> </span></p>
<p class="x_xmsonormal"><span style="color:#1F4E79">We recently made our PUI open the public and we are trying to find out about usage, particularly of a certain page within our repository. I’m trying to figure out if there’s any way to see this in the logs.
</span></p>
<p class="x_xmsonormal"><span style="color:#1F4E79"> </span></p>
<p class="x_xmsonormal"><span style="color:#1F4E79">I’ve looked in the application log archivesspace.out, but I’m not sure what I’m seeing here. I see records being accessed, with a response of 200, but I don’t know if this is the staff interface, the PUI,
or if it’s some indexing activity. Is there a way in the application log to see if a certain page has been accessed in the PUI? We have our log level set to “fatal” for the PUI, and the “pui_log” is default. I know should mean the log only reports on problematic
events, but since I see a lot of activity in the log, I’m wondering if this setting doesn’t actually have effect.</span></p>
<p class="x_xmsonormal"><span style="color:#1F4E79"> </span></p>
<p class="x_xmsonormal"><span style="color:#1F4E79">Alternately, does anyone know if there might be other server logs that would be of use? I’m looking in the Apache logs at /var/log/httpd but I’m not sure which of this logs would contain such information
if any.</span></p>
<p class="x_xmsonormal"><span style="color:#1F4E79"> </span></p>
<p class="x_xmsonormal"><span style="color:#1F4E79">Any information you had would be of great help. Thanks</span></p>
<p class="x_xmsonormal"><span style="color:#1F4E79"> </span></p>
<p class="x_xmsonormal"><span style="color:#1F4E79">If this isn’t </span></p>
<p class="x_xmsonormal"><span style="color:#1F4E79"> </span></p>
<p class="x_xmsonormal"><span style="color:#1F4E79">Henry Steele</span></p>
<p class="x_xmsonormal"><span style="color:#1F4E79">Systems Librarian</span></p>
<p class="x_xmsonormal"><span style="color:#1F4E79">Tufts University Library Technology Services</span></p>
<p class="x_xmsonormal"><span style="color:#1F4E79">(617)627-5239</span></p>
<p class="x_xmsonormal"> </p>
</div>
</div>
</div>
</div>
</body>
</html>