[Archivesspace_Users_Group] Disallows in robots.txt?

Blake Carver blake.carver at lyrasis.org
Wed May 15 18:42:52 EDT 2019

We've found this works pretty well:

Disallow: /search*
Disallow: /inventory/*
Disallow: /collection_organization/*
Disallow: /repositories/*/top_containers/*
Disallow: /check_session*
Disallow: /repositories/*/resources/*/tree/*

From: archivesspace_users_group-bounces at lyralists.lyrasis.org <archivesspace_users_group-bounces at lyralists.lyrasis.org> on behalf of Michelle Paquette <mpaquette at smith.edu>
Sent: Wednesday, May 15, 2019 3:30 PM
To: Archivesspace Users Group
Subject: [Archivesspace_Users_Group] Disallows in robots.txt?

Hello all,

For those of you who are already using the Public User Interface, I wondered if you had suggested disallows for the robots.txt. We're planning to disallow on search but wasn't sure of the arguments for or against other disallows. Any advice much appreciated!

Michelle Paquette

Michelle Paquette
Metadata & Technical Services Archivist
Special Collections
Smith College
mpaquette at smith.edu<mailto:mpaquette at smith.edu>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20190515/c7086bbb/attachment.html>

More information about the Archivesspace_Users_Group mailing list