[Archivesspace_Users_Group] Disallows in robots.txt?

[Blake Carver]

We've found this works pretty well:

Disallow: /search*
Disallow: /inventory/*
Disallow: /collection_organization/*
Disallow: /repositories/*/top_containers/*
Disallow: /check_session*
Disallow: /repositories/*/resources/*/tree/*


________________________________
From: archivesspace_users_group-bounces at lyralists.lyrasis.org <archivesspace_users_group-bounces at lyralists.lyrasis.org> on behalf of Michelle Paquette <mpaquette at smith.edu>
Sent: Wednesday, May 15, 2019 3:30 PM
To: Archivesspace Users Group
Subject: [Archivesspace_Users_Group] Disallows in robots.txt?

Hello all,

For those of you who are already using the Public User Interface, I wondered if you had suggested disallows for the robots.txt. We're planning to disallow on search but wasn't sure of the arguments for or against other disallows. Any advice much appreciated!

Michelle Paquette

--
Michelle Paquette
Metadata & Technical Services Archivist
Special Collections
Smith College
413-585-7029
mpaquette at smith.edu<mailto:mpaquette at smith.edu>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20190515/c7086bbb/attachment.html>