[Archivesspace_Users_Group] Intermittent Outages

[Chelsea Lobdell]

Hi Alex,

Not sure if this is going to be helpful or not but we were seeing a lot of
CPU load warnings on our dev server that would bring the frontend of the
application to a grinding halt until it was restarted. I was unable to see
anything in the logs or any sort of malicious traffic so I put a few stop
gaps in place. First, I adjusted IP Table and Firewall rules to lock down
the application to only campus IP ranges. The application is only used by
staff currently and they know if they are coming from off campus they need
to use EZProxy or VPN for access. Secondly, I set a nightly restart cron
from the root user's crontab:
# restart archivesspace every night at 2am
00 02 * * * /etc/init.d/archivesspace stop
05 02 * * * /etc/init.d/archivesspace start

We haven't experience CPU load warnings since I put these stop gaps in
place. So I'm still uncertain of what was causing the issue but maybe these
stop gaps can help you too.

- Chelsea

*---------------*
*Chelsea Lobdell*
*Library Web Developer/ Swarthmore College*
*clobdel1 at swarthmore.edu <clobdel1 at swarthmore.edu> / (610)690-6818*

On Tue, Feb 28, 2017 at 12:27 PM, Alexander Duryee <alexanderduryee at nypl.org
> wrote:

> Our production instance of ASpace has been suffering from intermittent
> outages for the past few weeks.  Roughly once or twice a day, the
> application will go down for 1-5 minutes, then bring itself back up.
> Perplexingly, this doesn't seem to impact the backend - for example, the
> application went down during a reindex, but the reindex continued (and was
> logged) during the outage.  However, API requests are dropped and the
> frontend is inaccessible during the outage.
>
> We haven't identified a cause for the outages (e.g. heavy traffic, lots of
> database calls), as there's no consistent pattern.  There's nothing in
> archivesspace.out to indicate any troubles.
>
> Has anyone else experienced similar issues?  Is there another log that we
> could refer to (e.g. ASpace's Apache instance) to diagnose the outages?
>
> Thanks!
> --Alex
>
> --
> Alexander Duryee
> Metadata Archivist
> New York Public Library
> (917)-229-9590 <(917)%20229-9590>
> alexanderduryee at nypl.org
>
> _______________________________________________
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group at lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20170228/0ab8487b/attachment.html>