[Archivesspace_Users_Group] Trying to spin up a new server and it won't load... Unhandled Java exception: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8089 error
Gadsby, Eric T.
egadsby at towson.edu
Fri May 19 13:59:48 EDT 2023
John,
I made the suggestion you suggested and now the server boots with both Splunk and ArchivesSpace running. I still can’t get to it from browsers I am still waiting to hear back from our firewall group. Thanks!
[Towson University logo]<http://www.towson.edu/>
Eric T. Gadsby
Pronouns: he/him/his
IT Operations Specialist | Albert S. Cook Library
—
P: 410-704-3340
egadsby at towson.edu<mailto:egadsby at towson.edu> | libraries.towson.edu<http://www.towson.edu/https:/libraries.towson.edu>
—
Confidentiality Notice: This message may contain information that is confidential, privileged, proprietary, or otherwise legally exempt from disclosure. If you are not the intended recipient, you are notified that you are not authorized to read, print, copy or disseminate this message, any part of it, or any attachments. If this message has been sent to you in error, please notify the sender by replying to this transmission, or by calling Albert S. Cook Library at 410-704-3340 .
From: archivesspace_users_group-bounces at lyralists.lyrasis.org <archivesspace_users_group-bounces at lyralists.lyrasis.org> on behalf of Rees, John (NIH/NLM) [E] <reesj at mail.nlm.nih.gov>
Date: Friday, May 19, 2023 at 11:58 AM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org>
Subject: Re: [Archivesspace_Users_Group] Trying to spin up a new server and it won't load... Unhandled Java exception: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8089 error
[ CAUTION: This email is from outside of TU. Use caution before clicking links or opening attachments. If suspicious, report to phishing at towson.edu. ]
We also have SPLUNKD running across all our local PCs, so my local/C: drive aspace v3.3.1 runs on port 8087 (or any other port not list in the config) to get around the cannot bind error.
#
## The ArchivesSpace backend listens on port 8089 by default. You can set it to
## something else below.
AppConfig[:backend_url] = http://localhost:8087
#
## The ArchivesSpace staff interface listens on port 8080 by default. You can
## set it to something else below.
#AppConfig[:frontend_url] = http://localhost:8080
#
## The ArchivesSpace public interface listens on port 8081 by default. You can
## set it to something else below.
#AppConfig[:public_url] = http://localhost:8081
#
## The ArchivesSpace OAI server listens on port 8082 by default. You can
## set it to something else below.
#AppConfig[:oai_url] = http://localhost:8082
#
## The ArchivesSpace Solr index listens on port 8090 by default. You can
## set it to something else below.
#AppConfig[:solr_url] = http://localhost:8090
#
## The ArchivesSpace indexer listens on port 8091 by default. You can
## set it to something else below.
#AppConfig[:indexer_url] = http://localhost:8091
#
## The ArchivesSpace API documentation listens on port 8888 by default. You can
## set it to something else below.
#AppConfig[:docs_url] = http://localhost:8888
From: archivesspace_users_group-bounces at lyralists.lyrasis.org <archivesspace_users_group-bounces at lyralists.lyrasis.org> On Behalf Of Gadsby, Eric T.
Sent: Friday, May 19, 2023 11:37 AM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org>
Subject: [EXTERNAL] Re: [Archivesspace_Users_Group] Trying to spin up a new server and it won't load... Unhandled Java exception: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8089 error
A search of the config could not find anything.
[Towson University logo]<http://www.towson.edu/>
Eric T. Gadsby
Pronouns: he/him/his
IT Operations Specialist | Albert S. Cook Library
—
P: 410-704-3340
egadsby at towson.edu<mailto:egadsby at towson.edu> | libraries.towson.edu<http://www.towson.edu/https:/libraries.towson.edu>
—
Confidentiality Notice: This message may contain information that is confidential, privileged, proprietary, or otherwise legally exempt from disclosure. If you are not the intended recipient, you are notified that you are not authorized to read, print, copy or disseminate this message, any part of it, or any attachments. If this message has been sent to you in error, please notify the sender by replying to this transmission, or by calling Albert S. Cook Library at 410-704-3340 .
From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> <archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>> on behalf of Blake Carver <blake.carver at lyrasis.org<mailto:blake.carver at lyrasis.org>>
Date: Friday, May 19, 2023 at 11:31 AM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>>
Subject: Re: [Archivesspace_Users_Group] Trying to spin up a new server and it won't load... Unhandled Java exception: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8089 error
[ CAUTION: This email is from outside of TU. Use caution before clicking links or opening attachments. If suspicious, report to phishing at towson.edu<mailto:phishing at towson.edu>. ]
Is this set in your config?
AppConfig[:enable_solr] = false
________________________________
From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> <archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>> on behalf of Gadsby, Eric T. <egadsby at towson.edu<mailto:egadsby at towson.edu>>
Sent: Friday, May 19, 2023 11:22 AM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>>
Subject: Re: [Archivesspace_Users_Group] Trying to spin up a new server and it won't load... Unhandled Java exception: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8089 error
ps -ef | grep java =
“java -server -Xms512m -Xmx512m -XX:+UseG1GC -XX:+PerfDisableSharedMem -XX:+ParallelRefProcEnabled -XX :MaxGCPauseMillis=250 -XX:+UseLargePages -XX:+AlwaysPreTouch -XX:+ExplicitGCInvokesConcurrent -Xlog:gc*:file=/var/solr/logs/solr_gc.log:time,uptime:filec ount=9,filesize=20M -Dsolr.jetty.inetaccess.includes= -Dsolr.jetty.inetaccess.excludes= -Dsolr.log.dir=/var/solr/logs -Djetty.port=8983 -DSTOP.PORT=7983 -DSTOP.KEY=solrrocks -Duser.timezone=UTC -XX:-OmitStackTraceInFastThrow -XX:OnOutOfMemoryError=/opt/solr/bin/oom_solr.sh 8983 /var/solr/logs -Djetty.home =/opt/solr/server -Dsolr.solr.home=/var/solr/data -Dsolr.data.home= -Dsolr.install.dir=/opt/solr -Dsolr.default.confdir=/opt/solr/server/solr/configsets/ _default/conf -Dlog4j.configurationFile=/var/solr/log4j2.xml -Xss256k -Dsolr.log.muteconsole -jar start.jar --module=http --module=gzip
egadsby+ 6561 2546 0 11:18 pts/0 00:00:00 grep --color=auto java”
Right now it just seems to be Solr running…. Based on my read of this output. I might be wrong.
It is a new 3.3.1 server with a fresh install with the data base from an old server being imported from a SQL dump and then migrated using the script. Thanks!
[Towson University logo]<http://www.towson.edu/>
Eric T. Gadsby
Pronouns: he/him/his
IT Operations Specialist | Albert S. Cook Library
—
P: 410-704-3340
egadsby at towson.edu<mailto:egadsby at towson.edu> | libraries.towson.edu<http://www.towson.edu/https:/libraries.towson.edu>
—
Confidentiality Notice: This message may contain information that is confidential, privileged, proprietary, or otherwise legally exempt from disclosure. If you are not the intended recipient, you are notified that you are not authorized to read, print, copy or disseminate this message, any part of it, or any attachments. If this message has been sent to you in error, please notify the sender by replying to this transmission, or by calling Albert S. Cook Library at 410-704-3340 .
From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> <archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>> on behalf of Tom Hanstra <hanstra at nd.edu<mailto:hanstra at nd.edu>>
Date: Friday, May 19, 2023 at 11:14 AM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>>
Subject: Re: [Archivesspace_Users_Group] Trying to spin up a new server and it won't load... Unhandled Java exception: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8089 error
[ CAUTION: This email is from outside of TU. Use caution before clicking links or opening attachments. If suspicious, report to phishing at towson.edu<mailto:phishing at towson.edu>. ]
When connected to the server, use the command:
ps -ef | grep java
and look to see if there are java processes running. If there is an old one, you will have to stop/kill it before starting the new one.
Is this a completely new server only running 3.3.1 ? If so, the solr process will also show up using that command.
Tom
On Fri, May 19, 2023 at 10:55 AM Gadsby, Eric T. <egadsby at towson.edu<mailto:egadsby at towson.edu>> wrote:
I wondered that, how would I check? Going to a browser yields nothing… that said my firewall rules may not be in place? Thanks!
[Towson University logo]<http://www.towson.edu/>
Eric T. Gadsby
Pronouns: he/him/his
IT Operations Specialist | Albert S. Cook Library
—
P: 410-704-3340
egadsby at towson.edu<mailto:egadsby at towson.edu> | libraries.towson.edu<http://www.towson.edu/https:/libraries.towson.edu>
—
Confidentiality Notice: This message may contain information that is confidential, privileged, proprietary, or otherwise legally exempt from disclosure. If you are not the intended recipient, you are notified that you are not authorized to read, print, copy or disseminate this message, any part of it, or any attachments. If this message has been sent to you in error, please notify the sender by replying to this transmission, or by calling Albert S. Cook Library at 410-704-3340 .
From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> <archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>> on behalf of Tom Hanstra <hanstra at nd.edu<mailto:hanstra at nd.edu>>
Date: Friday, May 19, 2023 at 10:53 AM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>>
Subject: Re: [Archivesspace_Users_Group] Trying to spin up a new server and it won't load... Unhandled Java exception: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8089<http://0.0.0.0/0.0.0.0:8089> error
[ CAUTION: This email is from outside of TU. Use caution before clicking links or opening attachments. If suspicious, report to phishing at towson.edu<mailto:phishing at towson.edu>. ]
First thought would be that the new service is trying to start while the old one is still running. Could that be?
Tom
On Fri, May 19, 2023 at 10:44 AM Gadsby, Eric T. <egadsby at towson.edu<mailto:egadsby at towson.edu>> wrote:
Dear Friends,
I am in the process of moving our old v2.7.1 to V3.3.1. I have been able to set up Solr, the application, move the data and start ArchivesSpace twice in our testing environment (CentOS8 Stream). Today I ran the same procedure in our production environment (RHEL8.5) and ArchivesSpace won't start.
The log reports this error: "Unhandled Java exception: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8089<http://0.0.0.0/0.0.0.0:8089>" -- of at least I think that's the problem (I'm going to provide the whole log:
"$ cat archivesspace.out
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Loading ArchivesSpace configuration file from path: /opt/archivesspace/config/config.rb
Loading ArchivesSpace configuration file from path: /opt/archivesspace/config/config.rb
Loading ArchivesSpace configuration file from path: /opt/archivesspace/config/config.rb
May 19, 2023 10:01:02 AM org.eclipse.jetty.util.log.Log initialized
INFO: Logging initialized @7281ms to org.eclipse.jetty.util.log.Slf4jLog
May 19, 2023 10:01:03 AM org.eclipse.jetty.server.Server doStart
INFO: jetty-9.4.44.v20210927; built: 2021-09-27T23:02:44.612Z; git: 8da83308eeca865e495e53ef315a249d63ba9332; jvm 11.0.19+7-LTS
May 19, 2023 10:01:03 AM org.eclipse.jetty.server.session.DefaultSessionIdManager doStart
INFO: DefaultSessionIdManager workerName=node0
May 19, 2023 10:01:03 AM org.eclipse.jetty.server.session.DefaultSessionIdManager doStart
INFO: No SessionScavenger set, using defaults
May 19, 2023 10:01:03 AM org.eclipse.jetty.server.session.HouseKeeper startScavenging
INFO: node0 Scavenging every 600000ms
May 19, 2023 10:01:03 AM org.eclipse.jetty.server.handler.ContextHandler$Contextlog
INFO: INFO: jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.19+7-LTS on 11.0.19+7-LTS +jit [linux-x86_64]
May 19, 2023 10:01:03 AM org.eclipse.jetty.server.handler.ContextHandler$Contextlog
INFO: INFO: using a shared (threadsafe!) runtime
uri:classloader:/jruby/rack/response.rb:294: warning: constant ::Fixnum is deprecated
uri:classloader:/jruby/rack/core_ext.rb:26: warning: constant ::NativeExceptionis deprecated
Loading ArchivesSpace configuration file from path: /opt/archivesspace/config/config.rb
ArchivesSpaceThreadDump: Touch the file '/opt/archivesspace/thread_dump_backend.txt' to trigger a thread dump
I, [2023-05-19T10:01:14.847038 #1570] INFO -- : Thread-2002: Connecting to database: jdbc:mysql://localhost:3306/archivesspace?user=[REDACTED]&password=[REDACT ED]&useUnicode=true&characterEncoding=UTF-8. Max connections: 28
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver classis `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
I, [2023-05-19T10:01:15.866939 #1570] INFO -- : Thread-2002: All tables checkedand confirmed set to UTF-8. Nice job!
I, [2023-05-19T10:01:16.615214 #1570] INFO -- : Thread-2002: Solr config checksum verification ok.
I, [2023-05-19T10:01:19.772389 #1570] INFO -- : Thread-2002: Starting job scheduler
/opt/archivesspace/gems/gems/rufus-scheduler-2.0.24/lib/rufus/sc/cronline.rb:80:warning: constant ::Fixnum is deprecated
I, [2023-05-19T10:01:20.446880 #1570] INFO -- : Thread-2002: Updating system preferences
I, [2023-05-19T10:01:21.067956 #1570] INFO -- : Thread-2914: Starting background job thread 1 (2914)
I, [2023-05-19T10:01:21.074939 #1570] INFO -- : Thread-2916: Starting background job thread 2 (2916)
May 19, 2023 10:01:21 AM org.eclipse.jetty.server.handler.ContextHandler doStart
INFO: Started o.e.j.w.WebAppContext at 26f5e45d{/,file:///opt/archivesspace/data/tmp/jetty-0_0_0_0-8089-backend_war-_-any-5382710903715848903/webapp/,AVAILABLE}{/opt/archivesspace/wars/backend.war}<mailto:o.e.j.w.WebAppContext at 26f5e45d%7b/,file:///opt/archivesspace/data/tmp/jetty-0_0_0_0-8089-backend_war-_-any-5382710903715848903/webapp/,AVAILABLE%7d%7b/opt/archivesspace/wars/backend.war%7d>
========================================================================
A trace file has been written to the following location: /tmp/aspace_diagnostic_1684504881.txt
This file contains information that will assist developers in diagnosing
problems with your ArchivesSpace installation. Please review the file's
contents for sensitive information (such as passwords) that you might not
want to share.
========================================================================
Unhandled Java exception: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8089<http://0.0.0.0/0.0.0.0:8089>
java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8089<http://0.0.0.0/0.0.0.0:8089>
openAcceptChannel at org/eclipse/jetty/server/ServerConnector.java:349
open at org/eclipse/jetty/server/ServerConnector.java:310
doStart at org/eclipse/jetty/server/AbstractNetworkConnector.java:80
doStart at org/eclipse/jetty/server/ServerConnector.java:234
start at org/eclipse/jetty/util/component/AbstractLifeCycle.java:73
doStart at org/eclipse/jetty/server/Server.java:401
start at org/eclipse/jetty/util/component/AbstractLifeCycle.java:73
invoke0 at jdk/internal/reflect/NativeMethodAccessorImpl.java:-2
invoke at jdk/internal/reflect/NativeMethodAccessorImpl.java:62
invoke at jdk/internal/reflect/DelegatingMethodAccessorImpl.java:43
invoke at java/lang/reflect/Method.java:566
invokeDirectWithExceptionHandling at org/jruby/javasupport/JavaMethod.java:441
invokeDirect at org/jruby/javasupport/JavaMethod.java:305
invokeOther132:start at launcher/launcher.rb:90
start_server at launcher/launcher.rb:90
invokeOther193:start_server at launcher/launcher.rb:144
main at launcher/launcher.rb:144
invokeOther303:main at launcher/launcher.rb:242
<main> at launcher/launcher.rb:242
invokeWithArguments at java/lang/invoke/MethodHandle.java:710
runScript at org/jruby/Ruby.java:1205
runNormally at org/jruby/Ruby.java:1128
runNormally at org/jruby/Ruby.java:1146
runFromMain at org/jruby/Ruby.java:958
doRunFromMain at org/jruby/Main.java:400
internalRun at org/jruby/Main.java:292
run at org/jruby/Main.java:234
main at org/jruby/Main.java:206
Caused by:
java.net.BindException: Address already in use
bind0 at sun/nio/ch/Net.java:-2
bind at sun/nio/ch/Net.java:459
bind at sun/nio/ch/Net.java:448
bind at sun/nio/ch/ServerSocketChannelImpl.java:227
bind at sun/nio/ch/ServerSocketAdaptor.java:80
openAcceptChannel at org/eclipse/jetty/server/ServerConnector.java:344
open at org/eclipse/jetty/server/ServerConnector.java:310
doStart at org/eclipse/jetty/server/AbstractNetworkConnector.java:80
doStart at org/eclipse/jetty/server/ServerConnector.java:234
start at org/eclipse/jetty/util/component/AbstractLifeCycle.java:73
doStart at org/eclipse/jetty/server/Server.java:401
start at org/eclipse/jetty/util/component/AbstractLifeCycle.java:73
invoke0 at jdk/internal/reflect/NativeMethodAccessorImpl.java:-2
invoke at jdk/internal/reflect/NativeMethodAccessorImpl.java:62
invoke at jdk/internal/reflect/DelegatingMethodAccessorImpl.java:43
invoke at java/lang/reflect/Method.java:566
invokeDirectWithExceptionHandling at org/jruby/javasupport/JavaMethod.java:441
invokeDirect at org/jruby/javasupport/JavaMethod.java:305
call at org/jruby/java/invokers/InstanceMethodInvoker.java:32
cacheAndCall at org/jruby/runtime/callsite/CachingCallSite.java:355
call at org/jruby/runtime/callsite/CachingCallSite.java:144
invokeOther132:start at launcher/launcher.rb:90
RUBY$method$start_server$1 at launcher/launcher.rb:90
call at org/jruby/internal/runtime/methods/CompiledIRMethod.java:80
call at org/jruby/internal/runtime/methods/CompiledIRMethod.java:179
cacheAndCall at org/jruby/runtime/callsite/CachingCallSite.java:397
call at org/jruby/runtime/callsite/CachingCallSite.java:206
invokeOther193:start_server at launcher/launcher.rb:144
RUBY$method$main$6 at launcher/launcher.rb:144
call at org/jruby/internal/runtime/methods/CompiledIRMethod.java:156
cacheAndCall at org/jruby/runtime/callsite/CachingCallSite.java:355
call at org/jruby/runtime/callsite/CachingCallSite.java:144
invokeOther303:main at launcher/launcher.rb:242
RUBY$script at launcher/launcher.rb:242
invokeWithArguments at java/lang/invoke/MethodHandle.java:710
load at org/jruby/ir/Compiler.java:89
runScript at org/jruby/Ruby.java:1205
runNormally at org/jruby/Ruby.java:1128
runNormally at org/jruby/Ruby.java:1146
runFromMain at org/jruby/Ruby.java:958
doRunFromMain at org/jruby/Main.java:400
internalRun at org/jruby/Main.java:292
run at org/jruby/Main.java:234
main at org/jruby/Main.java:206"
I have Solr and ArchivesSpace set to start as init scripts. I would appreciate any insight that folks could offer would be great. Thanks!
[Towson University logo]<http://www.towson.edu/>
Eric T. Gadsby
Pronouns: he/him/his
IT Operations Specialist | Albert S. Cook Library
—
P: 410-704-3340
egadsby at towson.edu<mailto:egadsby at towson.edu> | libraries.towson.edu<http://www.towson.edu/https:/libraries.towson.edu>
—
Confidentiality Notice: This message may contain information that is confidential, privileged, proprietary, or otherwise legally exempt from disclosure. If you are not the intended recipient, you are notified that you are not authorized to read, print, copy or disseminate this message, any part of it, or any attachments. If this message has been sent to you in error, please notify the sender by replying to this transmission, or by calling Albert S. Cook Library at 410-704-3340 .
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
--
Tom Hanstra
Sr. Systems Administrator
hanstra at nd.edu<mailto:hanstra at nd.edu>
Error! Filename not specified.
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
--
Tom Hanstra
Sr. Systems Administrator
hanstra at nd.edu<mailto:hanstra at nd.edu>
[Image removed by sender.]
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and are confident the content is safe.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20230519/47058ec2/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 1826 bytes
Desc: image001.jpg
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20230519/47058ec2/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 1826 bytes
Desc: image002.jpg
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20230519/47058ec2/attachment-0001.jpg>
More information about the Archivesspace_Users_Group
mailing list