[Archivesspace_Users_Group] Trying to spin up a new server and it won't load... Unhandled Java exception: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8089 error

Blake Carver blake.carver at lyrasis.org
Fri May 19 11:26:11 EDT 2023


Is this set in your config?


AppConfig[:enable_solr] = false



________________________________
From: archivesspace_users_group-bounces at lyralists.lyrasis.org <archivesspace_users_group-bounces at lyralists.lyrasis.org> on behalf of Gadsby, Eric T. <egadsby at towson.edu>
Sent: Friday, May 19, 2023 11:22 AM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org>
Subject: Re: [Archivesspace_Users_Group] Trying to spin up a new server and it won't load... Unhandled Java exception: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8089 error


ps -ef  | grep java =



“java -server -Xms512m -Xmx512m -XX:+UseG1GC -XX:+PerfDisableSharedMem -XX:+ParallelRefProcEnabled -XX                                                             :MaxGCPauseMillis=250 -XX:+UseLargePages -XX:+AlwaysPreTouch -XX:+ExplicitGCInvokesConcurrent -Xlog:gc*:file=/var/solr/logs/solr_gc.log:time,uptime:filec                                                             ount=9,filesize=20M -Dsolr.jetty.inetaccess.includes= -Dsolr.jetty.inetaccess.excludes= -Dsolr.log.dir=/var/solr/logs -Djetty.port=8983 -DSTOP.PORT=7983                                                              -DSTOP.KEY=solrrocks -Duser.timezone=UTC -XX:-OmitStackTraceInFastThrow -XX:OnOutOfMemoryError=/opt/solr/bin/oom_solr.sh 8983 /var/solr/logs -Djetty.home                                                             =/opt/solr/server -Dsolr.solr.home=/var/solr/data -Dsolr.data.home= -Dsolr.install.dir=/opt/solr -Dsolr.default.confdir=/opt/solr/server/solr/configsets/                                                             _default/conf -Dlog4j.configurationFile=/var/solr/log4j2.xml -Xss256k -Dsolr.log.muteconsole -jar start.jar --module=http --module=gzip

egadsby+    6561    2546  0 11:18 pts/0    00:00:00 grep --color=auto java”



Right now it just seems to be Solr running…. Based on my read of this output.   I might be wrong.



It is a new 3.3.1 server with a fresh install with the data base from an old server being imported from a SQL dump and then migrated using the script.  Thanks!







[Towson University logo]<http://www.towson.edu/>

Eric T. Gadsby

Pronouns: he/him/his

IT Operations Specialist  |  Albert S. Cook Library

—

P: 410-704-3340
egadsby at towson.edu<mailto:egadsby at towson.edu>  |  libraries.towson.edu<http://www.towson.edu/https:/libraries.towson.edu>
 —



Confidentiality Notice: This message may contain information that is confidential, privileged, proprietary, or otherwise legally exempt from disclosure. If you are not the intended recipient, you are notified that you are not authorized to read, print, copy or disseminate this message, any part of it, or any attachments. If this message has been sent to you in error, please notify the sender by replying to this transmission, or by calling Albert S. Cook Library at 410-704-3340 .







From: archivesspace_users_group-bounces at lyralists.lyrasis.org <archivesspace_users_group-bounces at lyralists.lyrasis.org> on behalf of Tom Hanstra <hanstra at nd.edu>
Date: Friday, May 19, 2023 at 11:14 AM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org>
Subject: Re: [Archivesspace_Users_Group] Trying to spin up a new server and it won't load... Unhandled Java exception: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8089 error

[ CAUTION: This email is from outside of TU. Use caution before clicking links or opening attachments. If suspicious, report to phishing at towson.edu. ]

When connected to the server, use the command:

ps -ef  | grep java

and look to see if there are java processes running. If there is an old one, you will have to stop/kill it before starting the new one.

Is this a completely new server only running 3.3.1 ? If so, the solr process will also show up using that command.



Tom



On Fri, May 19, 2023 at 10:55 AM Gadsby, Eric T. <egadsby at towson.edu<mailto:egadsby at towson.edu>> wrote:

I wondered that, how would I check? Going to a browser yields nothing… that said my firewall rules may not be in place? Thanks!







[Towson University logo]<http://www.towson.edu/>

Eric T. Gadsby

Pronouns: he/him/his

IT Operations Specialist  |  Albert S. Cook Library

—

P: 410-704-3340
egadsby at towson.edu<mailto:egadsby at towson.edu>  |  libraries.towson.edu<http://www.towson.edu/https:/libraries.towson.edu>
 —



Confidentiality Notice: This message may contain information that is confidential, privileged, proprietary, or otherwise legally exempt from disclosure. If you are not the intended recipient, you are notified that you are not authorized to read, print, copy or disseminate this message, any part of it, or any attachments. If this message has been sent to you in error, please notify the sender by replying to this transmission, or by calling Albert S. Cook Library at 410-704-3340 .







From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> <archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>> on behalf of Tom Hanstra <hanstra at nd.edu<mailto:hanstra at nd.edu>>
Date: Friday, May 19, 2023 at 10:53 AM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>>
Subject: Re: [Archivesspace_Users_Group] Trying to spin up a new server and it won't load... Unhandled Java exception: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8089<http://0.0.0.0/0.0.0.0:8089> error

[ CAUTION: This email is from outside of TU. Use caution before clicking links or opening attachments. If suspicious, report to phishing at towson.edu<mailto:phishing at towson.edu>. ]

First thought would be that the new service is trying to start while the old one is still running. Could that be?



Tom



On Fri, May 19, 2023 at 10:44 AM Gadsby, Eric T. <egadsby at towson.edu<mailto:egadsby at towson.edu>> wrote:

Dear Friends,



I am in the process of moving our old v2.7.1 to V3.3.1. I have been able to set up Solr, the application, move the data and start ArchivesSpace twice in our testing environment (CentOS8 Stream). Today I ran the same procedure in our production environment (RHEL8.5) and ArchivesSpace won't start.



The log reports this error: "Unhandled Java exception: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8089<http://0.0.0.0/0.0.0.0:8089>" -- of at least I think that's the problem (I'm going to provide the whole log:



"$ cat archivesspace.out

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.

Loading ArchivesSpace configuration file from path: /opt/archivesspace/config/config.rb

Loading ArchivesSpace configuration file from path: /opt/archivesspace/config/config.rb

Loading ArchivesSpace configuration file from path: /opt/archivesspace/config/config.rb

May 19, 2023 10:01:02 AM org.eclipse.jetty.util.log.Log initialized

INFO: Logging initialized @7281ms to org.eclipse.jetty.util.log.Slf4jLog

May 19, 2023 10:01:03 AM org.eclipse.jetty.server.Server doStart

INFO: jetty-9.4.44.v20210927; built: 2021-09-27T23:02:44.612Z; git: 8da83308eeca865e495e53ef315a249d63ba9332; jvm 11.0.19+7-LTS

May 19, 2023 10:01:03 AM org.eclipse.jetty.server.session.DefaultSessionIdManager doStart

INFO: DefaultSessionIdManager workerName=node0

May 19, 2023 10:01:03 AM org.eclipse.jetty.server.session.DefaultSessionIdManager doStart

INFO: No SessionScavenger set, using defaults

May 19, 2023 10:01:03 AM org.eclipse.jetty.server.session.HouseKeeper startScavenging

INFO: node0 Scavenging every 600000ms

May 19, 2023 10:01:03 AM org.eclipse.jetty.server.handler.ContextHandler$Contextlog

INFO: INFO: jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.19+7-LTS on 11.0.19+7-LTS +jit [linux-x86_64]

May 19, 2023 10:01:03 AM org.eclipse.jetty.server.handler.ContextHandler$Contextlog

INFO: INFO: using a shared (threadsafe!) runtime

uri:classloader:/jruby/rack/response.rb:294: warning: constant ::Fixnum is deprecated

uri:classloader:/jruby/rack/core_ext.rb:26: warning: constant ::NativeExceptionis deprecated

Loading ArchivesSpace configuration file from path: /opt/archivesspace/config/config.rb



ArchivesSpaceThreadDump: Touch the file '/opt/archivesspace/thread_dump_backend.txt' to trigger a thread dump

I, [2023-05-19T10:01:14.847038 #1570]  INFO -- : Thread-2002: Connecting to database: jdbc:mysql://localhost:3306/archivesspace?user=[REDACTED]&password=[REDACT                                       ED]&useUnicode=true&characterEncoding=UTF-8. Max connections: 28

Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver classis `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.

I, [2023-05-19T10:01:15.866939 #1570]  INFO -- : Thread-2002: All tables checkedand confirmed set to UTF-8.  Nice job!

I, [2023-05-19T10:01:16.615214 #1570]  INFO -- : Thread-2002: Solr config checksum verification ok.

I, [2023-05-19T10:01:19.772389 #1570]  INFO -- : Thread-2002: Starting job scheduler

/opt/archivesspace/gems/gems/rufus-scheduler-2.0.24/lib/rufus/sc/cronline.rb:80:warning: constant ::Fixnum is deprecated

I, [2023-05-19T10:01:20.446880 #1570]  INFO -- : Thread-2002: Updating system preferences

I, [2023-05-19T10:01:21.067956 #1570]  INFO -- : Thread-2914: Starting background job thread 1 (2914)

I, [2023-05-19T10:01:21.074939 #1570]  INFO -- : Thread-2916: Starting background job thread 2 (2916)

May 19, 2023 10:01:21 AM org.eclipse.jetty.server.handler.ContextHandler doStart

INFO: Started o.e.j.w.WebAppContext at 26f5e45d{/,file:///opt/archivesspace/data/tmp/jetty-0_0_0_0-8089-backend_war-_-any-5382710903715848903/webapp/,AVAILABLE}{/opt/archivesspace/wars/backend.war}<mailto:o.e.j.w.WebAppContext at 26f5e45d%7b/,file:///opt/archivesspace/data/tmp/jetty-0_0_0_0-8089-backend_war-_-any-5382710903715848903/webapp/,AVAILABLE%7d%7b/opt/archivesspace/wars/backend.war%7d>

      ========================================================================

      A trace file has been written to the following location: /tmp/aspace_diagnostic_1684504881.txt



      This file contains information that will assist developers in diagnosing

      problems with your ArchivesSpace installation.  Please review the file's

      contents for sensitive information (such as passwords) that you might not

      want to share.

      ========================================================================

Unhandled Java exception: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8089<http://0.0.0.0/0.0.0.0:8089>

java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8089<http://0.0.0.0/0.0.0.0:8089>

                  openAcceptChannel at org/eclipse/jetty/server/ServerConnector.java:349

                               open at org/eclipse/jetty/server/ServerConnector.java:310

                            doStart at org/eclipse/jetty/server/AbstractNetworkConnector.java:80

                            doStart at org/eclipse/jetty/server/ServerConnector.java:234

                              start at org/eclipse/jetty/util/component/AbstractLifeCycle.java:73

                            doStart at org/eclipse/jetty/server/Server.java:401

                              start at org/eclipse/jetty/util/component/AbstractLifeCycle.java:73

                            invoke0 at jdk/internal/reflect/NativeMethodAccessorImpl.java:-2

                             invoke at jdk/internal/reflect/NativeMethodAccessorImpl.java:62

                             invoke at jdk/internal/reflect/DelegatingMethodAccessorImpl.java:43

                             invoke at java/lang/reflect/Method.java:566

  invokeDirectWithExceptionHandling at org/jruby/javasupport/JavaMethod.java:441

                       invokeDirect at org/jruby/javasupport/JavaMethod.java:305

               invokeOther132:start at launcher/launcher.rb:90

                       start_server at launcher/launcher.rb:90

        invokeOther193:start_server at launcher/launcher.rb:144

                               main at launcher/launcher.rb:144

                invokeOther303:main at launcher/launcher.rb:242

                             <main> at launcher/launcher.rb:242

                invokeWithArguments at java/lang/invoke/MethodHandle.java:710

                          runScript at org/jruby/Ruby.java:1205

                        runNormally at org/jruby/Ruby.java:1128

                        runNormally at org/jruby/Ruby.java:1146

                        runFromMain at org/jruby/Ruby.java:958

                      doRunFromMain at org/jruby/Main.java:400

                        internalRun at org/jruby/Main.java:292

                                run at org/jruby/Main.java:234

                               main at org/jruby/Main.java:206



Caused by:

java.net.BindException: Address already in use

                              bind0 at sun/nio/ch/Net.java:-2

                               bind at sun/nio/ch/Net.java:459

                               bind at sun/nio/ch/Net.java:448

                               bind at sun/nio/ch/ServerSocketChannelImpl.java:227

                               bind at sun/nio/ch/ServerSocketAdaptor.java:80

                  openAcceptChannel at org/eclipse/jetty/server/ServerConnector.java:344

                               open at org/eclipse/jetty/server/ServerConnector.java:310

                            doStart at org/eclipse/jetty/server/AbstractNetworkConnector.java:80

                            doStart at org/eclipse/jetty/server/ServerConnector.java:234

                              start at org/eclipse/jetty/util/component/AbstractLifeCycle.java:73

                            doStart at org/eclipse/jetty/server/Server.java:401

                              start at org/eclipse/jetty/util/component/AbstractLifeCycle.java:73

                            invoke0 at jdk/internal/reflect/NativeMethodAccessorImpl.java:-2

                             invoke at jdk/internal/reflect/NativeMethodAccessorImpl.java:62

                             invoke at jdk/internal/reflect/DelegatingMethodAccessorImpl.java:43

                             invoke at java/lang/reflect/Method.java:566

  invokeDirectWithExceptionHandling at org/jruby/javasupport/JavaMethod.java:441

                       invokeDirect at org/jruby/javasupport/JavaMethod.java:305

                               call at org/jruby/java/invokers/InstanceMethodInvoker.java:32

                       cacheAndCall at org/jruby/runtime/callsite/CachingCallSite.java:355

                               call at org/jruby/runtime/callsite/CachingCallSite.java:144

               invokeOther132:start at launcher/launcher.rb:90

         RUBY$method$start_server$1 at launcher/launcher.rb:90

                               call at org/jruby/internal/runtime/methods/CompiledIRMethod.java:80

                               call at org/jruby/internal/runtime/methods/CompiledIRMethod.java:179

                       cacheAndCall at org/jruby/runtime/callsite/CachingCallSite.java:397

                               call at org/jruby/runtime/callsite/CachingCallSite.java:206

        invokeOther193:start_server at launcher/launcher.rb:144

                 RUBY$method$main$6 at launcher/launcher.rb:144

                               call at org/jruby/internal/runtime/methods/CompiledIRMethod.java:156

                       cacheAndCall at org/jruby/runtime/callsite/CachingCallSite.java:355

                               call at org/jruby/runtime/callsite/CachingCallSite.java:144

                invokeOther303:main at launcher/launcher.rb:242

                        RUBY$script at launcher/launcher.rb:242

                invokeWithArguments at java/lang/invoke/MethodHandle.java:710

                               load at org/jruby/ir/Compiler.java:89

                          runScript at org/jruby/Ruby.java:1205

                        runNormally at org/jruby/Ruby.java:1128

                        runNormally at org/jruby/Ruby.java:1146

                        runFromMain at org/jruby/Ruby.java:958

                      doRunFromMain at org/jruby/Main.java:400

                        internalRun at org/jruby/Main.java:292

                                run at org/jruby/Main.java:234

                               main at org/jruby/Main.java:206"



I have Solr and ArchivesSpace set to start as init scripts. I would appreciate any insight that folks could offer would be great. Thanks!





[Towson University logo]<http://www.towson.edu/>

Eric T. Gadsby

Pronouns: he/him/his

IT Operations Specialist  |  Albert S. Cook Library

—

P: 410-704-3340
egadsby at towson.edu<mailto:egadsby at towson.edu>  |  libraries.towson.edu<http://www.towson.edu/https:/libraries.towson.edu>
 —



Confidentiality Notice: This message may contain information that is confidential, privileged, proprietary, or otherwise legally exempt from disclosure. If you are not the intended recipient, you are notified that you are not authorized to read, print, copy or disseminate this message, any part of it, or any attachments. If this message has been sent to you in error, please notify the sender by replying to this transmission, or by calling Albert S. Cook Library at 410-704-3340 .





_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group




--

Tom Hanstra

Sr. Systems Administrator

hanstra at nd.edu<mailto:hanstra at nd.edu>



Error! Filename not specified.

_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group at lyralists.lyrasis.org<mailto:Archivesspace_Users_Group at lyralists.lyrasis.org>
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group




--

Tom Hanstra

Sr. Systems Administrator

hanstra at nd.edu<mailto:hanstra at nd.edu>



[Image removed by sender.]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20230519/16dc72df/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 1826 bytes
Desc: image001.jpg
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20230519/16dc72df/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 1826 bytes
Desc: image001.jpg
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20230519/16dc72df/attachment-0001.jpg>


More information about the Archivesspace_Users_Group mailing list