I recently upgraded Java JRE to version 7 update 51 on the server running Splunk (Windows Server 2008 R2, SP1). After doing so, the Java Bridge stopped running. There don't appear to be any errors in either dbx.log or jbridge.log; however, what is in those two logs seems a little odd (see below).
These are my settings:
Java home: C:\Program Files (x86)\Java\jre7
JVM Commandline options: -Xmx256m -Dfile.encoding=UTF-8 -Duser.language=en -Duser.region=
I seem to recall the last time we upgraded Java JRE, we had to make some change to the commandline options, but I don't remember what that change was.
This is what's in the dbx.log (repeats roughly every 4.5 seconds):
2014-02-26 09:54:38.251 main:INFO:Splunkd - Initialized Splunkd REST service service=127.0.0.1:8089 sessionKey=0fc57988c8061f0326499d14eff6d710
This is what's in the jbridge.log (repeats roughly every 4.5 seconds with a different PID every time):
2014-02-26 09:54:45,332 INFO Checking for obsolete java libraries in E:\Splunk\etc\apps\dbx\bin\lib
2014-02-26 09:54:45,332 INFO Started JavaBridgeServer PID=2808
Has anyone else run into this? If so, what did you do?
EDIT: additional log output after upgrading DB Connect to 1.1.3 and implementing workaround from http://answers.splunk.com/answers/105605/the-java-bridge-server-is-not-running-dbx-110
2014-02-27 08:43:11,065 ERROR Java process returned error code 1! Error: Initializing Splunk context... Environment: SplunkEnvironment{SPLUNK_HOME=E:\Splunk,SPLUNK_DB=E:\Splunk\var\lib\splunk} Configuring Log4j... [Fatal Error] :1:1: Premature end of file. Exception in thread "main" com.splunk.config.SplunkConfigurationException: Error creating PersistentValueStore type xstream: com.thoughtworks.xstream.io.StreamException: : Premature end of file. at com.splunk.persistence.PersistentValueStoreFactory.createStoreInstance(PersistentValueStoreFactory.java:119) at com.splunk.persistence.PersistentValueStoreFactory.createStore(PersistentValueStoreFactory.java:71) at com.splunk.persistence.PersistentValueStoreFactory.createGlobalStore(PersistentValueStoreFactory.java:51) at com.splunk.env.SplunkContext.initialize(SplunkContext.java:108) at com.splunk.bridge.JavaBridgeServer.main(JavaBridgeServer.java:34) Caused by: com.thoughtworks.xstream.io.StreamException: : Premature end of file. at com.thoughtworks.xstream.io.xml.DomDriver.createReader(DomDriver.java:105) at com.thoughtworks.xstream.io.xml.DomDriver.createReader(DomDriver.java:81) at com.thoughtworks.xstream.XStream.fromXML(XStream.java:904) at com.splunk.persistence.impl.XStreamStore.loadState(XStreamStore.java:113) at com.splunk.persistence.impl.XStreamStore.<init>(XStreamStore.java:49) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) at java.lang.reflect.Constructor.newInstance(Unknown Source) at com.splunk.util.Utils$Reflection.instantiate(Utils.java:880) at com.splunk.persistence.PersistentValueStoreFactory.createStoreInstance(PersistentValueStoreFactory.java:117) ... 4 more Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Premature end of file. at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(Unknown Source) at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(Unknown Source) at com.thoughtworks.xstream.io.xml.DomDriver.createReader(DomDriver.java:98) ... 14 more
2014-02-27 08:43:11,065 ERROR Command output: None
EDIT: NEW log output after uninstalling/reinstalling DB Connect:
2014-03-04 08:51:11,897 ERROR Java process returned error code 1! Error: Initializing Splunk context... Environment: SplunkEnvironment{SPLUNK_HOME=E:\Splunk,SPLUNK_DB=E:\Splunk\var\lib\splunk} Configuring Log4j... Exception in thread "main" com.splunk.config.SplunkConfigurationException: Error creating PersistentValueStore type xstream: com.splunk.persistence.PersistentValueStoreException: File not found while trying to load persistent values from XML file at com.splunk.persistence.PersistentValueStoreFactory.createStoreInstance(PersistentValueStoreFactory.java:119) at com.splunk.persistence.PersistentValueStoreFactory.createStore(PersistentValueStoreFactory.java:71) at com.splunk.persistence.PersistentValueStoreFactory.createGlobalStore(PersistentValueStoreFactory.java:51) at com.splunk.env.SplunkContext.initialize(SplunkContext.java:108) at com.splunk.bridge.JavaBridgeServer.main(JavaBridgeServer.java:34) Caused by: com.splunk.persistence.PersistentValueStoreException: File not found while trying to load persistent values from XML file at com.splunk.persistence.impl.XStreamStore.loadState(XStreamStore.java:115) at com.splunk.persistence.impl.XStreamStore.<init>(XStreamStore.java:49) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) at java.lang.reflect.Constructor.newInstance(Unknown Source) at com.splunk.util.Utils$Reflection.instantiate(Utils.java:880) at com.splunk.persistence.PersistentValueStoreFactory.createStoreInstance(PersistentValueStoreFactory.java:117) ... 4 more Caused by: java.io.FileNotFoundException: E:\Splunk\var\lib\splunk\persistentstorage\dbx\global\state.xml (Access is denied) at java.io.FileInputStream.open(Native Method) at java.io.FileInputStream.<init>(Unknown Source) at com.splunk.persistence.impl.XStreamStore.loadState(XStreamStore.java:113) ... 11 more
2014-03-04 08:51:11,897 ERROR Command output: None
2014-03-04 08:51:13,644 INFO 10.110.68.254 - admin [04/Mar/2014:08:51:13.628 -0600] "GET /en-US/api/messages/index HTTP/1.1" 200 341 "http://iprdq-analytics.iproduction.com/en-US/manager/sourcemedia/data/ui/nav/default?uri=%2FservicesNS%2Fnobody%2Fsourcemedia%2Fdata%2Fui%2Fnav%2Fdefault&f_ns=sourcemedia&ns=sourcemedia&action=edit" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" - 5315e861a0416beb8 16ms
2014-03-04 08:51:14,923 INFO Checking for obsolete java libraries in E:\Splunk\etc\apps\dbx\bin\lib
2014-03-04 08:51:14,923 INFO Started JavaBridgeServer PID=1936
... View more