The Java Bridge server is not running DBX 1.10

[davi1046]

Hello,

The java bridge is not re-starting after PC reboot. Tried several install / re-install. Each time the java bridge runs ok until the PC is rebooted.

tail for dbx.log:
2013-10-10 08:54:02.489 main:INFO:Splunkd - Initialized Splunkd REST service service=127.0.0.1:8089 sessionKey=
2013-10-10 08:54:04.723 main:INFO:Splunkd - Initialized Splunkd REST service service=127.0.0.1:8089 sessionKey=bc1a0cf1d00918af41f3923c2ca249f3
2013-10-10 08:54:09.218 main:INFO:Splunkd - Initialized Splunkd REST service service=127.0.0.1:8089 sessionKey=bc1a0cf1d00918af41f3923c2ca249f3

tail for jbridge.log:
2013-10-10 08:54:41,239 INFO Checking for obsolete java libraries in C:\Program Files\Splunk\etc\apps\dbx\bin\lib
2013-10-10 08:54:41,242 INFO Started JavaBridgeServer PID=4288
2013-10-10 08:54:45,650 INFO Checking for obsolete java libraries in C:\Program Files\Splunk\etc\apps\dbx\bin\lib
2013-10-10 08:54:45,651 INFO Started JavaBridgeServer PID=4048
2013-10-10 08:54:50,346 INFO Checking for obsolete java libraries in C:\Program Files\Splunk\etc\apps\dbx\bin\lib
2013-10-10 08:54:50,348 INFO Started JavaBridgeServer PID=3556

tail from splunkd.log:
10-10-2013 08:55:01.210 +0200 ERROR ExecProcessor - message from "python "C:\Program Files\Splunk\etc\apps\dbx\bin\jbridge_server.py"" No handlers could be found for logger "spp.java"

Any idea ?

splunk 5 & 6, dbx connect 1.1 & 1.09, win7

[letienne]

I applied the proposed fix (modification of spp/init.py), and now it starts successfully.

However, my jbridge.log file grows and grows and grows...

It seems that it contains pretty much everything splunk logs.

Is there any way to control its growth?

Thanks !

[letienne]

Issue is gone with the latest release. The release notes say it was a bug.

[benvidulichxero]

Did you find a solution to control the growth rate of jbridge.log?

Do you have the log level set to DEBUG or something?

[btsay_splunk · Accepted solution]

The workaround for this logger issue is to setup a root logger: http://answers.splunk.com/answers/109920/splunk6-db-connect-111-no-handlers-could-be-found-for-logge...

Let's add a root logger to "spp", so all "spp.*" will work. In bin/spp/java/init.py, add the following lines:

logging.basicConfig(level=logging.DEBUG,
     filename=os.path.join(os.environ["SPLUNK_HOME"], "var", "log", "splunk", "jbridge.log"), format="%(asctime)s %(levelname)s %(message)s", filemode="a")

before the line:

logger=logging.getLogger("spp.java")

Then, restart splunk.

Note: the better place for the spp root logger is in bin/spp/_init_.py, which covers all loggers in bin/spp and below packages.

[splunkIT]

DB Connect 1.1.1 or later should have addressed this problem. I have also seen this issue with java 1.6. You should also upgrade your java to the latest 1.7 version:

http://www.oracle.com/technetwork/java/javase/downloads/index.html

Better yet, consider using the java sdk package, because Support may request for jstack outputs if in-depth troubleshooting is required:

http://answers.splunk.com/answers/102822/how-to-generate-java-jstack-of-the-db-connect-jbridge-proce...

[davi1046]

Thanks, I'll try this new release.

[araitz]

EDIT:

There is another issue, DBX-122, which we fixed but caused a regression in the root logger. See btsay's answer above for the correct workaround.

ORIGINAL:

This is a known issue, DBX-269. Here is the proposed work around:

In the file:

bin/spp/java/__init__.py

Change this line:

logger = logging.getLogger("spp.java")

To this:

try:
    logger = logging.getLogger("spp.java")
except:
    logger = logging.getLogger()

Then, restart Splunk.

[timrcase]

I tried to apply this work around, but I'm continuing to get the same error. Has anyone else experienced this?

[davi1046]

Thanks everyone !

[araitz]

http://docs.splunk.com/Documentation/DBX/1.1/DeployDBX/Releasenotes#Known_issue

[lukejadamec]

So far so good. Thanks!
By the way, do you know where to find the list of dbx known issues?

[lukejadamec]

Is the jbridge server running now?

When I see this error the jbridge server is running, but dbx queries never complete.

[davi1046]

Hello, I found an offending file with no owner (??) nor write access.

Getting a new error:
2013-10-10 17:07:48,684 INFO Started JavaBridgeServer PID=2288
...
2013-10-10 17:07:50,908 ERROR Java process returned error code 1! Error: Initializing Splunk context... Environment: SplunkEnvironment{SPLUNK_HOME...} Configuring Log4j... Exception in thread "main" com.splunk.config.SplunkConfigurationException: IO Error while reading configuration from Splunkd: java.io.IOException: Server returned HTTP response code: 401 for URL: https://127.0.0.1:8089/servicesNS/nobody/-/admin/conf-java?count=10000

[lukejadamec]

If you're still getting logs then you might have some luck with debug. To turn on jbridge and dbx debugging do this:
etc/apps/dbx/local/java.conf
[logging]
level = DEBUG
And restart splunkd

[davi1046]

Thanks Lukejadamec, I tried your suggestions but no luck I still have the issue

[lukejadamec]

Are you also seeing the java process constantly restarting, and the dbx log has nothing, at all. I saw this for weeks.

At first opening the dbx settings and saving them fixed it. But after a while that did not work either.
My most recent fix was to downgrade dbx to 1.09, and then upgrade Splunk to 5.0.5.
My server is a 2K8R2 64bit hardened system.