Why am I getting the following connection error with DB Connect App: "Cannot communicate with task server, please check your settings issue."

[mpragav143]

I have installed the DB connect app on my instance of Splunk. I am facing an error message that reads "Cannot communicate with task server, please check your settings issue."

I found the below error in the logs.

ERROR ExecProcessor - message from

"/home/iapsp02/etc/apps/splunk_app_db_connect/linux_x86_64/bin/server.sh" com.splunk.modularinput.Event.writeTo(Event.java:65)\\com.splunk.modularinput.EventWriter.writeEvent(EventWriter.java:134)\\com.splunk.dbx.server.bootstrap.TaskServerStart.streamEvents(TaskServerStart.java:77)\\com.splunk.modularinput.Script.run(Script.java:66)\\com.splunk.modularinput.Script.run(Script.java:44)\\com.splunk.dbx.server.bootstrap.TaskServerStart.main(TaskServerStart.java:150)\\

And here is another error:

ERROR ExecProcessor - message from 

    "/home/iapsp02/etc/apps/splunk_app_db_connect/linux_x86_64/bin/server.sh" 03:08:51.576 [main] INFO com.splunk.dbx.utils.TrustManagerUtil - action=load_key_manager_succeed

[ssadh_splunk]

So i ran into the same issue & managed to find a way around.

Upon manually executing the script outlined in the log;
$SPLUNK_HOME/splunk_app_db_connect/linux_x86_64/bin/server.sh

got below issues:
./server.sh: line 33: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64/jre/bin/java/bin/java: Not a directory
./server.sh: line 33: exec: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64/jre/bin/java/bin/java: cannot execute: Not a directory

Seems like the JAVA_HOME variable was being set incorrectly & had an extra

/bin/java

at the end.
Tweaked JAVA_HOME variable in bash_profile & restarted Splunk. DB connect works fine now.

.
Hence tweaked the variable so that

[bcyates]

Usually this is just an issue with your TaskServer port. Make sure whatever port you entered for TaskServer port when configuring DBconnect is open and available.

[kheo_splunk]

There is a few steps to check whether taskserver functions properly.

1] Check whether java program is running using "ps -ef | grep java" command
Example output

splunk    8371  8123  0 01:11 ?        00:05:20 /opt/splunk/jdk1.8.0_181/bin/java -Ddw.server.applicationConnectors[0].port=9998  -jar /opt/splunk/HF2/etc/apps/splunk_app_db_connect/linux_x86_64/bin/../../jars/server.jar

2] If java process with server.jar is not up and running, check whether java path is configured properly in $SPLUNK_HOME/splunk_app_db_connect/your_platform/bin/customized.java.path and then correct java path in your system if it points to the wrong path.

Example of customized.java.path in /opt/splunk/HF2/etc/apps/splunk_app_db_connect/linux_x86_64/bin

 /opt/splunk/jdk1.8.0_181/bin/java

3] If Java process is running with server.jar, check the connectivity to task server using CURL command after login to the Splunk server where DB Connect app is installed
Example

curl -k -u admin:password https://localhost:<mgmt_port>/servicesNS/nobody/splunk_app_db_connect/db_connect/dbxproxy/taskserver

curl -k http://localhost:9998/api/taskserver

curl --noproxy localhost -k http://localhost:9998/api/taskserver

Example output

{"status":"**RUNNING**","schedulerStatus":0,"message":"task server is running","info":{"ProcessName":"8371@kheolin05.sv.splunk.com","Uptime":"57931358","javaVMName":"Java HotSpot(TM) Server VM","javaVersion":"1.8.0_181","TotalMemory":"357302272","javaVendor":"Oracle Corporation","javaSpecificationVersion":"1.8","MaxMemory":"954466304","FreeMemory":"92405888"},"serverId":"16df13b1-549c-4d42-9a79-1e0ae5d163fd"}[/opt/splunk/HF2/etc/apps/splunk_app_db_connect/linux_x86_64/bin:HF2]$

** If curl command only works with "--noproxy localhost" option, check whether http_proxy is configured in your system environment variable and then configure no_proxy environment variable for localhost and restart Splunk instance
Example

no_proxy=localhost,127.0.0.1 

** Here tcp port# 9998 is default port for taskserver, port for taskserver can be found in step#1 above in port=9998

[harsmarvania57]

Hi @mpragav143,

I am getting same error with Splunk 7.1.2 and DB Connect 3.1.3. After getting this error you are not able to setup new connections ? What problem are you facing ?

[mpragav143]

When i open db connect app, i could see this error.
"Cannot communicate with task server, please check your settings" , and one more error is "Bad gateway"

[richgalloway]

Do you have the right JDBC driver installed?

[mpragav143]

yes java 1.8 installed.

[richgalloway]

I asked about the JDBC driver, NOT Java. They're different things, installed differently. See http://docs.splunk.com/Documentation/DBX/3.1.3/DeployDBX/Installdatabasedrivers

[harsmarvania57]

You need to make sure that you have oracle JRE or JDK. Additionally task server runs on port 9998 by default if this port is occupied by other process then task server will not start in that case you need to use other port for task server.

[richgalloway]

DB Connect wants the JRE, not the JDK.

[renjith_nair]

@mpragav143,

It looks like a connection issue from your splunk server to DB server

• Check your central firewall settings from the splunk server to the DB server.
• Also check the local iptables/firewalld settings on your machine.
• Make sure java is installed

[mpragav143]

App deployed on same server. iptables/firewalld service already stopped on server. Java 1.8 is also installed.