Why Can't I connect to heavy forwarder GUI?

[jmrubio]

I am getting a 500 internal server error when I try to connect to the HF GUI. I ran firewall-cmd --list-ports, and it shows 8000/tcp. I also checked web.conf, and it shows enableSplunkWebSSL = 1, as well as httport = 8000. What else can I check? I appreciate the help in advance!

[isoutamo]

Hi

when you got error code 500 you obviously got connection to splunkd's http part, but for some reason it didn't work correctly. How you are try to connect and from where?

Did this work on HF host?

curl -vk https://localhost:8000

And how about when you switch localhost to your host real name and/or IP?

Are there anything on splunk's internal logs under /opt/splunk/var/log/splunk (access + splunkd) logs?

r. Ismo 

[jmrubio]

Hello @isoutamo ,

Sorry for the late response. So in the splunkd.logs all of the messages are - WARN TcpOutputProc [16779 indexerPipe] - The TCP output processor has paused the data flow. Forwarding to host_dest=<IP> inside output group default-autolb-group from host_src=Splunk_Heavy_Forwarder has been blocked for blocked_seconds=355350. This can stall the data flow towards indexing and other network outputs. Review the receiving system's health in the Splunk Monitoring Console. It is probably not accepting data. 

And the last message in the splunkd_access.log is from June  : (

I am trying to connect to the HF through the web, and the cURL command returned a 303 error "The resource has moved temporarily"

[PickleRick]

If there are no new entries in your access log it could signal storage problems. Did you check your free disk space?

[jmrubio]

Hello @gcusello,

Firewalld is running, and I do not see anything disabling web interface in server.conf. The "trustedIP" is commented out, but I do not know if that matters.

[gcusello]

Hi @jmrubio,

if firewalld is running this could be the issue.

Try to disable it (or permit traffic on port 8000) and check if you can access web interface.

Ciao.

Giuseppe

[jmrubio]

Hello @gcusello,

I tried ss -na | grep 8000 and it returned:

tcp  LISTEN  0  128  *:8000  *:*

[gcusello]

Hi @jmrubio,

did you disabled firewalld?

systemctl stop firewalld
systemctl disable firewalld

Ciao.

Giuseppe

[jmrubio]

Is that the only way to get the data in? This is a production server and I don't think I will be able to disable/stop firewalld.

[gcusello]

Hi @jmrubio ,

did you disabled local firewall on this server?

check if you disabled web interface in server.conf.

Ciao.

Giuseppe