Deployment Architecture

Why Can't I connect to heavy forwarder GUI?

jmrubio
Explorer

I am getting a 500 internal server error when I try to connect to the HF GUI. I ran firewall-cmd --list-ports, and it shows 8000/tcp. I also checked web.conf, and it shows enableSplunkWebSSL = 1, as well as httport = 8000. What else can I check? I appreciate the help in advance!

Labels (1)
Tags (1)
0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

when you got error code 500 you obviously got connection to splunkd's http part, but for some reason it didn't work correctly. How you are try to connect and from where?

Did this work on HF host?

curl -vk https://localhost:8000

And how about when you switch localhost to your host real name and/or IP?

Are there anything on splunk's internal logs under /opt/splunk/var/log/splunk (access + splunkd) logs?

r. Ismo 

jmrubio
Explorer

Hello @isoutamo ,

Sorry for the late response. So in the splunkd.logs all of the messages are - WARN TcpOutputProc [16779 indexerPipe] - The TCP output processor has paused the data flow. Forwarding to host_dest=<IP> inside output group default-autolb-group from host_src=Splunk_Heavy_Forwarder has been blocked for blocked_seconds=355350. This can stall the data flow towards indexing and other network outputs. Review the receiving system's health in the Splunk Monitoring Console. It is probably not accepting data. 

And the last message in the splunkd_access.log is from June  : (

I am trying to connect to the HF through the web, and the cURL command returned a 303 error "The resource has moved temporarily"

0 Karma

PickleRick
SplunkTrust
SplunkTrust

If there are no new entries in your access log it could signal storage problems. Did you check your free disk space?

0 Karma

jmrubio
Explorer

Hello @gcusello,

Firewalld is running, and I do not see anything disabling web interface in server.conf. The "trustedIP" is commented out, but I do not know if that matters.

 

 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @jmrubio,

if firewalld is running this could be the issue.

Try to disable it (or permit traffic on port 8000) and check if you can access web interface.

Ciao.

Giuseppe

0 Karma

jmrubio
Explorer

Hello @gcusello,

I tried ss -na | grep 8000 and it returned:

tcp  LISTEN  0  128  *:8000  *:*

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @jmrubio,

did you disabled firewalld?

systemctl stop firewalld
systemctl disable firewalld

Ciao.

Giuseppe

0 Karma

jmrubio
Explorer

Is that the only way to get the data in? This is a production server and I don't think I will be able to disable/stop firewalld.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @jmrubio ,

did you disabled local firewall on this server?

check if you disabled web interface in server.conf.

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...