Splunk HEC token is not working

[uagraw01]

As per the below screenshot my server is not giving any health status of hec port 8088. Due to this I am not able to publish anything by using hec token in Splunk for an example :

curl -k "Authorization: Splunk ee6d8a90-4863-4789-9ff1-fda810bee6f2" http://walvau-vidi-1:8000/services/collector/event -d '{"event": "hello world"}'.

Please guide me what will issue, how I investigate further on this.

default inputs.conf :

[http]
disabled=1
port=8088
enableSSL=1
dedicatedIoThreads=2
maxThreads = 0
maxSockets = 0
useDeploymentServer=0
# ssl settings are similar to mgmt server
sslVersions=*,-ssl2
allowSslCompression=true
allowSslRenegotiation=true
ackIdleCleanup=true


local inputs.conf:

[http]
disabled = 0
enableSSL = 0

[PickleRick]

The usual debugging steps apply:

1) Check if the receiving side is listening on the port (use netstat to list open ports and verify if 8088 is among them).

2) Check the network connectivity from the client

3) Verify firewall rules

4) If needed, run tcpdump/wireshark on the server and see if any traffic from the client is reaching the server at all.

When you can connect to your HEC service port you can start debugging the token settings.

[uagraw01]

@PickleRick @marnall After further investigation I found that the tcp port 8088 is being used under another app . I removed the config from there and now all are working fine.

Issued screenshot:

Resolved screenshot:

Thanks both of your support and suggestions.

[marnall]

I would not recommend posting valid authorization tokens on the internet, as unscrupulous people or bots could abuse them.

Could you try curl-ing the collector health endpoint using HTTPS instead of http?

If it still does not give a response, it might be a firewall issue. Try connecting to the machine itself using ssh and then doing a curl on localhost, like this:

curl -k https://127.0.0.1:8088/services/collector/health

[uagraw01]

@marnall For your information I already tried with https before posting this to Splunk answers and for your information on windows server is using telnet instead to SSH. 

Can you please help me to understand the significance why you suggested https ? Because on other server posted command is working fine with “http”

Please provide your more suggestion on this.

[marnall]

Depending on how your server is configured, it may reject http connections. Are you able to connect to the collector health endpoint on 127.0.0.1 by connecting to the server via telnet and sending the request to localhost?

[uagraw01]

@marnall I have opened inbound port also 8088 also so I think firewall related issue also not be the concern now.