Splunk HTTP Event Collector works only on localhost.

[syedimranstonex]

I have set up a Splunk Enterprise trial instance on a red-hat Linux server.

I enabled and setup the HEC, however when I try the curl command to check: 

Case 1: I get a successful response if I use localhost or <server name> from within the server.

curl -k http://<servername>:8088/services/collector -H "Authorization: Splunk <token>" -d '{"event": "hello world again"}'
{"text":"Success","code":0}

Case 2: I get a 404 error when I use the <servername> in the command from outside the server

curl -k http://<servername>:8088/services/collector -H "Authorization: Splunk <token>" -d '{"event": "hello world again"}'
{"text":"The requested URL was not found on this server.","code":404}

I verified that the outside world can telnet the port 8088 on the server.  Also, the console opens on port 8000. 

Are there any changes I need to make on the server? Or any configs on the Splunk end?

[syedimranstonex · Accepted solution]

i found that /raw at the end makes it work
Here's the URL thats working now

curl -k -H "Authorization: Splunk <token>" http://<servername>:8088/services/collector/raw -d "testing"

[isoutamo]

Hi

it should work just like from local host. Can you add /event to the end of url?

One parameter which can case this is acceptFrom. It should allow it frame everywhere, but if you have changed it somewhere else it could affect here.

Just try splunk btool inputs list http —debug and look what is it’s value.

r. Ismo

[syedimranstonex]

Added /event  at the end of the URL .This also returns 404 from outside the server but works inside the server. 

I did not see the acceptFrom parameter in the output of splunk btool inputs list http —debug

[isoutamo]

Have you anything interesting on Splunk’s logs?

[syedimranstonex]

Nothing that helps on the splunk logs.  

[isoutamo]

404 means that item not found. Basically it means that your URL is not correct. Which client and client os you are using? Are it same than you have on server?

[syedimranstonex]

Well the Curl on the url Gives a successful response when tried on the server. And I see the hello world on splunk web console as well. 
The issue is when I try to curl from outside the server in the same network.

i am using splunk enterprise 8.0.3 running on red hat Linux 7.x 64-bit os 

[isoutamo]

Is the second node where you can’t connect also rhel? As in Windows cmd line there could be some issues with “.

[syedimranstonex]

No. It’s my windows machine where from where I use the splunk web console. I tried curl on a different Linux server in d same network, got 404 again. Basically http://<servername>:8088 doesn’t seem to be working anywhere outside the splunk server. 

[isoutamo]

Can you try curl -v ... from that another Linux server and paste it here?

[syedimranstonex]

I verified that the firewalld is not running on the server and there are no iptables entry on the server.