HTTP Event collector - call not properly authenticated

[sdkp03]

Have tried to setup HTTPEventCollector via cli using splunk documentation link: https://docs.splunk.com/Documentation/Splunk/8.0.3/Data/UseHECfromtheCLI

Commands i have executed are as below:

/opt/splunk/bin/splunk http-event-collector create sdapp01 -uri https://localhost:8089 -description "this is a new token" -disabled 1

/opt/splunk/bin/splunk http-event-collector enable -name sdapp01 -uri https://localhost:8089 -auth admin:changeme

curl -k -u admin:changeme https://localhost:8089/servicesNS/admin/splunk_httpinput/data/inputs/http

splunk http-event-collector send -uri https://localhost:8089 -token 206f9ca0-24bd-48fd-95e8-dfdcaa17657a {"this is some data"}

curl -k https://localhost:8089/services/collector -H 'Authorization: Splunk 206f9ca0-24bd-48fd-95e8-dfdcaa17657a' -d '{"sourcetype": "demo", "event":"Hello, world!"}'

while sending data am getting error as below:

    <?xml version="1.0" encoding="UTF-8"?>
    <response>
      <messages>
        <msg type="WARN">call not properly authenticated</msg>
      </messages>
    </response>

Config details are as mentioned below:

local/inputs.conf
[http://sdapp01]
disabled = 0
token = 206f9ca0-24bd-48fd-95e8-dfdcaa17657a

default/inputs.conf
[http]
disabled=1
port=8088
enableSSL=1
dedicatedIoThreads=2
maxThreads = 0
maxSockets = 0
useDeploymentServer=0
# ssl settings are similar to mgmt server
sslVersions=*,-ssl2
allowSslCompression=true
allowSslRenegotiation=true

Not sure what have i missed. Token is enabled, not expired. Have tried creating multiple tokens but stuck with same issue. Can someone please help.

[tauliang]

Here is my take:

• It might be a red herring, but are you sure the credentials are accepted at CLI? In Splunk 8.0.x, if you use the default admin credentials, it would ask the user to change the password from changeme to something else before it allows the user to do anything else.

  $ ./splunk http-event-collector create sdapp01 -uri https://localhost:8089 -description "this is a new token" -disabled 1
  Splunk username: admin
  Password: 
  The administrator requires you to change your password.
  Please enter a new password: 
  

• Also, can you go to Splunk web and make sure that this HEC token is indeed shown asenabled there? If not, click on Global Settings and make sure that Enabled is set there.

• IF STILL TO NO VAIL, TRY THIS, this is the nuke of all CLI commands arsenal for HEC, this is the ULTIMATE ULTIMATE WEAPON

  curl -k https://127.0.0.1:8088/services/collector/event -H "Authorization: Splunk [your token]" -d '{"event": "Dodge this!"}'

And the end of all these, you should be able to get a response saying

{"text":"Success","code":0}

Then go to Splunk Web to do a search:

source=http:sdapp01

You should be able to see the event there. Good luck!

[sdkp03]

Have verified from web console, and can confirm that the token is enabled. I did try executing ULTIMATE WEAPON command. still no luck. have pasted output here for your reference:
[splunk@## ~]$ curl -k https://127.0.0.1:8088/services/collector/event -H "Authorization: Splunk 206f9ca0-24bd-48fd-95e8-dfdcaa17657a" -d '{"event": "Dodge this!"}'
curl: (7) couldn't connect to host
[splunk@## ~]$ curl -k https://127.0.0.1:8089/services/collector/event -H "Authorization: Splunk 206f9ca0-24bd-48fd-95e8-dfdcaa17657a" -d '{"event": "Dodge this!"}'

<msg type="WARN">call not properly authenticated</msg>

For some strange reason 8088 doesnt work for me as am always prompted with error "host not found". When i use port 8089, i end up with error - call not authenticated.

[tauliang]

This is strange indeed. Did you see any other errors in _internal index?
Also, if you do a port scan of local ports, what do you see? Is port 8088 open?

Port Scan has started…

Port Scanning host: 127.0.0.1

     Open TCP Port:     8000        irdmi
     Open TCP Port:     8065
     Open TCP Port:     8088        radan-http
     Open TCP Port:     8089
     Open TCP Port:     8191
Port Scan has completed…

[masonmorales · Accepted solution]

Change disabled=1 to disabled=0, restart splunk. Then, change your curl command to port 8088 not 8089 and try again.

[sdkp03]

Had to add an extra stanza in local/inputs.conf
[http]
disabled=1

Thats it, and it works like magic.

[sdkp03]

in local/inputs.conf, the disabled field is set to 0. I have restarted splunk services multiple times, dont see any change. I have tried accessing port 8088, and the output was - couldn't connect to host!

[tauliang]

Did you restart Splunk after enabling HEC?

[sdkp03]

Yes i did restart multiple times but with no luck