HTTP Event Collector and curl: How to pass the hostname variable in Chef? (BOUNTY!)

[a212830]

Hi,

(Not Splunk questions per say...)

I'm setting up the HTTP Event Collector, so that our chef recipes can log to Splunk and we can have stats on it's usefulness and such...

I was able to communicate in the chef recipe to the HEC, using a basic curl command. Now, I want to pass the hostname of the server where I'm running, but having problems getting that HOSTNAME variable to appear. Wondering if any Linux guru's out there can help me out... OR.... help me map out how chef does an http_request to the Splunk parameters.

So, here's the curl command and the results:

curl -k https://myserver.com:8088/services/collector/event -H 'Authorization: Splunk 7E36213E-03E1-4246-93B9-81931D303A58' -d '{"event": "hello from ""'"$HOSTNAME"'""}'
{"text":"Invalid data format","code":6,"invalid-event-number":0}[a212830@vc2crtp1102248n ~]

If I just say "hello world" it works. But passing in the hostname variable throws a wrench into it.

That said, chef has an http_request function, but I'm not sure how to map the required Splunk parameters to it. Anyone tried it?

http_request 'posting data' do
  action :post
  url 'http://example.com/check_in'
  message ({:some => 'data'}.to_json)
  headers({'AUTHORIZATION' => "Basic #{
    Base64.encode64('username:password')}",
    'Content-Type' => 'application/data'
  })
end

[sogema]

Can I use splunk light cloud for this ? I am trying and it does not work. What URL should I use?

1st attempt
https://prd-p-xxxxxxx.cloud.splunk.com:8088/services/collector/event
result: time out

2nd attempt
https://input-prd-p-XXXXXXX.cloud.splunk.com:8088/services/collector/event
result: time out

3rd attempt
result: time out
https://http-inputs-XXXXXXXX.splunkcloud.com/services/collector/event

[gblock_splunk]

One way is to send "host" in the request as you are doing here. Alternatively you can configure Splunk per token, so it will resolve the host based on the client that is sending. You do this in inputs.conf under the token stanza by setting the connection_host to "ip" or "dns". You can see the setting here

connection_host = [ip|dns|none]
* Specify the host if an event doesn't have host set.
* "ip" sets the host to the IP address of the system sending the data. 
* "dns" sets the host to the reverse DNS entry for IP address of the system sending the data.
* "none" leaves the host as specified in the HTTP header.

[a212830]

fyi for those who have chef and want to do the same thing via http_request:

  http_request 'posting data' do
  action :post
  url "https://myserver.com:8088/services/collector/event"
  message ({:event => "splunk installer complete.  type=#{type}", :host => "#{HOSTNAME} ", :index => "main"}.to_json)
  headers({
    'Authorization' => 'Splunk 7E36213E-03E1-4246-93B9-81931D303A58'
  })
end

[gblock_splunk]

Just to clarify, the config setting above will allow you to configure this on the Splunk side so the client doesn't have to send it.

[scottjpack · Accepted solution]

Definitely more of a curl issue than a HEC issue, but I was able to get it to work right with the following:

curl -k https://<splunk-server>:8088/services/collector/event -H 'Authorization: Splunk <HEC Token>' -d"{\"event\": \"hello $HOSTNAME\", \"index\":\"<Index Name>\", \"host\":\"$HOSTNAME\"}"

[a212830]

Agreed. Not a HEC issue.

Same error though:

[a212830@vc2crtp1102248n ~]$ echo $HOSTNAME
vc2crtp1102248n.fmr.com
[a212830@vc2crtp1102248n ~]$ curl -k https://myserver.com:8088/services/collector/event -H 'Authorization: Splunk 7E36213E-03E1-4246-93B9-81931D303A58' -d"{\"event\": \"hello $HOSTNAME\", \"index\":\"<Index Name>\", \"host\":\"$HOSTNAME\"}"
{"text":"Incorrect index","code":7,"invalid-event-number":1}[a212830@vc2crtp1102248n ~]$

[marco_gomiero]

it worked very fine for me, tnx!

[pinterl]

Do not use the backslashes (). The command works fine without them.

[a212830]

Bingo! I accepted the answer. Do you automatically get the points?

[scottjpack]

Looks like they were awarded. Thanks!

[scottjpack]

Try specifying an index that the HEC has access to write to, I get weird stuff if no index is specified.