topic Re: HTTP Event Collector and curl: How to pass the hostname variable in Chef? (BOUNTY!) in Getting Data In

HTTP Event Collector and curl: How to pass the hostname variable in Chef? (BOUNTY!)

a212830 — Tue, 30 Aug 2016 15:12:02 GMT

Hi,

(Not Splunk questions per say...)

I'm setting up the HTTP Event Collector, so that our chef recipes can log to Splunk and we can have stats on it's usefulness and such...

I was able to communicate in the chef recipe to the HEC, using a basic curl command. Now, I want to pass the hostname of the server where I'm running, but having problems getting that HOSTNAME variable to appear. Wondering if any Linux guru's out there can help me out... OR.... help me map out how chef does an http_request to the Splunk parameters.

So, here's the curl command and the results:

curl -k https://myserver.com:8088/services/collector/event -H 'Authorization: Splunk 7E36213E-03E1-4246-93B9-81931D303A58' -d '{"event": "hello from ""'"$HOSTNAME"'""}'
{"text":"Invalid data format","code":6,"invalid-event-number":0}[a212830@vc2crtp1102248n ~]

If I just say "hello world" it works. But passing in the hostname variable throws a wrench into it.

That said, chef has an http_request function, but I'm not sure how to map the required Splunk parameters to it. Anyone tried it?

http_request 'posting data' do
  action :post
  url 'http://example.com/check_in'
  message ({:some => 'data'}.to_json)
  headers({'AUTHORIZATION' => "Basic #{
    Base64.encode64('username:password')}",
    'Content-Type' => 'application/data'
  })
end

Re: HTTP Event Collector and curl: How to pass the hostname variable in Chef? (BOUNTY!)

scottjpack — Tue, 30 Aug 2016 17:26:19 GMT

Definitely more of a curl issue than a HEC issue, but I was able to get it to work right with the following:

curl -k https://<splunk-server>:8088/services/collector/event -H 'Authorization: Splunk <HEC Token>' -d"{\"event\": \"hello $HOSTNAME\", \"index\":\"<Index Name>\", \"host\":\"$HOSTNAME\"}"

Re: HTTP Event Collector and curl: How to pass the hostname variable in Chef? (BOUNTY!)

a212830 — Tue, 30 Aug 2016 17:40:58 GMT

Agreed. Not a HEC issue.

Same error though:

[a212830@vc2crtp1102248n ~]$ echo $HOSTNAME
vc2crtp1102248n.fmr.com
[a212830@vc2crtp1102248n ~]$ curl -k https://myserver.com:8088/services/collector/event -H 'Authorization: Splunk 7E36213E-03E1-4246-93B9-81931D303A58' -d"{\"event\": \"hello $HOSTNAME\", \"index\":\"<Index Name>\", \"host\":\"$HOSTNAME\"}"
{"text":"Incorrect index","code":7,"invalid-event-number":1}[a212830@vc2crtp1102248n ~]$

Re: HTTP Event Collector and curl: How to pass the hostname variable in Chef? (BOUNTY!)

scottjpack — Tue, 30 Aug 2016 17:58:35 GMT

Try specifying an index that the HEC has access to write to, I get weird stuff if no index is specified.

Re: HTTP Event Collector and curl: How to pass the hostname variable in Chef? (BOUNTY!)

a212830 — Tue, 30 Aug 2016 18:33:34 GMT

Bingo! I accepted the answer. Do you automatically get the points?

Re: HTTP Event Collector and curl: How to pass the hostname variable in Chef? (BOUNTY!)

gblock_splunk — Tue, 30 Aug 2016 18:56:27 GMT

One way is to send "host" in the request as you are doing here. Alternatively you can configure Splunk per token, so it will resolve the host based on the client that is sending. You do this in inputs.conf under the token stanza by setting the connection_host to "ip" or "dns". You can see the setting here

connection_host = [ip|dns|none]
* Specify the host if an event doesn't have host set.
* "ip" sets the host to the IP address of the system sending the data. 
* "dns" sets the host to the reverse DNS entry for IP address of the system sending the data.
* "none" leaves the host as specified in the HTTP header.

Re: HTTP Event Collector and curl: How to pass the hostname variable in Chef? (BOUNTY!)

scottjpack — Tue, 30 Aug 2016 19:07:07 GMT

Looks like they were awarded. Thanks!

Re: HTTP Event Collector and curl: How to pass the hostname variable in Chef? (BOUNTY!)

a212830 — Wed, 31 Aug 2016 01:14:44 GMT

fyi for those who have chef and want to do the same thing via http_request:

  http_request 'posting data' do
  action :post
  url "https://myserver.com:8088/services/collector/event"
  message ({:event => "splunk installer complete.  type=#{type}", :host => "#{HOSTNAME} ", :index => "main"}.to_json)
  headers({
    'Authorization' => 'Splunk 7E36213E-03E1-4246-93B9-81931D303A58'
  })
end

Re: HTTP Event Collector and curl: How to pass the hostname variable in Chef? (BOUNTY!)

gblock_splunk — Wed, 31 Aug 2016 01:20:22 GMT

Just to clarify, the config setting above will allow you to configure this on the Splunk side so the client doesn't have to send it.

Re: HTTP Event Collector and curl: How to pass the hostname variable in Chef? (BOUNTY!)

sogema — Wed, 13 Sep 2017 22:14:22 GMT

Can I use splunk light cloud for this ? I am trying and it does not work. What URL should I use?

1st attempt
https://prd-p-xxxxxxx.cloud.splunk.com:8088/services/collector/event
result: time out

2nd attempt
https://input-prd-p-XXXXXXX.cloud.splunk.com:8088/services/collector/event
result: time out

3rd attempt
result: time out
https://http-inputs-XXXXXXXX.splunkcloud.com/services/collector/event

Re: HTTP Event Collector and curl: How to pass the hostname variable in Chef? (BOUNTY!)

pinterl — Tue, 23 Jan 2018 01:05:31 GMT

Do not use the backslashes (). The command works fine without them.

Re: HTTP Event Collector and curl: How to pass the hostname variable in Chef? (BOUNTY!)

marco_gomiero — Thu, 13 Feb 2020 11:19:15 GMT

it worked very fine for me, tnx!