topic Wrap a webhook for use with HTTP Event Collector in Getting Data In

Wrap a webhook for use with HTTP Event Collector

joxley — Fri, 22 Jan 2016 14:45:19 GMT

I have an external system that generates a Webhook that can be posted to a URL of my choosing. I would like to log this event as is with a sourcetype of my choosing to an index of my choosing. I looked at the Splunk HTTP Event Collector, but it requires the event to be

{
  "event": { "webhook": "data" }
}

as well as having the Authorization header added onto it.

How can I do this?

Re: Wrap a webhook for use with HTTP Event Collector

joxley — Fri, 22 Jan 2016 14:45:42 GMT

The solution I used was to run Nginx in front of the universal forwarder to wrap the event data and add the header:

location /webhook-GUID {
    proxy_pass            https://localhost:8088/services/collector;
    proxy_read_timeout    90;
    proxy_connect_timeout 90;
    proxy_redirect        off;
    proxy_set_header      Host $host;
    proxy_set_header      X-Real-IP $remote_addr;
    proxy_set_header      X-Forwarded-For $proxy_add_x_forwarded_for;

    # wrap the webhook body for splunk
    proxy_set_body        "{\"event\":$request_body}";
    # Add the Splunk token into the Authorization header
    proxy_set_header      Authorization "Splunk HTTP-EC-TOKEN";
}

Wrap a webhook for delivery to an HTTP event collector is my blog post about getting webhook data from FogBugz into Splunk.

Re: Wrap a webhook for use with HTTP Event Collector

gblock_splunk — Sun, 14 Feb 2016 04:57:10 GMT

@joxley if you are using Splunk Cloud you can use our new /services/collector/raw endpoint which allows sending arbitrary data formats. This will also be available soon in Splunk Enterprise.

If you are not in cloud today, then doing something like you did with nginx is a reasonable work around.

Re: Wrap a webhook for use with HTTP Event Collector

himynamesdave — Fri, 09 Sep 2016 20:26:47 GMT

You can also add HTTP webhooks really simply using HTTP Forwarder:

https://www.httpforwarder.com/

It will automatically reformat the event to fit Splunk's _JSON sourcetype and append correct Splunk headers.

Re: Wrap a webhook for use with HTTP Event Collector

gblock_splunk — Sat, 10 Sep 2016 05:38:58 GMT

@joxley today this using something like nginx is the right way to do it, but a better way is coming shortly, stay tuned. Also another option is to use this node app which gives your a proxy. In terms of the payload, our newer raw endpoint is perfect for receiving the standard webhook payload. As you observed the auth header is still required today, but that will change.

Re: Wrap a webhook for use with HTTP Event Collector

japala — Tue, 27 Sep 2016 18:29:31 GMT

when i click on the link you gave, it says page doesn't exist. ;(

Re: Wrap a webhook for use with HTTP Event Collector

jtlittle — Wed, 16 May 2018 23:15:57 GMT

page does not exist!