Wrap a webhook for use with HTTP Event Collector

[joxley]

I have an external system that generates a Webhook that can be posted to a URL of my choosing. I would like to log this event as is with a sourcetype of my choosing to an index of my choosing. I looked at the Splunk HTTP Event Collector, but it requires the event to be

{
  "event": { "webhook": "data" }
}

as well as having the Authorization header added onto it.

How can I do this?

[gblock_splunk]

@joxley today this using something like nginx is the right way to do it, but a better way is coming shortly, stay tuned. Also another option is to use this node app which gives your a proxy. In terms of the payload, our newer raw endpoint is perfect for receiving the standard webhook payload. As you observed the auth header is still required today, but that will change.

[himynamesdave]

You can also add HTTP webhooks really simply using HTTP Forwarder:

https://www.httpforwarder.com/

It will automatically reformat the event to fit Splunk's _JSON sourcetype and append correct Splunk headers.

[gblock_splunk]

@joxley if you are using Splunk Cloud you can use our new /services/collector/raw endpoint which allows sending arbitrary data formats. This will also be available soon in Splunk Enterprise.

If you are not in cloud today, then doing something like you did with nginx is a reasonable work around.

[joxley · Accepted solution]

The solution I used was to run Nginx in front of the universal forwarder to wrap the event data and add the header:

location /webhook-GUID {
    proxy_pass            https://localhost:8088/services/collector;
    proxy_read_timeout    90;
    proxy_connect_timeout 90;
    proxy_redirect        off;
    proxy_set_header      Host $host;
    proxy_set_header      X-Real-IP $remote_addr;
    proxy_set_header      X-Forwarded-For $proxy_add_x_forwarded_for;

    # wrap the webhook body for splunk
    proxy_set_body        "{\"event\":$request_body}";
    # Add the Splunk token into the Authorization header
    proxy_set_header      Authorization "Splunk HTTP-EC-TOKEN";
}

Wrap a webhook for delivery to an HTTP event collector is my blog post about getting webhook data from FogBugz into Splunk.

[japala]

when i click on the link you gave, it says page doesn't exist. ;(

[jtlittle]

page does not exist!