Getting Data In

Wrap a webhook for use with HTTP Event Collector

joxley
Path Finder

I have an external system that generates a Webhook that can be posted to a URL of my choosing. I would like to log this event as is with a sourcetype of my choosing to an index of my choosing. I looked at the Splunk HTTP Event Collector, but it requires the event to be

{
  "event": { "webhook": "data" }
}

as well as having the Authorization header added onto it.

How can I do this?

1 Solution

joxley
Path Finder

The solution I used was to run Nginx in front of the universal forwarder to wrap the event data and add the header:

location /webhook-GUID {
    proxy_pass            https://localhost:8088/services/collector;
    proxy_read_timeout    90;
    proxy_connect_timeout 90;
    proxy_redirect        off;
    proxy_set_header      Host $host;
    proxy_set_header      X-Real-IP $remote_addr;
    proxy_set_header      X-Forwarded-For $proxy_add_x_forwarded_for;

    # wrap the webhook body for splunk
    proxy_set_body        "{\"event\":$request_body}";
    # Add the Splunk token into the Authorization header
    proxy_set_header      Authorization "Splunk HTTP-EC-TOKEN";
}

Wrap a webhook for delivery to an HTTP event collector is my blog post about getting webhook data from FogBugz into Splunk.

View solution in original post

0 Karma

gblock_splunk
Splunk Employee
Splunk Employee

@joxley today this using something like nginx is the right way to do it, but a better way is coming shortly, stay tuned. Also another option is to use this node app which gives your a proxy. In terms of the payload, our newer raw endpoint is perfect for receiving the standard webhook payload. As you observed the auth header is still required today, but that will change.

0 Karma

himynamesdave
Contributor

You can also add HTTP webhooks really simply using HTTP Forwarder:

https://www.httpforwarder.com/

It will automatically reformat the event to fit Splunk's _JSON sourcetype and append correct Splunk headers.

0 Karma

gblock_splunk
Splunk Employee
Splunk Employee

@joxley if you are using Splunk Cloud you can use our new /services/collector/raw endpoint which allows sending arbitrary data formats. This will also be available soon in Splunk Enterprise.

If you are not in cloud today, then doing something like you did with nginx is a reasonable work around.

0 Karma

joxley
Path Finder

The solution I used was to run Nginx in front of the universal forwarder to wrap the event data and add the header:

location /webhook-GUID {
    proxy_pass            https://localhost:8088/services/collector;
    proxy_read_timeout    90;
    proxy_connect_timeout 90;
    proxy_redirect        off;
    proxy_set_header      Host $host;
    proxy_set_header      X-Real-IP $remote_addr;
    proxy_set_header      X-Forwarded-For $proxy_add_x_forwarded_for;

    # wrap the webhook body for splunk
    proxy_set_body        "{\"event\":$request_body}";
    # Add the Splunk token into the Authorization header
    proxy_set_header      Authorization "Splunk HTTP-EC-TOKEN";
}

Wrap a webhook for delivery to an HTTP event collector is my blog post about getting webhook data from FogBugz into Splunk.

0 Karma

japala
Path Finder

when i click on the link you gave, it says page doesn't exist. ;(

0 Karma

jtlittle
Path Finder

page does not exist!

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...