Getting Data In

Wrap a webhook for use with HTTP Event Collector

joxley
Path Finder

I have an external system that generates a Webhook that can be posted to a URL of my choosing. I would like to log this event as is with a sourcetype of my choosing to an index of my choosing. I looked at the Splunk HTTP Event Collector, but it requires the event to be

{
  "event": { "webhook": "data" }
}

as well as having the Authorization header added onto it.

How can I do this?

1 Solution

joxley
Path Finder

The solution I used was to run Nginx in front of the universal forwarder to wrap the event data and add the header:

location /webhook-GUID {
    proxy_pass            https://localhost:8088/services/collector;
    proxy_read_timeout    90;
    proxy_connect_timeout 90;
    proxy_redirect        off;
    proxy_set_header      Host $host;
    proxy_set_header      X-Real-IP $remote_addr;
    proxy_set_header      X-Forwarded-For $proxy_add_x_forwarded_for;

    # wrap the webhook body for splunk
    proxy_set_body        "{\"event\":$request_body}";
    # Add the Splunk token into the Authorization header
    proxy_set_header      Authorization "Splunk HTTP-EC-TOKEN";
}

Wrap a webhook for delivery to an HTTP event collector is my blog post about getting webhook data from FogBugz into Splunk.

View solution in original post

0 Karma

gblock_splunk
Splunk Employee
Splunk Employee

@joxley today this using something like nginx is the right way to do it, but a better way is coming shortly, stay tuned. Also another option is to use this node app which gives your a proxy. In terms of the payload, our newer raw endpoint is perfect for receiving the standard webhook payload. As you observed the auth header is still required today, but that will change.

0 Karma

himynamesdave
Contributor

You can also add HTTP webhooks really simply using HTTP Forwarder:

https://www.httpforwarder.com/

It will automatically reformat the event to fit Splunk's _JSON sourcetype and append correct Splunk headers.

0 Karma

gblock_splunk
Splunk Employee
Splunk Employee

@joxley if you are using Splunk Cloud you can use our new /services/collector/raw endpoint which allows sending arbitrary data formats. This will also be available soon in Splunk Enterprise.

If you are not in cloud today, then doing something like you did with nginx is a reasonable work around.

0 Karma

joxley
Path Finder

The solution I used was to run Nginx in front of the universal forwarder to wrap the event data and add the header:

location /webhook-GUID {
    proxy_pass            https://localhost:8088/services/collector;
    proxy_read_timeout    90;
    proxy_connect_timeout 90;
    proxy_redirect        off;
    proxy_set_header      Host $host;
    proxy_set_header      X-Real-IP $remote_addr;
    proxy_set_header      X-Forwarded-For $proxy_add_x_forwarded_for;

    # wrap the webhook body for splunk
    proxy_set_body        "{\"event\":$request_body}";
    # Add the Splunk token into the Authorization header
    proxy_set_header      Authorization "Splunk HTTP-EC-TOKEN";
}

Wrap a webhook for delivery to an HTTP event collector is my blog post about getting webhook data from FogBugz into Splunk.

0 Karma

japala
Path Finder

when i click on the link you gave, it says page doesn't exist. ;(

0 Karma

jtlittle
Path Finder

page does not exist!

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...

Stay Connected: Your Guide to April Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars in April. This post ...