topic Re: How to view raw events in Stream?? in Getting Data In

How to view raw events in Stream??

tjago11 — Wed, 30 Sep 2020 02:05:57 GMT

Getting a Splunk Stream feed from a Gigamon tap for HTTP.

I can see all the default fields to pull from the HTTP stream, including the one I care about "src_headers". However, I can't log the full src_headers because it potentially contains authentication tokens and other stuff I don't want to log. I have been able to extract a couple fields successfully by guessing at the regex, but been thrashing on the rest.

Is there some way to view the data that is coming to Stream without being forced to log all of it?? Something like a preview??

Would be nice to see a sample event before logging everything. Thanks.

-Tyler

Re: How to view raw events in Stream??

jacobpevans — Mon, 09 Sep 2019 21:41:13 GMT

Please provide example data, what you want the data to look like, and the regex you've constructed so far. Be sure to censor any sensitive data.

Re: How to view raw events in Stream??

tjago11 — Wed, 30 Sep 2020 02:07:18 GMT

That's the problem, I don't have sample data. 😉

My data is getting fed from a Gigamon tap at the LTM level. I don't have visibility into the raw tap data and I don't want to log it because it has authentication tokens and other scary stuff.

Working with the OTB HTTP Stream Protocol:
https://docs.splunk.com/Documentation/StreamApp/7.1.3/DeployStreamApp/FileTransfer#HTTP

The field that I want to build an extraction for is src_headers:
src_headers All HTTP headers sent from client to server http.cs-headers

I was able to extract a known header element that is added by an application team using this:
x-parentspantoken[\s:=]+(.{36})

The one I'm struggling with is the x-forwarded-for that is added by our F5 devices. I have this regex:
x-forwarded-for([:\s]+[\d.:\s,]+)

Which is just a guess because I can't see the actual raw header to know if that is right. Possibly a casing issue?? Tried adding (?i) to the front and it gave me an error:
Invalid regex Expression

Poop.