Splunk Search

How to parse the payload inside a csv log to run a stats count search on the extracted fields?

anthonycopus
Path Finder

Hi,

What would be the simplest way of parsing the following logs so I can search what is inside the {} field:

"Feb 25, 2015 11:59 PM",20,PreviewScreen_Next,,201502051926,iPhone,Apple iPhone 6,51FC9233-F23B-44FA-B3CC-E91F56F26C30,{ field1 : Original;  media : MediaType;  post : mu},

The name of the field in the csv is "Params". All the other fields are fine as I can search them, but I need to be able to do a stats count by media for example.

Thanks

0 Karma

emiller42
Motivator

The content of that field is ALMOST json, which would make it super easy to parse. If it were:

{"field1": "Original", "media": "MediaType", "post": "mu"}

You could use the spath command to parse it.

| spath input=Params

But it isn't, so that's not useful. However, you should be able to use extract without having to use a complex regex.

... | extract pairdelim=";{}" kvdelim=":" 

I tested it with the following:

index=* | head 1 | eval _raw="{ field1 : Original;  media : MediaType;  post : mu}"  | extract pairdelim=";{}" kvdelim=":" | table field1 media post

which outputs:

field1 media post
Original MediaType mu

This will be flexible to any kvpairs that happen to show up in the Params field. (Note it'll get weird if the values themselves contain {}:; as that'll be what it's looking for to separate kvpairs)

0 Karma

MuS
SplunkTrust
SplunkTrust

Hi anthonycopus,

try something like this:

your base search here | rex "field1\s:\s(?<field1>.+?);\s+media\s:\s(?<media>.+?);\s+post\s:\s(?<post>.+?)}" | stats count by media

Hope that helps ...

cheers, MuS

anthonycopus
Path Finder

Thanks, this works. Is there a combination which would work if the logs follow this trend, but some may have more fields than others? E.g. one log has field1 and media in the payload, but the other has only field1

0 Karma

MuS
SplunkTrust
SplunkTrust

You could setup field extraction for each field separately, take a look at the docs here http://docs.splunk.com/Documentation/Splunk/6.2.2/Knowledge/ExtractfieldsinteractivelywithIFX

0 Karma
Get Updates on the Splunk Community!

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...

Reminder! Splunk Love Promo: $25 Visa Gift Card for Your Honest SOAR Review With ...

We recently launched our first Splunk Love Special, and it's gone phenomenally well, so we're doing it again, ...