Splunk Search

How to get multiple dynamic values from a single log line

littlgra
Engager

We have numerous log lines that are in a format similar to the following:-

2019-04-09 13:00:03 DEBUG DynamicName1 1000 (1.00) ; DynamicName2 2000 (2.00) ; 
2019-04-09 13:00:02 DEBUG DynamicName2 500 (0.50) ; DynamicName4 3100 (3.10) ; DynamicName5 12000 (12.00) ;
2019-04-09 13:00:00 DEBUG DynamicName1 600 (0.60) ; DynamicName5 2100 (2.10) ;

The DynamicName# is a dynamic string that can have multiple values per line (but never the same value per line), the numbers after it represent a timing in milliseconds and then seconds.

What I want to get is a table of all the unique DynamicName(s), their average execution times and counts

However, I can't quite get the extraction correct. When I use a rex, for example

rex field=_raw "(?<name>\w+) (?<time>\d+) \(\d+.\d+\)"  | table name time

However this creates a table of multiple values per row and then I can't use other commands on it correctly. For example:-

rex field=_raw "(?<name>\w+) (?<time>\d+) \(\d+.\d+\) ; "  | table name time | sort -time

Does not result in the correct result I am expecting.

Is there a way I can correctly extract the data to get true dynamic multiple values that I can then table with 1 DynamicName per table row

1 Solution

grittonc
Contributor

Try splitting it up into a mv field after stripping out the first characters that aren't needed:

| eval foo=replace(_raw, "\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} DEBUG", "")
| makemv delim=";" foo
| mvexpand foo
| rex field=foo "(?<name>\w+) (?<time>\d+) \(\d+.\d+\)"
| table name time | sort -time

Then you can use mvexpand to split it up into multiple events and your regex can work on that.

View solution in original post

0 Karma

grittonc
Contributor

Try splitting it up into a mv field after stripping out the first characters that aren't needed:

| eval foo=replace(_raw, "\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} DEBUG", "")
| makemv delim=";" foo
| mvexpand foo
| rex field=foo "(?<name>\w+) (?<time>\d+) \(\d+.\d+\)"
| table name time | sort -time

Then you can use mvexpand to split it up into multiple events and your regex can work on that.

0 Karma

littlgra
Engager

Awesome that worked. I had played with the mv functions before but couldn't get it to work. Much appreciated

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...