topic Re: How to get multiple dynamic values from a single log line in Splunk Search

How to get multiple dynamic values from a single log line

littlgra — Tue, 09 Apr 2019 06:10:46 GMT

We have numerous log lines that are in a format similar to the following:-

2019-04-09 13:00:03 DEBUG DynamicName1 1000 (1.00) ; DynamicName2 2000 (2.00) ; 
2019-04-09 13:00:02 DEBUG DynamicName2 500 (0.50) ; DynamicName4 3100 (3.10) ; DynamicName5 12000 (12.00) ;
2019-04-09 13:00:00 DEBUG DynamicName1 600 (0.60) ; DynamicName5 2100 (2.10) ;

The DynamicName# is a dynamic string that can have multiple values per line (but never the same value per line), the numbers after it represent a timing in milliseconds and then seconds.

What I want to get is a table of all the unique DynamicName(s), their average execution times and counts

However, I can't quite get the extraction correct. When I use a rex, for example

rex field=_raw "(?<name>\w+) (?<time>\d+) \(\d+.\d+\)"  | table name time

However this creates a table of multiple values per row and then I can't use other commands on it correctly. For example:-

rex field=_raw "(?<name>\w+) (?<time>\d+) \(\d+.\d+\) ; "  | table name time | sort -time

Does not result in the correct result I am expecting.

Is there a way I can correctly extract the data to get true dynamic multiple values that I can then table with 1 DynamicName per table row

Re: How to get multiple dynamic values from a single log line

grittonc — Wed, 10 Apr 2019 14:56:54 GMT

Try splitting it up into a mv field after stripping out the first characters that aren't needed:

| eval foo=replace(_raw, "\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} DEBUG", "")
| makemv delim=";" foo
| mvexpand foo
| rex field=foo "(?<name>\w+) (?<time>\d+) \(\d+.\d+\)"
| table name time | sort -time

Then you can use mvexpand to split it up into multiple events and your regex can work on that.

Re: How to get multiple dynamic values from a single log line

littlgra — Thu, 11 Apr 2019 03:06:55 GMT

Awesome that worked. I had played with the mv functions before but couldn't get it to work. Much appreciated